Alternate data streams
Subversive Technologies & Countermeasures
| |||||||||||
Many forms of storing or embedding information associated with file systems have been in existence for quite some time. Macintosh MFS, HFS, and HFS Plus use a system known as resource forks, OS/2 HPFS uses a system known as Extended attributes, and Windows NTFS uses alternate data streams. Implementation and design of these methods varies, but the core ideology remains the same across them all; store or embed extended data associated with entries in a file system.
Contents |
[edit] NTFS volumes
Even though alternate data streams have not been widely adopted in practice they have been available since Windows NT 3.1 (1993), but Microsoft makes no guaranteed that they will be supported in any newer Windows file systems. Each individual stream is unique, and there may be multiple streams associated with a single file or directory. Streams can be used to store any type of information that a normal file in a NTFS volume may store. Non NTFS Windows supported file systems such as FAT allow access to only one unnamed stream of data that is perceived as the actual contents of the file. If a file or directory with alternate stream data is moved to a non NTFS volume the stream data and other non supported attributes will be unrecoverable and lost forever. Windows Operating Systems currently only support alternate data streams on NTFS volumes but they do not provide any means for disabling them.
Many of the shared components of alternate data streams are defined by the unnamed default stream. Some of these are:
- Extended attributes
- Security descriptors
- Timestamps
- File modified times will reflect the creation of streams as well as changes in the unnamed default stream.
- Access time will only display the default unnamed stream access.
Other components that are unique to each stream are:
- Opportunistic locks
- Allocation size
- Actual size
- Valid data length
- Sharing modes
- Compression
- Encryption
- Sparseness
When creating or viewing streams only the file system of the volume that contains the entry needs to support streams. You will still be able to access a named stream on a shared folder on a NTFS volume from a system that does not have native support for streams.
[edit] Creating streams
Creating a new file on a NTFS volume without any stream specifications will default the data to the unnamed stream. This unnamed default stream is defined as a 0 length string or NULL. Every other stream associated with the entry will be considered an alternate stream and must have a string value name with a length greater then zero. According to Microsoft's file streams documentation, stream names can be created using standard file naming conventions. These file naming conventions make no reference to * or ? as illegal characters but attempting to create a file from a command shell or Windows Explorer in Windows XP SP2 will throw an invalid character error.
Attempt to create a file name with invalid characters:
echo bad file name > bad*file?name.txt
Creating a stream with these invalid file name conventions does not produce the same error and will in fact create a stream containing invalid characters not possible in the default stream name.
Creating a stream name with invalid characters:
echo bad file name > badname.txt:bad*file?name.txt
The affects of this inconsistency on avoiding detection, causing errors, or being used to exploit the system are unknown.
If the entry is a directory the default unnamed stream will be used to store the contents of the directory and therefore is not able to be used in the same manner as other streams. A user will only need write access to a file or directory in order to create a stream associated with that entry. The syntax provided by Microsoft for stream manipulation is filename:stream name:stream type. Any API or program that is stream aware will more then likely use the same syntax in its interface to provide access or control of the associated stream. APIs such as CreateFile(), ReadFile() , and WriteFile() support the transparent use of streams.
Below are some simple programs present in a Windows installation that a user with the proper permissions and access to a command shell (cmd.exe) can use to create streams.
Creating an alternate data stream with echo:
echo This is text data stored in ADSFile.txt:alternate_stream.txt > ADSFile.txt:alternate_stream.txt
- echo will create the unnamed default data stream (ADSFile.txt::$DATA) by default when creating a named stream on a non existent file.
Creating an alternate data stream for a directory with echo:
echo This is text data stored in ADSDirectory:alternate_stream.txt > ADSDirectory:alternate_stream.txt
- For the above example ADSDirectory must be already exist, it must be in the current directory or a path must be prefixed to the ADSDirectory:alternate_stream.txt portion of the example.
Attaching files as streams can be done with the type command. Below the type command is used to attach the notepad executable to a directory and the file ADSFilet.txt. Other easy examples for an executable could be calc.exe, sol.exe, cmd.exe, etc.
Attaching executable binary data to a directory stream:
type notepad.exe > %systemroot%:notepad.exe
Attaching executable binary data to a file stream:
type notepad.exe > ADSFile.txt:notepad.exe
- You will need to include the path to notepad.exe or copy it into the directory containing ADSFile.tx
Scripting languages such as VBScript can be used to create script files that can be attached and run from alternate data streams.
Create a simple script wsh.vbs to attach as a stream:
echo MsgBox "Hello world",85,"WSH Embedded stream example" > wsh.vbs
Attaching wsh.vbs script to a file stream:
type wsh.vbs > ADSFile.txt:wsh.vbs
Attaching wsh.vbs script to a directory stream:
type wsh.vbs > %systemroot%:wsh.vbs
File name extensions on a stream like any other file are not validated at the time of creation making it possible to give it any extension.
Attaching a script with an incorrect extensions to a file stream:
type wsh.vbs > ADSFile.txt:wsh.xyz
Most if not all Windows APIs are stream aware giving programmers a semi transparent ability to make their programs stream aware. Stream awareness in a program is not a given because APIs have support for them, it takes a conscious effort and understanding of streams to make use of them in an interface or algorithm. Below is a small source fragment from Microsoft demonstrating the syntax used for creating streams using their API CreateFile().
Creating alternate data stream c source example:
#include <windows.h>
#include <stdio.h>
void main( )
{
HANDLE hFile, hStream;
DWORD dwRet;
hFile = CreateFile( "testfile",
GENERIC_WRITE,
FILE_SHARE_WRITE,
NULL,
OPEN_ALWAYS,
0,
NUL );
if( hFile == INVALID_HANDLE_VALUE )
printf( "Cannot open testfile\n" );
else
WriteFile( hFile, "This is testfile", 16, &dwRet, NULL );
hStream = CreateFile( "testfile:stream",
GENERIC_WRITE,
FILE_SHARE_WRITE,
NULL,
OPEN_ALWAYS,
0,
NULL );
if( hStream == INVALID_HANDLE_VALUE )
printf( "Cannot open testfile:stream\n" );
else
WriteFile(hStream, "This is testfile:stream", 23, &dwRet, NULL);
}
C source example from MSDN
[edit] Viewing streams
Windows installations do not contain tools with the necessary abilities to locate or display detailed information about streams associated with files or directories. Although programs such as more and notepad can be used to view the :$DATA portion of an attached stream, the properties page from Windows explorer or a dir listing from a command shell will not reveal any information about associated streams. The reported file size and other attributes will only be that of the default unnamed stream, any other streams will be ignored. Windows Explorer extensions such as StrmExt.dll included in NTFSext.exe can be used to add a Stream tab to the properties page in Windows Explorer.
Windows Explorer view of ADSFile.txt stream tab using the StrmExt.dll extension.
To register StrmExt.dll perform the following steps provided below:
- Move StrmExt.dll to %SystemRoot%\System32
- Open a command shell or click Start --> Run and enter the following:
- regsvr32 StrmExt.dll
- To provide a stream tab for folder and drive properties you must add two new registry keys.
- HKEY_CLASSES_ROOT\Directory\shellex\PropertySheetHandlers\{C3ED1679-814B-4DA9-AB00-1CAC71F5E337}
- HKEY_CLASSES_ROOT\Drive\shellex\PropertySheetHandlers\{C3ED1679-814B-4DA9-AB00-1CAC71F5E337}
- To enable the "Edit Streams..." and "Create Hard Link..." buttons from the tab you must copy RWStream.vbs and HardLinks.vbs into %SystemRoot%\System32 directory.
Windows installations have a few tools available that allow a user with the correct privileges the ability to view a streams $DATA member. Below are a few examples of how to use more, notepad, and wordpad with streams from a command line.
Reading a known alternate data stream from a file:
more < ADSFile.txt:alternate_stream.txt
Reading a known alternate data stream from a file specifying stream type:
more < ADSFile.txt:alternate_stream.txt:$DATA
Two examples of reading a files default unnamed data stream:
more < ADSFile.txt::$DATA more < ADSFile.txt
Using notepad.exe from a command shell to view a named stream:
notepad.exe ADSFile.txt:alternate_stream.txt
If using notepad.exe to view streams without an extension do not forget to add the period ('.') to the end of the statement.
Using wordpad.exe from a command shell to view a named stream:
wordpad.exe ADSFile.txt:alternate_stream.txt
wordpad.exe is not usually include in the PATH and its default location is \Program Files\Windows NT\Accessories\wordpad.exe
Reading a known alternate data stream from a directory:
more < ADSDirectory:alternate_stream.txt
[edit] Searching for streams
Although Windows installations do not provide any tools for finding streams in a NTFS volume a few third party developers have made some simple utilities that can assist in locating and manipulating streams.
- SysInternals has a free Streams program that will enumerate streams. (Contains source code)
- LADS is also another free utility that enumerates streams.
- ADS Spy a nice free GUI tool for searching and removing streams.
- ADS Locator a free GUI tool that searches for streams.
[edit] Executing streams
Executable files can be executed using the start command from the command shell. The steps below assume that you have created the needed files and streams as discussed in the Creating Streams section. Alternate data streams that contain executable code will not be run simply by accessing the default stream, alternate data stream code must be called directly.
Executing a binary data stream from a file:
start ./ADSFile.txt:notepad.exe
Executing a binary data stream from a directory:
start %systemroot%:notepad.exe
You can use regedit to add your executable stream to the Windows startup process.
Executing from the registry:
1. Click Start --> Run and enter regedit 2. Navigate to the key HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Run 3. Create a REG_SZ (string value) 3. Right click on the new entry and select Modify 4. In the Value Data: box enter (PATH TO FILE)\ADSFile.txt:notepad.exe
Using wscript to run scripts from streams.
Executing WSH from a stream:
wscript ADSFile.txt:wsh.vbs
Forced execution of a script from a stream with an incorrect extension:
wscript //E:vbs ADSFile.txt:wsh.bob
- //E:engine switch explicitly defines which engine to use when executing the script.
[edit] Removing streams
Windows installations do not provide any specific tools for removing streams directly but by leveraging the lack of stream support in some tools and file systems it can be accomplished. The simplest method available for removing ALL alternate data streams from a file or directory is to move the entry to a non stream supported file system. ISO 9660, FAT, ext2, all will not support alternate data streams so moving a file or directory to any of these and back will strip all but the unnamed default stream. Windows will throw a warning box about stream data loss when copying a file or directory containing alternate streams to a non NTFS volume through explorer. The command shell on the other hand is not as forgiving and will copy only the default unnamed stream data without this warning and confirmation. Most if not all Internet protocols do not support alternate data streams so transferring the file via email, ftp, etc will also strip all but the unnamed default stream as well. The Streams program is a command line tool that will remove ALL alternate data streams from a file or directory. The StrmExt.dll extension provides a more granular approach with a Delete button from its tabbed properties page that enables a user to delete individual streams from the selected file or directory. ADSspy is a nice GUI tool for the shell challenged that provides a mix of search and removal options as well.
Using notepad to remove streams:
1. notepad.exe ADSFILE.TXT:alternate_stream.txt 2. Click on Edit --> Select ALL 3. Backspace 4. Click File --> Exit 5. Click Yes to save changes
- This will only delete the data contained within the stream, the stream will still exist as an empty container.
Command shell example of removing all streams from a file:
1. rename ADSfile.txt ADSFile.old 2. type ADSFile.old > ADSFile.txt 3. del ADSFile.old
Using the more command textual data from a named stream can be copied into a new files default unnamed stream or a named stream. This technique however cannot be used to transfer textual data from a file or directory to another directories named stream.
Using more to create a new file from an existing file stream:
more < ADSFile.txt:alternate_stream.txt > ADSFile2.txt
Using more to create a new file and associated stream from an existing file stream:
more < ADSFile.txt:alternate_stream.txt > ADSFile2:alternate_stream.txt
Using more to create a new file from a directories named stream:
more < ADSDirectory:alternate_stream.txt > ADSFile3.txt
Using more to create a new file and associated stream from a directories named stream:
more < ADSDirectory:alternate_stream.txt > ADSFile3.txt:alternate_stream.txt
The more command can also be used to copy the textual contents of one stream into a newly created named stream on the same file or directory.
Using more to copy the contents of one stream to another in the same file:
more < ADSFile.txt:alternate_stream.txt > ADSFile:alternate_stream2.txt
[edit] Associated risks of streams
Streams are not inherently a security risk, but the lack of native Windows support for locating, editing, and removing them does leave open opportunities for potential abuse. Depending on how a systems virus scanner and other security mechanisms deal with streams, if at all, plays a large part in mitigating potential risks associated with using NTFS volumes. Ultimately the benefits of NTFS volumes far out weight the potential risks of streams as long as system Administrators are aware of streams and their security model have the proper tools to handle them.
[edit] Using streams to avoid detection
There are probably as many different reasons for hiding data in streams as there are types of data to be hidden. The methods for obfuscation of data in streams ultimately are limited only by the imagination of the person who wishes to achieve such a task. Below are a few examples of ways streams may be used to avoid detection.
Kernel space filter drivers such as kdl can make use of streams by attaching its log file to any number of system files or directories. Extensions such as StrmExt.dll will not show the existence of a stream used by kdl while active, but other tools such as ADSspy, LADS, and Streams, will reveal the stream since they process streams differently. By holding the lock kdl makes sure the stream cannot be deleted or altered by external tools until the lock is released. Since kdl is set up by default to use registry keys for configuration changing the log file to be stored as an alternate stream is trivial.
kdl registry key changes to make use of a streamed log:
- Click Start --> Run and enter regedit
- Click Edit --> Find and enter \SystemRoot\kdl
- Change the Storage value to:
- For a directory stream \systemroot\System32:kdl.log
- For a file stream \systemroot\System32\ntoskrnl.exe:kdl.log
The above information is valid for a default configuration version of kdl 1.0.3 and should be noted that attaching a stream to a critical file or directory has a high probability of triggering a security mechanism.
The System File Checker (sfc.exe) will verify versions of protected system files, but it ignores any stream data associated with those files. This leaves any user with the appropriate permissions with the ability to attach stream data to protected system files that will not be detected by sfc.exe.
A few methods that may also avoid detection:
- Using stream names such as encrypted, archive or other common Windows terms
- Creating streams with no extension identifier
- Creating streams attached to obscure system files for data dumps, log files, etc. (packager.exe, sqlsodbc.chm, etc)
- Storing encrypted data in single or across multiple streams
- Storing binary data across multiple streams to be re assembled and executed at time of use to avoid detection
- Store device drivers as streams.
[edit] Destructive use of streams
Malice and destructive behavior seem to be a common thread for those who wish to experiment with technology in an attention getting way. The motivation and reasoning behind such acts are as boundless as the imagination used to conceive them, but they should not be ignored. Destructive behavior is a fact of life that all system Administrators must at some point be ready to deal with, clean up, and counter.
Below are just a few possible behaviors that should be taken into account when assessing the possible threats of streams:
- Flooding a guaranteed available critical file such as ntoskrnl.exe with useless stream data to use all available disk space.
- Trojan, worm, virus, malware, spyware, etc attached as streams.
- Stealing trade secrets via streamed files
[edit] Other uses for streams
Alternate data streams are often overlooked as viable solutions to adding features due most likely to the fact that they are presently bound to NTFS volumes, and have minimal documentation. Nevertheless alternate data streams contain endless constructive possibilities for enhancing many tools that exist today as well those to be developed in the future.
Some possibilities of added functionality through streams:
- Encryption information
- Backup, maintenance, etc information on files and directories
- Extended information about file activity
- md5 data
