Monday, 21 April 2014

Search history on windows 8.1 - Part 2

I have recently blogged about windows 8.1 search history and how searched terms/phrases are recorded as LNK files in a post here. But windows also logs searched terms (search history) to the event log and web history (and cache).

From the LNK files, we know the first time a term was searched for, but not the next time or the last time it was searched, which is usually more relevant from an investigation perspective. However, this information can be obtained from the Connected-Search event log file. On disk it would be under:
 \Windows\System32\Winevt\Logs\Microsoft-Windows-Connected-Search%4Operational.evtx

Under Event viewer, you can find it under:
 \Applications and Services Logs\Microsoft\Windows\Connected-Search\Operational

Below is a screenshot for one such log entry.

Searched keyword is 'enscript' and machine was online when search was run

Windows logs all URLs and reference links here. Windows, by default tries to search for everything online as well as on the machine. Even if you are offline, a search URL for online searches is generated and seen here. The screenshot below shows the same search run when machine was offline (not connected to internet).

Searching for 'enscript' when machine is offline

Each time a search is run, an entry is created, sometimes multiple entries (this probably has to do with different views when browsing the search results). For searches (if machine was online), the URL requests and responses are also found in the IE web history and cache database (WebcacheV01.dat). The database is located at
\Users\<USER>\AppData\Local\Microsoft\Windows\WebCache\

The best way to study this data would be by parsing this database either manually (using libesedb and lots of data formatting with additional parsing!) or use a free program like IE10 History reader (or an expensive brand forensic tool). However, if you are just interested in the search terms without dates or other information, a raw search into the datebase files will suffice.

To find searched terms, you will need to search  for URLs beginning with
 https://www.windowssearch.com/search?q=

The screenshot below shows hits when searching the IE web cache files for the above URL using Encase.
Search hits in IE web cache database as seen in encase

Search as you type

An aspect missing from LNK files and the event logs is the search suggestions and interim search results. As Rob Lee hinted to me earlier that a user could search for a term (without hitting Enter or the search button after entering the term in the search box), but not click on any results and none of the above artifacts would be created. Windows uses the 'search as you type' feature and  terms windows guessed for you (as you were typing into the search box) or interim search results are discarded. However you would find some traces of the terms as windows will make online queries for the 'search as you type' feature. If this is not clear, just recall how you search with google. As you begin typing the letters of your keyword into the search box, google automatically suggests most popular searches beginning with those letters. Windows also does the same thing.

Google's search as you type feature

To find such searches which were discarded, search for URLs beginning with
  https://www.windowssearch.com/suggestions?q=

Actually, these are not the suggestions, but the lookups for term/phrase entered into the search box. The data returned (query response) will contain the suggestions.

Below is a screenshot showing the webcacheV01.dat file (and supporting db files) with search hits as displayed in Encase.

Search hits showing Windows querying for popular search term suggestions based on user entered input

Thus there are multiple locations (connected search LNK files, event logs, web cache) where an investigator can find evidence of searches run by a user. Each has its uses and caveats.


Friday, 4 April 2014

Windows 8 Thumbs.db files - still the same and not the same!

Screenshot of folder in Windows 8 showing Thumbs.db

Thumbs.db files have made a comeback in windows 8. Now, like in windows XP, explorer will create these files in every folder containing media files. This used to be a great forensic resource for investigators because thumbnails once created and stored in the Thumbs.db remained there even after the image file itself was deleted. This behavior is also noted with Windows 8.

The only thing that is different is the format of these new Thumbs.db files. It is not the Windows XP format and the usual thumbs.db file viewers including most forensic tools will not parse this file correctly. The format is actually the same as Windows 7 Thumbs.db files. Yes, that was not a typo, I said 'Windows 7'. I had looked into this earlier and the details are available here.

An interesting thing to note is that in windows 8, the same Thumbcache_*.db files are still maintained on a per user basis like windows 7 does. So the Thumbs.db is really a redundant location for these thumbnails as they are already cached in the Thumbcache database. So why the duplication?

Update (Thanks proneer for this tip!):
There are some caveats here. On windows 8, Thumbs.db will only be created in folders under a user profile folder, so anything created in C:\ or C:\program files or C:\program data or any other folder not under a user profile, ie, C:\Users\<USER>\* will not have thumbs.db files. 

But this has got nothing to do with a particular logged in user. A thumbs.db file will be created even when the logged in user browses folders of another user under their profile (as long as file permissions allow that user to write files to the other users' folder).

This behavior is different from Windows 7 thumbs.db where the location does not matter for creation of thumbs.db files.

There is another oddity noted. Sometimes a thumbs.db is created immediately upon folder being opened in explorer, on other occasions it has be triggered by changing the 'view' of the folder to 'Large icons'.

Tuesday, 1 April 2014

Search history on Windows 8 and 8.1

Windows 8 introduced a new feature of saving previously searched terms/keywords. I am refering to the Windows Search functionality which moved from the Start-menu in Windows 7 to the Charms bar in Windows 8.

Search terms are saved on a per user basis. In Windows 8, this is stored as an MRU (Most Recently Used) list in the NTUSER.dat file under the key:
Software\Microsoft\Windows\CurrentVersion\Explorer\SearchHistory\Microsoft.Windows.FileSearchApp

Figure 1 - Search history (MRU) in Windows 8 registry

Windows 8.1

On Windows 8.1 this has changed! These entries are no longer stored in the registry, instead they are stored on disk at:
\Users\<USER>\AppData\Local\Microsoft\Windows\ConnectedSearch\History

They are stored as individual link (LNK) files. Each link file holds a single previously searched for keyword (or phrase).

Figure 2 - Search history in Windows 8.1 stored as LNK files

The format of this link file is similar to the one we are familiar with from earlier versions of windows, however, no dates or other details typically seen in link files are included. All it contains is a link header and a shell item id list. The shell item id list contains the keyword/phrase searched for. Current link file parser scripts/tools will not be able to parse this correctly as they are either not parsing the Shell item id list or not (yet) looking for this specific information. (A shell item id list is seen in many places in the registry, one of the more popular artifacts that uses it is the 'shell bags').

Figure 3 - Search history LNK file showing searched term 'enscript'
As seen in figure 3 above, this link file has the same header as well as basic format. The link guid at offset 0x4 is also the same. Link flags (0x80) indicate only a Shell Item Id List will be present and all other fields are blank (zero). The shell item id list contains a single property identified by guid '{F29F85E0-4FF9-1068-AB91-08002B27B3D9}'. This guid identifies the Microsoft Office Summary Information Properties. Only a single value is populated and that is the keyword/phrase searched for.

Forensic Importance

From a forensic perspective, this ties a search keyword to a user and a date. This means that we now know the date and time when a particular user searched for a specific keyword on the machine. The last modified timestamp gives us the first time that keyword is searched and it does not get updated after, even if the search is repeated. On my machines, all 4 timestamps (created, accessed, modified, entry modified) hold the same value for a single file (see figure 2 above) and don't seem to get updated/altered once created.

Tuesday, 24 December 2013

Device LastRemovalDate & LastArrivalDate Behavior in Windows 8

Many people have asked me the conditions when the LastRemovalDate property gets populated and why its missing in some cases. I had run some test cases to determine the conditions and behavior of windows 8 with device insertions and removals earlier and am now documenting the results here. For those unaware of these timestamps, please read the post here first.

Device activity behavior

Whenever a device is plugged into a windows 8 machine, the LastArrivalDate timestamp gets set (to current date & time). At the same time, the LastRemovalDate gets deleted (if it was set earlier). Now whenever the device is removed from the system (when system is running!) that is the only time the LastRemovalDate will get set (to current date & time). Windows can detect both a clean eject as well as an unclean direct disconnect of the device, and in both cases the LastRemovalDate timestamp gets set.

If a device is attached to a system and then the system is shutdown subsequently with device still attached, then the LastRemovalDate will NOT get updated! So if you are seeing a missing value for LastRemovalDate, this is likely what happened, ie, the device was still plugged into the system when it was shut down. So the windows last shutdown timestamp for that session could be taken as the LastRemovalDate by an analyst.
Now on subsequent reboot(s), this device timestamp (LastRemovalDate) will not get updated and it will remain missing, until the device is seen by windows again and windows witnesses a removal of that device (as noted above).

However, also note that even if the device is NOT removed and re-plugged in, windows will still treat it that way when you reboot the system. So, reboots with a USB disk plugged in will update the LastArrivalDate as if it had been inserted immediately on boot.  This means that if you have a USB disk always connected to the system and never removed, windows will still update the LastArrivalDate each time on a reboot.

How this impacts an analysis?

The forensic analyst must be careful about interpretation here, the LastArrivalDate may not be the last time the device was physically connected by a user, it may have been there (connected) for a long time prior! One way to check is compare this with the system boot time. If they are quite close (within a few seconds or a minute), then its probably connected prior to boot, else it was indeed the last time device was physically connected.

Also because LastRemovalDate is deleted upon subsequent device arrivals, you should never ever see LastRemovalDate that is prior to a LastArrivalDate. If you do, then that probably means the clock on the machine has been altered between insertion and removal of the device!

The table below summarizes activity and behavior of these timestamps.

Activity / Action
LastArrivalDate
LastRemovalDate
Device Plugged in
SET
DELETED
Device Removed
 (Both Clean Eject & Direct Removal)
-
SET
Machine Shutdown with device still plugged in
-
-
Machine Restarted with device still plugged in (device not removed and re-attached)
SET
DELETED
    The dash ( - ) indicates no changes occured, values remain what they were earlier.

Monday, 16 December 2013

Amcache.hve - Part 2

My last post about the Amcache.hve file only concentrated on the 'File' key since that's where all of the good stuff is! This post describes the remaining contents of the Amcache.hve file, the other files in the AppCompat folder (where Amcache.hve is located) and useful information contained therein.

As noted in the earlier post, there are 4 sub-Keys containing data - File, Generic, Orphan, Programs. There is also one value called Sync as shown below.

Contents of Amcache.hve/Root

The Sync value holds an 8 byte FILETIME timestamp. I believe this represents the last time this data was synced with the 'AEINV_CURRENT.xml' file also contained in the same folder as amcache.hve. However, not all information is synced. The synced information appears to be mostly about installed programs or installers run. Traces for standalone application (applications that are not installed) runs are never synced and only remain in the Amcache.hve file. Update (9 Jan 2014): Standalone applications runs are also seen here at times.

Programs Key

The 'Programs' key contains data about installed programs, the same information you can find in the Control Panel -> Programs & Features. This is somewhat similar to the data in the File key. Each subkey contains a ProgramID, which is an ID assigned to every MSI (installer) package when it is compiled. Each of these contain values as seen below. The interpretation of these values differ from the ones found under 'File'.


Here is the description for values that exist under Programs.

ValueDescriptionData Type
0Program NameUNICODE string
1Program VersionUNICODE string
2PublisherUNICODE string
3Language code (1033 for en-US)UNICODE string
4~ Not seen ~
5Unknown Flags (usually 256)DWORD
6Entry Type (usually AddRemoveProgram)UNICODE string
7Registry Uninstall KeyUNICODE string
8~ Not seen ~
9~ Not seen ~
aInstall DateQWORD (Lower 4 bytes is unix date)
bUnknown (always zero?)QWORD
c~ Not seen ~
dList of File PathsUNICODE strings (REG_MULTI_SZ)
fProduct Code (GUID)UNICODE string
10Package Code (GUID)UNICODE string
11MSI Product Code (GUID)UNICODE string
12MSI Package Code (GUID)UNICODE string
13Unknown (usually zero)QWORD
FilesList of  Files in this package (VolumeGuid@FileRefUNICODE strings (REG_MULTI_SZ)

In my analysis, most of the files (not all) referenced in the 'Files' list here could be found in the 'File' key.

Orphan and Generic Keys

The Orphan Key contains keys having the name in the format VolumeGuid@FileRef. A sample key looks like this:
      Orphan\44177282-4260-11e3-9713-806e6f6e6963@30000e61a
where '44177282-4260-11e3-9713-806e6f6e6963' is the Volume GUID and '30000e61a' is the file reference number. Beneath this key is a single Value by name 'c' containing an unknown DWORD value which is either 0 or 1.

'Orphan' key in Amcache.hve Hive
See the below screenshot for a view of the Generic Key. Under the '0' subkey you find many keys which are either GUIDs or File IDs. These File IDs (as Microsoft calls them) are simply SHA1 hashes of the files they represent.  It is unknown what the GUIDs represent. Similar to the Orphan keys, here too each of these leaf node keys (GUID or File ID) has a value by name '0' containing an unknown DWORD which is either 0 or 1.
'Generic' key in Amcache.hve Hive
Cross referencing entries from the 'File' and 'Programs' keys to the files referenced by Generic and Orphan shows many matches, many missing as well as extra entries. So the relationship between these is not entirely clear.

Other files in this folder

Apart from the log/cache files associated with the Amcache.hve hive, there are some other files in the AppCompat folder:
  1. AEINV_AMI_WER_{MachineID-GUID}_DATE_TIME.xml
  2. AEINV_CURRENT.xml
  3. AEINV_PREVIOUS.xml
The AEINV here stands for 'Application Experience Inventory'.
All of the above are XML files containing similar data about installed programs, files, application metadata and IE Addons (toolbars and plugins) information. The AEINV_AMI_WER_{MachineID-GUID}_DATE_TIME.xml file is related to Windows Error Reporting (WER). Here the MachineID-Guid is a value generated and used by WER only. This file existed in Windows 7 too with almost the same contents.

Device Information (new in Windows 8)

In Windows 8, this file also stores machine Device information containing among other things USBSTOR information although not in the detail found elsewhere in the registry. So you don't have device unique serial IDs or container IDs but you do get some descriptive strings like 'Seagate Backup+' or 'Sandisk Cruzer v3'. It does contain some Device GUIDs (although I am unable to match it to anything in the registry or setupapi log yet).

Snippet from AIENV_AMI_WER_xxxxxx xml file showing USBSTOR device info

AEINV_PREVIOUS.xml also existed in same format in Windows 7. AEINV_CURRENT.xml is a new addition in Windows 8, but contains similar data. By analyzing the timestamps and USNJRNL log, it is apparent that periodically the 'PREVIOUS' file gets deleted, then the 'CURRENT' file get renamed to 'PREVIOUS' and a new 'CURRENT' file is created and populated with data. (That was obvious from the file names but I just had to confirm!)

Snippet from the parsed NTFS $USNJRNL.$J file

Wednesday, 4 December 2013

Amcache.hve in Windows 8 - Goldmine for malware hunters

Corey Harell has uploaded an excellent writeup on the working of Windows Application Experience and Compatibility features. Here he explains how process entries/traces show up in locations such as the ShimCache and RecentFileCache.bcf. For forensic/malware analysts, this is a great place to search for recent processes that were run.

This post is a logical continuation of Corey's post. In Windows 8, the 'RecentFileCache.bcf' file has been replaced by a registry hive named 'Amcache.hve'. The location of this file is the same as its predecessor:
<DRIVE>\Windows\AppCompat\Programs\Amcache.hve

This file stores information about recently run applications/programs. Some of the information found here includes Executable full path, File timestamps (Last Modified & Created), File SHA1 hash, PE Linker Timestamp, some PE header data and File Version information (from Resource section) such as FileVersion, ProductName, CompanyName and Description.

The Hive

Amcache is a small hive. Below is a view of the hive loaded in encase. There are only 4 keys under a 'Root' key. (Folders in the registry are called keys). The data of interest to us is located in the 'File' key. Files are grouped by their volume GUIDs. These are the same Volume GUIDs that you can find in the SYSTEM hive under MountedDevices and also under NTUSER.DAT MountPoints2.

File References

Under each volume guid are File Reference keys each representing a single unique file. In case of an NTFS volume, this key name will look something like this: e0000430d. This is the NTFS File Id and sequence number. Here sequence number is 0e and file id is 0000430d. For FAT volumes it is unknown what this value represents. On a FAT volume, this File Reference is the byte offset of the Directory entry for that file, ie, the offset from the start of volume where this file's directory entry resides.

The Last Modified date on this key may be taken as the first time a particular application was run. I have not seen it change on subsequent runs. Under this key reside several values holding details about that file. Refer the illustration below. This is for a file on a FAT volume on external USB disk.


Value Names are in hexadecimal and range from 0 to 17 and then two extra entries for 100 and 101 are seen. Here are the descriptions I have been able to decipher so far.

ValueDescriptionData Type
0Product NameUNICODE string
1Company NameUNICODE string
2File version number onlyUNICODE string
3Language code (1033 for en-US)DWORD
4SwitchBackContextQWORD
5File VersionUNICODE string
6File Size (in bytes)DWORD
7PE Header field - SizeOfImageDWORD
8Hash of PE Header (unknown algorithm)UNICODE string
9PE Header field - ChecksumDWORD
aUnknownQWORD
bUnknownQWORD
cFile DescriptionUNICODE string
dUnknown, maybe Major & Minor OS versionDWORD
fLinker (Compile time) TimestampDWORD - Unix time
10UnknownDWORD
11Last Modified TimestampFILETIME
12Created TimestampFILETIME
15Full path to fileUNICODE string
16UnknownDWORD
17Last Modified Timestamp 2FILETIME
100Program IDUNICODE string
101SHA1 hash of fileUNICODE string

I've written an Enscript to parse out this information to the console. Download here. This is code, not an enpack, so anyone can easily translate to python or perl or another open platform.
It outputs Amcache information as shown below:

File Reference = 03f180
Volume GUID = {8e49b4d2-4d4a-11e3-9717-000c29775430}
First Run Timestamp (Last Modified on key) = 11/15/13 19:48:19
Modified Time 2 = 11/03/13 17:42:39
File path = E:\Fetch.exe
Language Code = 0
PE Header Hash = 01012bb2314b06e59d290d4effbab22e77d7f87ecbeb
File Size = 58880
PE Header SizeOfImage = 77824
PE Header CheckSum = 0x00014D67
PE Header Linker Timestamp = 0x4E8B796E = 10/05/11 02:53:58
Modified Time = 11/03/13 17:42:40
Created  Time = 10/04/11 23:23:58
SHA1 hash = 000005b6d3ebc6a5484a270f4f0e04738d1e5a53ee25

The Unexplained

There are two Last Modified timestamps (11 and 17). I have noticed that the timestamp in 17 is almost always 1 second behind the timestamp for 11. This is a bit of a mystery, it is probably due to conversion to a DOS timestamp and back.

The SHA1 hash is a vital bit of information that MS has added, as now we can track malware even if its deleted/wiped itself from the system. Also, since the hive stores data about volume guids and file references, it can also be added to the list of location to review to aid in tracking of USB devices. 

Friday, 22 November 2013

Event Log entries for Devices in Windows 8

This post is about entries created when devices (USB or other) are connected to a Windows 8 system. This post does not talk about Windows Event log basics, its format or parsers or where you can find them on a system. I assume you are here because you already know about that and simply want to know about USB artifacts in event logs on Windows 8.

Windows 8 has added many new Logs and Sources to its core Event Logging system. Entries for device connections (insertions) are seen in at least 5 logs:

1. SYSTEM

Source Event IDs When it Occurs?
Ntfs 98, ?? Every time a storage device containing an NTFS volume is connected
DriverFrameworks-UserMode 10000 Device first connect only
UserPnp 20001, 20003 Device first connect only

Description snippets:
Ntfs (Event 98) - Volume E: (\Device\HarddiskVolume4) is healthy. No action is needed.

DriverFrameworks-UserMode (Event 10000) - A driver package which uses user-mode driver framework version 2.0.0 is being installed on device SWD\WPDBUSENUM\_??_USBSTOR#DISK&VEN_KINGSTON&PROD_DATATRAVELER_G3&REV_PMAP#000FEAFB7959BC7067D40086&0#{53F56307-B6BF-11D0-94F2-00A0C91EFB8B}.

UserPnp (Event 20001) - Driver Management concluded the process to install driver wpdfs.inf_x86_d67a8256c1147128\wpdfs.inf for Device Instance ID SWD\WPDBUSENUM\_??_USBSTOR#DISK&VEN_KINGSTON&PROD_DATATRAVELER_G3&REV_PMAP#000FEAFB7959BC7067D40086&0#{53F56307-B6BF-11D0-94F2-00A0C91EFB8B} with the following status: 0x0
.

2. Microsoft-Windows-DeviceSetupManager/Admin

Source Event IDs When it Occurs?
DeviceSetupManager 112 Device first connect only or when connected to a different port

Description snippet:
DeviceSetupManager (Event 112) - Device 'HASP HL 3.25' ({95abe994-529a-11e3-971d-806e6f6e6963}) has been serviced, processed 5 tasks, wrote 42 properties, active worktime was 136063 milliseconds.

3. Microsoft-Windows-DeviceSetupManager/Operational

SourceEvent IDsWhen it Occurs?
DeviceSetupManager300, 301Device first connect only or when connected to a different port

Description snippet:
DeviceSetupManager (Event 300) - The device container '{D7FD8C4F-2F70-A826-D5FA-20A112B90D4E}' has entered the ready state

4. Microsoft-Windows-Kernel-PnP/Device Configuration

SourceEvent IDsWhen it Occurs?
Kernel-PnP400, 410, 420Device first connect only

Description snippet:
Kernel-PnP (Event 400)Device USBSTOR\Disk&Ven_Kingston&Prod_DataTraveler_G3&Rev_PMAP\000FEAFB7959BC7067D40086&0 was configured.

5. Microsoft-Windows-Kernel-PnPConfig/Configuration

SourceEvent IDsWhen it Occurs?
Kernel-PnP1, 2, 3, 4Device first connect only or when connected to a different port

6. Security

SourceEvent IDsWhen it Occurs?
Microsoft-Windows-Security-Auditing4663Each time device is connected to system

Comment: Chad Tilbury alerted me to this one. Chad notes that this entry is only seen if “Audit Removable Storage” auditing is configured within the Object Access category of the Advanced Audit Policy Configuration.

The comments on occurrence are based on my limited experimentation/research with a Windows 8.1 system over the last few days. Please let me if you are seeing any other activity or behavior or log entries.