Be Kind, Rewind... The USN Journal

Summary

In this episode of 13 Cubed, the focus is on research by Cyber CX regarding the NTFS USN Journal. The video covers how the USN Journal tracks changes to files, such as creation, deletion, and movement, but faces challenges when trying to reconstruct full file paths due to MFT entry reuse. Cyber CX proposes a method of reading the journal in reverse, which allows investigators to reconstruct file paths, even after files are deleted or MFT entries reused. The presenter demonstrates how a Python tool automates this process, significantly enhancing forensic analysis and investigation accuracy.

Takeaways

• 😀 Cyber CX research introduces a method to track file paths using the USN Journal, addressing common issues with full file path tracking.
• 😀 The USN Journal logs the MFT entry number and sequence number, but it doesn't track full file paths directly.
• 😀 NTFS MFT entry numbers uniquely identify files, while sequence numbers help track file reuse, such as after deletion.
• 😀 Forensic tools typically cross-reference MFT entries with the USN Journal to determine file paths, but this process can fail if MFT entries have been reused.
• 😀 Cyber CX proposes a reverse-reading approach to reconstruct original file paths, even if the MFT entry has been reused multiple times.
• 😀 The reverse-read method can track file movements and deletions, helping investigators reconstruct paths for deleted or reused files.
• 😀 Cyber CX released a proof-of-concept Python script to automate this process using a CSV output from MFTECMD for both the MFT and USN Journal.
• 😀 The script writes the parsed results to an SQLite database and generates a CSV file containing complete file paths.
• 😀 The reverse method works even when the MFT entry has been reused for completely different files after the original file was deleted.
• 😀 Using the Cyber CX tool, investigators can recover file paths, as demonstrated by tracking the location of a Powershell script (Clore utility.ps1) after it was deleted.

Q & A

What is the USN journal in the context of NTFS file systems?

The USN journal is a feature of the NTFS file system that logs changes made to files and directories, including creations, modifications, and deletions. It records the MFT entry and sequence number, which allows forensic investigators to track changes to files on a system.

Why does the USN journal not track full file paths, and how does this affect forensic investigations?

The USN journal does not track full file paths because it only logs the MFT entry and sequence number associated with each file or directory. This can be problematic in forensic investigations when trying to reconstruct the original location of a file, especially after the MFT entry has been reused for a different file.

What is the difference between an MFT entry and a sequence number in NTFS?

In NTFS, the MFT entry is a unique identifier for a file or directory, corresponding to a specific record in the Master File Table (MFT). The sequence number is a version counter that increments every time an MFT entry is reused, such as after a file is deleted and a new one uses the same MFT entry.

How does NTFS prioritize the reuse of MFT entries over creating new ones?

NTFS prioritizes reusing existing MFT entries to save space and optimize performance. When a file is deleted, its MFT entry is marked as available for reuse, and a new file may use that same MFT entry almost immediately, which can overwrite the original file’s path information.

What problem arises when MFT entries are reused, and how does this impact reconstructing file paths from the USN journal?

When MFT entries are reused, the original path information may be lost because the new file will overwrite the previous entry's data. This makes it difficult to reconstruct the full file path of a file logged in the USN journal, especially after the MFT entry has been repurposed.

What solution does Cyber CX propose to address the challenge of reconstructing file paths from the USN journal?

Cyber CX proposes reading the USN journal in reverse and tracking stateful information about each journal entry. This method allows investigators to determine the full file paths for files, even after their MFT entries have been reused, by following the changes made to the files as they were created, moved, or deleted.

How does Cyber CX’s proof of concept tool automate the process of reconstructing file paths?

Cyber CX’s tool automates the process by taking CSV outputs from tools like mfte CMD (which parses the MFT and USN journal) and storing the data in an SQLite database. The tool then rewinds the USN journal entries, reconstructing the full file paths, and outputs the results in a CSV file with the resolved file paths.

What is the primary challenge when manually correlating USN journal entries with MFT records?

The primary challenge is that if the MFT entry has been reused, the original file path information may be lost. This can make it difficult to manually match the USN journal entry to the correct file, especially when the entry has been repurposed for a different file after the original file was deleted.

How does the Cyber CX tool help solve the problem of MFT entry reuse when correlating the USN journal with files?

The Cyber CX tool solves the problem by reversing the USN journal and tracking the state of each file and directory, including their parent paths. This allows the tool to reconstruct file paths even if the MFT entry has been reused for a different file, providing a more accurate and comprehensive view of the file system's history.

What is the role of Timeline Explorer in the demonstration of the Cyber CX tool?

Timeline Explorer is used in the demonstration to view and analyze the parsed outputs of the MFT and USN journal CSV files. It helps to visually correlate the data from these files and verify the reconstructed file paths after running the Cyber CX tool, as shown in the example with the Powershell script 'Clore utility.ps1'.