IT Forensics

The volumes of data that IT forensic investigators must examine are constantly growing. As a result, IT forensic methods must become more efficient. One key approach involves automating certain data analysis tasks. Today, this often relies on the recognition of known content. It is helpful for IT forensic investigators if both critical (blacklisting) and non-critical content (whitelisting) can be identified; the more that is recognized, the less needs to be manually reviewed in the end. In practice, cryptographic hashes are used for rapid recognition; however, these can only identify files that are exactly identical. Files with minor changes from the original file cannot be detected this way. It would be better if similar files could also be automatically recognized.

The similarity search allows for the filtering out of forensically irrelevant files during the analysis process. This means investigators do not have to spend time on manual review. Furthermore, it is also possible to identify forensically relevant files that have been intentionally modified to avoid detection. With tools for detecting similar files, for example, both irrelevant program files can be effectively filtered out and Office documents can be sorted by similarity, enabling a more efficient review by the investigator.

New Efficient Algorithm

A major challenge in the development of similarity search tools is the efficiency of the search algorithm. The lack of efficient solutions prevents the practical use of such tools in IT forensics. Fraunhofer SIT has achieved a breakthrough with the development of Fast Forensic Similarity Search (F2S2). This method operates orders of magnitude faster and more efficiently than previous solutions. F2S2 was developed for the class of so-called piecewise hashes. A dedicated software product is already available for the well-known ssdeep hashes, which can be licensed. With F2S2, an efficiency increase of more than 99% can be achieved compared to the ssdeep software.

By focusing on piecewise hashes, F2S2 is particularly well-suited for detecting similar program files and similar Office documents.

The F2S2 software is available in the following versions:

• Program library with the core algorithms of F2S2
• F2S2 software for ssdeep hashes with a graphical user interface, command-line program, and API

In addition to the software, Fraunhofer SIT also offers the following in this context:

• Development of custom F2S2 variants
• Support for integrating F2S2 into existing forensic software