Book

Reverse Engineering for Security Investigations

What happened, how did it happen, and what can the software tell us about it? This book develops the understanding needed to investigate compiled programs and connect their behavior to security incidents. Follow unfamiliar executables from an initial inspection through static and dynamic analysis, system interactions, vulnerabilities and concealed code, then combine the findings with incomplete incident evidence. Across eight substantial parts, Ghidra and GDB become tools for answering concrete questions. Assembly, executable formats and other foundations enter where they unlock a discovery. Explanatory prose carries the argument, supported by code, disassembly, diagrams and optional runnable examples. The goal is to find hidden capabilities, understand what activates them and establish what the evidence supports—from the first suspicious file to a technical account another investigator can assess.

HackerBoost

Subscriptions coming soon

We are preparing the HackerBoost library. You can create an account now; subscriptions will open later.

Sign in to read the first three chapters of the first part for free.

Contents

Part 1: Investigate an unfamiliar executable

Start with a suspicious program found during an incident. Establish what it can do, what deserves attention and which questions require deeper analysis. Introduce executable formats, imports and assembly through discoveries about the specimen, while distinguishing suspicious features from evidence of malicious behavior.

  1. 1.What did we find on this server?Free preview
  2. 2.A harmless name tells us very littleFree preview
  3. 3.The first clues inside the fileFree preview
Part 2: Ghidra: uncover capabilities and hidden logic

Reconstruct the mechanisms behind credential access, command execution, configuration handling and communication. Develop decompilation, cross-reference analysis, type recovery and scripting around the question: what capabilities does this binary contain, and what conditions activate them?

    Part 3: GDB: observe security-relevant behavior

    Catch a program decrypting its configuration, processing an unexpected input, accessing sensitive data or corrupting memory. Build a detailed understanding of debugging, execution state and observation effects. Distinguish behavior demonstrated in a controlled run from behavior established during the original incident.

      Part 4: Trace the program’s interaction with the system

      Follow activity across processes, libraries, files, permissions and network connections. Investigate persistence, process injection, privilege boundaries and communication with other components. Explain how local mechanisms fit into a larger intrusion—and what additional evidence is needed to connect them.

        Part 5: Investigate crashes, vulnerabilities and security patches

        Work backward from a failure and compare vulnerable and corrected versions. Recover the faulty assumption, understand the conditions that expose it and assess its security consequences. Connect binary analysis with crash artifacts and incident evidence without treating every crash as proof of exploitation.

          Part 6: Expose concealed code and evasive behavior

          Investigate packing, obfuscation, encrypted payloads, dynamic loading and anti-analysis techniques. Recover behavior that a first inspection misses, understand why an executable behaves differently under observation and identify what remains uncertain after unpacking.

            Part 7: Reconstruct activity from incomplete evidence

            Combine binaries with core files, memory captures, logs, network records and recovered configuration. Investigate what ran, what data may have been accessed and which sequence of events fits the evidence. Develop timelines, test competing explanations and distinguish an implemented capability from its actual use.

              Part 8: Complete and defend a security investigation

              Bring the methods together through several substantial cases: a suspicious service, a crash potentially linked to exploitation, a concealed payload and a compromised application component. Follow each case from initial evidence through technical analysis to a clear report that another investigator can reproduce and assess.