Windows Shellcode: Writing and Analyzing Position-Independent Payloads

Malware & C2
2025.10.26
Time it takes to read this article 6 minutes.

Introduction / Overview

Disclaimer: This article is for education and authorized security testing only. Write, run, and analyze shellcode exclusively in an isolated lab you own or have explicit written permission to test. Executing payloads against systems you do not control is illegal in most jurisdictions.

Shellcode is the rawest form of an exploit payload: a self-contained blob of machine code with no headers, no imports, and no loader to lean on. To run anywhere it lands, it must be position independent and resolve its own dependencies at runtime. On Windows, that means two classic techniques every analyst should understand cold: walking the PEB to find kernel32.dll, and API hashing to locate functions without storing readable strings.

This walkthrough builds a small x64 payload that pops MessageBoxA, explains every trick it uses, and then flips to the blue-team side: how to emulate, debug, and detect this code.

How it works / Background

A normal PE relies on the loader to populate the Import Address Table. Shellcode gets none of that, so it bootstraps itself:

  1. Find the PEB. On x64 the Process Environment Block is always reachable via the GS segment register at offset 0x60 (GS:[0x30] on x86 via FS). The PEB is process-relative, so no hardcoded address is needed.
  2. Walk the loader lists. PEB->Ldr->InMemoryOrderModuleList is a doubly linked list of loaded modules. ntdll.dll and kernel32.dll are present in every user-mode process, so the shellcode walks the list to grab kernel32's base address.
  3. Parse the export table. From the base address it reads the PE header, finds the IMAGE_EXPORT_DIRECTORY, and iterates AddressOfNames.
  4. API hashing. Instead of comparing against strings like "LoadLibraryA" (which AV/EDR signature on, and which bloat the payload), it hashes each export name with a small rolling hash (ROR-13 add is the canonical one popularized by Metasploit's block_api) and compares against a precomputed 4-byte constant. A match yields the function RVA, which is added to the base to get a callable pointer.

With LoadLibraryA and GetProcAddress resolved this way, the shellcode can pull in anything else (user32!MessageBoxA, ws2_32 sockets for a reverse shell, etc.).

Prerequisites / Lab setup

Use a disposable Windows VM (or a Linux box for assembly/emulation) with no network bridge to anything you care about.

  • NASM to assemble: apt install nasm / choco install nasm
  • scdbg (from David Zimmer / sandsprite) — a libemu-based shellcode emulator
  • x64dbg for live debugging on Windows
  • Ghidra or radare2 for static analysis
  • Optionally Metasploit to compare against a known-good msfvenom payload

A quick reference payload for comparison:

msfvenom -p windows/x64/messagebox TEXT="lab" -f raw -o msgbox.bin
# Inspect length and bytes
ls -l msgbox.bin
xxd msgbox.bin | head
Bash

Walkthrough / PoC (step by step)

1. PEB walk + hashed kernel32 resolution (x64 NASM skeleton)

The snippet below is the core bootstrap. It is position independent: no absolute addresses, only register-relative access. Comments mark each stage.

bits 64
default rel

start:
    mov  rax, gs:[0x60]          ; RAX = PEB
    mov  rax, [rax + 0x18]       ; PEB->Ldr
    mov  rsi, [rax + 0x20]       ; Ldr->InMemoryOrderModuleList (first entry)
next_module:
    mov  rbx, [rsi + 0x20]       ; module DllBase
    mov  rdi, [rsi + 0x50]       ; BaseDllName.Buffer (UNICODE)
    mov  rsi, [rsi]              ; follow Flink to next module
    test rbx, rbx
    jz   not_found
    ; (compare hashed module name to find kernel32, then parse exports)
    ; ... export-table walk + ROR13 hash compare omitted for brevity ...
ASM

The full version parses IMAGE_DOS_HEADER -> IMAGE_NT_HEADERS -> Export Directory, iterates AddressOfNames, computes the ROR-13 hash of each ASCII name, and matches against constants such as LoadLibraryA = 0xEC0E4E8E.

Assemble and extract raw bytes:

nasm -f bin shellcode.asm -o shellcode.bin
xxd -i shellcode.bin | head      # C array form for a test harness
Bash

2. Emulate without executing: scdbg

scdbg runs the bytes against an emulated Windows environment and logs every API call — the fastest, safest first look:

scdbg /f shellcode.bin            # auto-find entry, emulate
scdbg /f shellcode.bin /s -1      # try every offset as an entry point
scdbg /f shellcode.bin /api /foff 0   # verbose API logging from offset 0
Bash

Expected output names the resolved APIs in call order, e.g.:

LoadLibraryA(user32)
MessageBoxA(text="lab", caption=..., type=0)
Plaintext

This is exactly how you confirm what an unknown blob does before ever letting it touch a real CPU.

3. Live debugging with a harness

To debug on Windows, drop the bytes into a tiny loader that marks the buffer executable and jumps to it:

// loader.c  (build: cl loader.c)  — LAB ONLY
#include <windows.h>
unsigned char sc[] = { /* xxd -i output here */ };
int main(void) {
    void *m = VirtualAlloc(0, sizeof sc, MEM_COMMIT|MEM_RESERVE, PAGE_EXECUTE_READWRITE);
    memcpy(m, sc, sizeof sc);
    ((void(*)())m)();
    return 0;
}
C

Open the compiled binary in x64dbg, set a breakpoint on VirtualAlloc's return, then single-step into the buffer to watch the PEB walk (gs:[0x60]) and the export-table loop resolve hashes in real time.

Mermaid diagram

Windows Shellcode: Writing and Analyzing Position-Independent Payloads diagram 1

The diagram shows the runtime bootstrap: from the GS-relative PEB read, through the module-list walk and export parsing, to hash-matched API resolution and the final payload call.

Detection & Defense (Blue Team)

Shellcode behavior is noisy if you know where to look. Weight these defenses as seriously as the offense above.

Static and emulation triage

  • Treat any executable buffer reached via gs:[0x60]/fs:[0x30] access plus an export-table walk as high-confidence shellcode. Run unknown blobs through scdbg /f file.bin /s -1 in a sandbox before anything else.
  • Hunt for API hashing: the ROR-13 constant table and the 0x0D rotate are well-known signatures. YARA rules from Florian Roth and the block_api Metasploit stub catch a large share of real-world payloads.

Runtime / EDR detections (map to MITRE ATT&CK)

  • T1055 Process Injection and T1620 Reflective Code Loading: alert on VirtualAlloc/VirtualProtect creating PAGE_EXECUTE_READWRITE regions, especially in non-image, private memory.
  • Memory scanning: tools like Moneta, PE-sieve, and Hollows Hunter flag executable private regions with no backing module — the hallmark of in-memory shellcode.
  • ETW + call-stack telemetry: a kernel32!MessageBoxA (or WS2_32 connect) whose return address points into unbacked memory is a strong injection signal. Modern EDRs flag unbacked-stack-frame syscalls.

Hardening

  • Enable CFG (Control Flow Guard), CET shadow stacks, and ACG/Arbitrary Code Guard where supported to block jumps into unsigned RWX memory.
  • Apply Windows Defender Exploit Guard and Attack Surface Reduction (ASR) rules (e.g., "Block executable code from being created by Office/scripts") to cut common shellcode delivery paths.
  • Deny W^X: minimize processes that allocate RWX. Many legitimate apps never need it, so RWX allocation is itself a useful hunting predicate.

For deeper static workflows see Ghidra for malware analysis and reading PE export tables. For the injection side, see process injection techniques.

Conclusion

Position-independent Windows shellcode is a compact lesson in the whole Windows loader: PEB-relative addressing makes it relocatable, list walking finds kernel32 without imports, and API hashing hides intent from naive string scanners. The same structure that makes it powerful also makes it detectable — RWX allocations, unbacked call stacks, and the ROR-13 hash signature are reliable tells. Understanding both halves is what separates an analyst from a script user.

References

You Might Also Like

If you found this useful, these related deep-dives cover adjacent techniques and their defenses:

Comments

Copied title and URL