Integer Overflows and Type Confusion Bugs

Integer Overflows and Type Confusion Bugs - article cover image RE & Pwn
Time it takes to read this article 7 minutes.

Disclaimer: This article is provided strictly for educational purposes and authorized security testing. Only run these techniques against systems you own or have explicit written permission to assess. Unauthorized access to computer systems is illegal in virtually every jurisdiction and can carry severe penalties.

Introduction

Integer overflows and type confusion are two of the most reliable roots of memory-corruption bugs in native code, because both violate an assumption the rest of the program silently depends on without crashing at the point of violation. An integer overflow happens when an arithmetic result exceeds the range its type can represent and silently wraps (unsigned) or invokes undefined behavior (signed, in C/C++); type confusion happens when code accesses memory through a type different from the one it was actually allocated or initialized as. Neither corrupts memory by itself — the crash comes later, when the wrapped size feeds an allocation or the mistyped pointer is dereferenced, which is what makes both bug classes hard to spot in review and highly reliable once weaponized.

These bugs matter because they are upstream of almost every other memory-safety primitive: an integer overflow in a size calculation is the classic precursor to a heap buffer overflow, and type confusion is the primitive behind a large share of browser and JavaScript-engine exploits, since dynamically typed runtimes represent values with tagged unions reachable and corruptible from script. Both classes routinely appear in image/font/media parsers and network protocol decoders — any code computing a buffer size from attacker-controlled fields — which is why they show up so consistently in browser, kernel, and file-format CVEs.

Attack Prerequisites

Exploiting either bug class requires a reachable calculation and a consumer that trusts its result:

  • Attacker influence over a value that feeds arithmetic — a length field, a count parsed from network input, an array index, or an object’s declared type/shape in a dynamically typed runtime.
  • A size or index calculation with no range/overflow check — multiplication or addition on attacker-influenced integers feeding directly into an allocation size, buffer index, or loop bound.
  • A consumer that trusts the (now-wrong) result — malloc()/new[] receiving a truncated size, followed by a copy sized against the *original*, un-overflowed length.
  • For type confusion: a way to reach a mistyped access — an unchecked reinterpret_cast/union access in C++, a JIT type-speculation path gone stale, or deserialization that skips a type-tag check.

How It Works

Integer overflow mechanics differ by signedness and width. Unsigned integers wrap modulo 2^n by definition (0xFFFFFFFF + 1 == 0 for a 32-bit value) — well-defined but still dangerous when unanticipated. Signed overflow in C/C++ is undefined behavior, which in practice usually wraps two’s-complement on mainstream compilers but can also be optimized away (the compiler may assume signed overflow never happens and eliminate a subsequent check entirely). A related, equally common class is truncation: assigning a 64-bit or size_t length into a narrower int/short silently drops the high bits, so a large legitimate length becomes small — or negative if signed.

The exploitation pattern is almost always the same: a size is computed from attacker-controlled input, the calculation overflows or truncates to a small value, that value is passed to an allocator which hands back a small buffer, and then a *second*, separately computed (unoverflowed) length copies data into that buffer — “allocate small, write big.”

Type confusion works on a different axis: instead of a wrong-sized region, the program has a *correctly sized but wrongly interpreted* region. In C++ this happens through unchecked downcasts, unions, or manual object-lifetime management where a freed type-A object is reallocated as type B and a stale type-A reference accesses it. In JIT-compiled dynamic languages (V8’s JavaScript, for example) it often comes from speculative optimization: the JIT compiles a fast path assuming an object’s *hidden class*/shape, and if that assumption is invalidated mid-execution the compiled code keeps treating the object as the old, now-incorrect shape.

Vulnerable Code / Configuration

A classic multiplication overflow bypassing a size check — the textbook “allocate small, write big” pattern. count/elem_size are attacker-influenced, and their product wraps a 32-bit int before it reaches malloc:

// VULNERABLE: no overflow check on count * elem_size
int parse_records(uint8_t *input, uint32_t count, uint32_t elem_size) {
    // count=0x10000, elem_size=0x10000 -> product 0x100000000, truncates
    // to 0 in a 32-bit int.
    int total = count * elem_size;    // signed 32-bit overflow (UB)
    char *buf = malloc(total);        // malloc(0) -- tiny/zero alloc
    if (!buf) return -1;

    // Copy loop trusts the ORIGINAL count/elem_size, not the wrapped total
    for (uint32_t i = 0; i < count; i++) {
        memcpy(buf + i * elem_size, input + i * elem_size, elem_size);
        // Writes past the tiny allocation -- classic heap overflow.
    }
    return 0;
}
C

A signed/truncation variant: a length read as a signed int, never checked against the actual remaining buffer, trusted for both the allocation and a subsequent copy:

// VULNERABLE: signed 'len' from network input, no bounds/sign check
void handle_chunk(uint8_t *packet, int32_t len) {
    // Crafted packet sets len = -1 (0xFFFFFFFF as unsigned).
    char *buf = malloc(len);
    // malloc((size_t)-1) usually fails cleanly on 64-bit -- but if len is
    // first added to a constant (len + 8), the wrap can land on a small,
    // attacker-chosen allocation size instead.
    memcpy(buf, packet, len);   // len reinterpreted as huge size_t -> OOB
}
C

Type confusion via an unchecked C++ downcast — no runtime type tag prevents treating a Cat as a Dog if the cast site’s assumption is wrong:

struct Animal { virtual ~Animal() = default; };
struct Cat : Animal { int lives = 9; };
struct Dog : Animal { void (*bark)() = default_bark; };  // function pointer!

// VULNERABLE: static_cast trusts caller-supplied type with no runtime check
// (dynamic_cast would return nullptr on mismatch instead)
void handle(Animal *a, bool is_dog) {
    if (is_dog) {
        Dog *d = static_cast<Dog*>(a);   // no verification 'a' IS a Dog
        d->bark();  // if 'a' is really a Cat, this calls through
                    // Cat::lives (an int) reinterpreted as a function
                    // pointer -> hijacked control flow.
    }
}
C++

Walkthrough / Exploitation

Turning the multiplication-overflow example into a heap overflow means controlling both inputs so the product wraps to a small, nonzero size:

1. Find the vulnerable size expression via source review or by fuzzing
   parse_records() with libFuzzer/AFL++ -- exactly what ASan+UBSan
   (-fsanitize=address,undefined) catch fast.
2. Solve for count * elem_size == small_value (mod 2^32), e.g.
   count = 0x10001, elem_size = 0x10000 -> product 0x100010000
   truncates to 0x10000 -- a 64KB allocation instead of ~4GB.
3. Craft the header with those values; the tiny allocation now backs a
   copy loop that still runs 'count' iterations of 'elem_size' bytes,
   overflowing adjacent heap metadata or neighboring chunks.
4. Groom the heap beforehand (allocate/free matching-size chunks) so
   attacker-useful data -- a function pointer table, a vtable, another
   chunk's size field -- lands right after the undersized buffer.
TEXT

For the type-confusion example, exploitation is more direct: overlapping Dog::bark‘s function-pointer offset with Cat::lives means the attacker only needs to control the integer in lives to redirect execution — no separate corruption bug required, the type confusion *is* the primitive:

// If the attacker controls Cat::lives (e.g. via a scriptable API) before
// the object reaches handle(a, /*is_dog=*/true):
Cat *c = new Cat();
c->lives = (int)(uintptr_t)shellcode_addr;   // attacker-controlled field
handle(c, true);   // treated as Dog -> d->bark() calls shellcode_addr
C++

Note: Not every overflow is exploitable — many are caught by the allocator returning NULL on absurd sizes, or by the wrap producing zero. The interesting cases land in between: a wrap that produces a small but nonzero, attacker-chosen value, exactly what the multiplication example demonstrates. In JS engines, type confusion is usually found through JIT deoptimization edge cases rather than plain casts — e.g. a valueOf callback that shrinks an array mid-optimized-execution; the C++ downcast example here illustrates the same *class* of bug in a simpler context.

Detection and Defense

These bug classes are unusually amenable to automated tooling, which is why modern pipelines lean on sanitizers rather than manual review alone:

  • UBSan (-fsanitize=undefined) — catches signed overflow and other undefined-behavior arithmetic at the point it happens.
  • AddressSanitizer (ASan) — catches the resulting out-of-bounds write once the undersized buffer is overrun, with a precise crash report.
  • Checked arithmetic intrinsics — __builtin_mul_overflow/__builtin_add_overflow (GCC/Clang) or SafeInt/CheckedNumeric (Chromium) make overflow an explicit, checked branch instead of silent UB.
  • dynamic_cast/RTTI instead of static_cast/reinterpret_cast across polymorphic hierarchies, plus CFI (-fsanitize=cfi) for indirect calls.
  • Fuzzing with sanitizers attached (libFuzzer/AFL++ + ASan/UBSan) on any parser computing sizes from external input.
  • Strong typing — prefer size_t consistently, avoid mixing signed and unsigned in one expression, enable -Wconversion -Wsign-compare as errors.

Real-World Impact

Integer overflows feeding undersized heap allocations are among the most recurring root causes in browser and image/font-parser vulnerability reports, because size calculations from untrusted headers are everywhere in file-format and codec parsing. Type confusion is likewise a dominant primitive in JavaScript-engine exploitation (V8, JavaScriptCore, SpiderMonkey), where JIT type-speculation bugs recur in browser exploit chains and Pwn2Own submissions, since a type-confusion primitive typically yields arbitrary read/write without a separate corruption bug.

Conclusion

Both bug classes share the same root cause: code trusts a value — a size, a type tag, an object shape — that was computed once and silently became wrong before use. Checked arithmetic, sanitizer-instrumented fuzzing, and runtime type verification close off the overwhelming majority of real-world instances, which is why they are now default expectations for any parser or runtime handling untrusted input.

You Might Also Like

If you found this useful, these related deep-dives cover adjacent techniques and their defenses:

Comments

Copied title and URL