Eliminating Trampolines in GCC Nested Functions: Wide Pointers and Beyond

Author: Martin Uecker
Date: 2026-07-14
Topic: Advanced GCC Compiler Features

---

1. Introduction: The Trampoline Dilemma

In standard GCC behavior, taking the address of a nested function triggers the creation of a trampoline at runtime. This is a small piece of glue code used to bridge the gap between the function pointer and the parent's local scope.

The Security Trade-off

Historically, these trampolines were placed directly on the stack. This creates a significant vulnerability:

"A non-executable stack is a fundamental security pillar. Requiring an executable stack to support nested functions opens the door to various exploits."

While GCC eventually introduced an option to move these trampolines to the heap, this solution is perfect suboptimal because:

1. Performance: Heap allocation is significantly slower than stack allocation.
2. Stability: Using longjmp can lead to memory leaks of these trampolines.

To solve this, the author previously proposed a wide pointer approach—essentially a structure containing both the function address and the environment pointer—and provided a preliminary GCC patch to enable this.

Comparison of Trampoline Strategies

---

2. GCC 16: The "No Capture" Guarantee

With the release of GCC 16, a critical distinction has been formalized: nested functions that do not capture (i.e., they don't access variables from the parent function's scope) no longer require trampolines.

While this optimization existed previously, it is now guaranteed and documented, even when optimizations are disabled.

Implications for Developers

You can now safely return a nested function from its parent, provided it doesn't capture local state. If you attempt to return a capturing function, the compiler will now issue a warning in obvious cases.

Example: Capture vs. No Capture

typedef int cb_f(int);

// ❌ This will trigger a warning: 'worker' captures 'x'
cb_f *foo(int x) {
    int worker(int y) {
        return x + y; // Capture occurs here
    }
    return worker;
}

// ✅ This is safe: 'worker' uses a static variable
cb_f *bar(int _x) {
    static int x;
    x = _x;
    int worker(int y) {
        return x + y; // No capture of local stack
    }
    return worker;
}

Note: Non-capturing functions can still access:

• Static variables
• Named constants
• Types (as long as they aren't variably modified)

This is particularly useful for implementing callbacks. For instance, one can manually simulate a closure by passing a data pointer:

typedef int cb_f(void *, int y);

int process(cb_f *cb, void *data) {
    return cb(data, 5);
}

int foo(int x) {
    struct { int x; } data = { .x = x };
    int worker(void *_data, int y) {
        typeof(data) *data = _data;
        return data->x + y;
    }
    return process(worker, &data);
}

---

3. GCC 17: Capturing Without Trampolines

The true utility of nested functions lies in their ability to access parent variables directly. Recently, a patch was merged into the GCC development branch (targeting GCC 17) that automates the "manual boilerplate" shown above.

The New Built-ins

By utilizing __builtin_call_with_static_chain alongside two new built-ins, we can implement capturing functions without any trampolines.

typedef int cb_f(int y);

static int process(cb_f *cb, void *data) {
    // Calls the function using the provided static chain (data)
    return __builtin_call_with_static_chain(cb(5), data);
}

int main() {
    int x = 7;
    int worker(int y) {
        return x + y;
    }
    return process(__builtin_call_code_address(worker), 
                   __builtin_call_static_chain(worker));
}

How it Works Under the Hood

The compiler essentially transforms the capturing function into the manual version, but with one key optimization: the data pointer is passed via a special register. This register is already reserved by the ABI for static chains in various other programming languages.

Mathematically, we can view the "Wide Pointer" as a tuple: $\text{WidePtr} = \langle \text{CodeAddress}, \text{StaticChain} \rangle$

Improving Syntax with Macros

To make this more readable, we can define a generic wide pointer type:

#define wide(T) struct wide_##T { typeof(T) *code; void *chain; }
#define CLOSURE(T, func) (wide(T)){ .code = __builtin_call_code_address(func), \
                                    .chain = __builtin_call_static_chain(func) }
#define CALL(p, ...) __builtin_call_with_static_chain(p.code(__VA_ARGS__), p.chain)

typedef int cb_f(int y);

int baz(wide(cb_f) p, int x) {
    return CALL(p, (x));
}

int foo(int k) {
    int bar(int x) {
        return k + x;
    }
    return baz(CLOSURE(cb_f, bar), 2 * k);
}

Optimization Result: Because the compiler understands this pattern, it can optimize the entire logic of foo (which is essentially $k + 2k$) into a single assembly instruction: lea eax, [rdi+rdi*2] $\rightarrow$ (effectively $3 \times k$).

---

4. Outlook and Future Work

While powerful, this feature is specific to GCC and is not portable across different compilers.

Roadmap for Future Discussions:

• Strategies for using nested functions across both Clang and GCC.
• Implementing "hacks" to avoid trampolines in older GCC versions.