x86 calling conventions

This article describes the calling conventions used on x86 architecture chips.

Calling conventions describe the interface of called code:

• The order in which atomic (scalar) parameters, or individual parts of a complex parameter, are allocated
• How parameters are passed (pushed on the stack, placed in registers, or a mix of both)
• Which registers the callee must preserve for the caller
• How the task of preparing the stack for, and restoring after, a function call is divided between the caller and the callee

This is intimately related with the assignment of sizes and formats to programming-language types. Another closely related topic is name mangling, which determines how symbol names in the code map to symbol names used by the linker. Calling conventions, type representations, and name mangling are all part of what is known as an Application Binary Interface (ABI).

There are often subtle differences in how various compilers implement these conventions, so it is often difficult to interface code which is compiled by different compilers. On the other hand, conventions which are used as an API standard (such as stdcall) are very uniformly implemented.

Historical background[edit]

Prior to microcomputers, the machine manufacturer generally provided an operating system and compilers for several programming languages. The calling conventions adopted for the platform were those defined by the manufacturer's software implementation.

Early microcomputers before Apple II Computers generally came "bare" of an OS or compilers, as did the IBM PC. The only hardware standard for IBM PC compatiblemachines was defined by the Intel processors (8086, 80386) and the literal hardware IBM shipped. Hardware extensions and all software standards (save for a BIOScalling convention) were thrown open to market competition.

A multitude of independent software firms offered operating systems, compilers for many programming languages, and applications. Many different calling schemes were implemented by the firms, often mutually exclusive, based on different requirements, historical practices, and programmer creativity.

After the IBM compatible market shakeout, Microsoft operating systems and programming tools (with differing conventions) predominated, while second tier firms like Borland and Novell, and open source projects like GCC, still maintained their own standards. Provisions for inter-operability between vendors and products were eventually adopted, simplifying the problem of choosing a viable convention.^[1]

Caller clean-up[edit]

In these conventions the caller cleans the arguments from the stack, which allows for variable argument lists; e.g., printf().

cdecl[edit]

The cdecl (which stands for C declaration) is a calling convention that originates from the C programming language and is used by many C compilers for the x86 architecture.^[1] In cdecl, subroutine arguments are passed on the stack. Integer values and memory addresses are returned in the EAX register, floating point values—in the ST0 x87 register. Registers EAX, ECX, and EDX are caller-saved, and the rest are callee-saved. The x87 floating point registers ST0 to ST7 must be empty (popped or freed) when calling a new function, and ST1 to ST7 must be empty on exiting a function.

In context of the C programming language, function arguments are pushed on the stack in the reverse order. In GNU/Linux, GCC sets the de facto standard for calling conventions. Since GCC version 4.5, the stack must be aligned to a 16-byte boundary when calling a function (previous versions only required a 4-byte alignment.)^{[citation needed]}

Consider the following C source code snippet:

int callee(int, int, int);
 
int caller(void)
{
        register int ret;
 
        ret = callee(1, 2, 3);
        ret += 5;
        return ret;
}

On x86, it will produce the following assembly code (AT&T syntax):

        .globl  caller
caller:
        pushl   %ebp
        movl    %esp,%ebp
        pushl   $3
        pushl   $2
        pushl   $1
        call    callee
        addl    $12,%esp
        addl    $5,%eax
        leave
        ret

The calling function cleans the stack after the function call returns.

There are some variations in the interpretation of cdecl,^[2] particularly in how to return values. As a result, x86 programs compiled for different operating system platforms and/or by different compilers can be incompatible, even if they both use the "cdecl" convention and do not call out to the underlying environment. Some compilers return simple data structures with a length of 2 registers or less in the register pair EAX:EDX, and larger structures and class objects requiring special treatment by the exception handler (e.g., a defined constructor, destructor, or assignment) are returned in memory. To pass "in memory", the caller allocates memory and passes a pointer to it as a hidden first parameter; the callee populates the memory and returns the pointer, popping the hidden pointer when returning.

In Linux/GCC, double/floating point values should be pushed on the stack via the x87 pseudo-stack. Like so:

        sub esp, 8      ; make room for the double
        fld [ebp + x]   ; load our double onto the floating point stack
        fstp [esp]      ; push our double onto the stack
        call funct
        add esp, 8

Using this method ensures it is pushed on the stack in the correct format.

The cdecl calling convention is usually the default calling convention for x86 C compilers, although many compilers provide options to automatically change the calling conventions used. To manually define a function to be cdecl, some support the following syntax:

void _cdecl funct();

The _cdecl modifier must be included in the function prototype, and in the function declaration to override any other settings that might be in place.