x86 calling conventions

Calling conventions, type representations, and name mangling are all part of what is known as an application binary interface (ABI).

Hardware extensions and all software standards (save for a BIOS calling convention) were thrown open to market competition.

A multitude of independent software firms offered operating systems, compilers for many programming languages, and applications.

After the IBM compatible market shakeout, Microsoft operating systems and programming tools (with differing conventions) predominated, while second-tier firms like Borland and Novell, and open-source projects like GNU Compiler Collection (GCC), still maintained their own standards.

Provisions for interoperability between vendors and products were eventually adopted, simplifying the problem of choosing a viable convention.

Some compilers return simple data structures with a length of 2 registers or less in the register pair EAX:EDX and larger structures and class objects requiring special treatment by the exception handler (e.g., a defined constructor, destructor, or assignment) are returned in memory.

However, EAX, ECX, and EDX are not preserved, and the size of the parameter list in doublewords is passed in AL. Syscall is the standard calling convention for 32 bit OS/2 API.

Functions which use these conventions are easy to recognize in ASM code because they will unwind the stack after returning.

The x86 ret instruction allows an optional 16-bit parameter that specifies the number of stack bytes to release after returning to the caller.

Such code looks like this: Conventions named fastcall or register have not been standardized, and have been implemented differently, depending on the compiler vendor.

Returning the result works as follows: This calling convention was common in the following 16-bit APIs: OS/2 1.x, Microsoft Windows 3.x, and Borland Delphi version 1.x.

Modern versions of the Windows API use stdcall, which still has the callee restoring the stack as in the Pascal convention, but the parameters are now pushed right to left.

Microsoft __fastcall convention (aka __msfastcall[citation needed]) passes the first two arguments (evaluated left to right) that fit, into ECX and EDX.

In Visual Studio 2013, Microsoft introduced the __vectorcall calling convention in response to efficiency concerns from game, graphic, video/audio, and codec developers.

The scheme allows for larger vector types (float, double, __m128, __m256) to be passed in registers as opposed to on the stack.

[10] For IA-32 and x64 code, __vectorcall is similar to __fastcall and the original x64 calling conventions respectively, but extends them to support passing vector arguments using SIMD registers.

In x64, however, the rule from the original x64 convention still apply, so that XMM/YMM0-5 only hold floating-point, vector, or HVA arguments when they happen to be the first through the sixth.

[17] GCC and Clang can be made to use a similar calling convention by using __stdcall with the regparm function attribute or the -mregparm=3 switch.

In Delphi and Free Pascal on Microsoft Windows, the safecall calling convention encapsulates COM (Component Object Model) error handling, thus exceptions aren't leaked out to the caller, but are reported in the HRESULT return value, as required by COM/OLE.

The safecall calling convention is the same as the stdcall calling convention, except that exceptions are passed back to the caller in EAX as a HResult (instead of in FS:[0]), while the function result is passed by reference on the stack as though it were a final "out" parameter.

For the GCC compiler, thiscall is almost identical to cdecl: The caller cleans the stack, and the parameters are passed in right-to-left order.

According to the Intel ABI to which the vast majority of compilers conform, the EAX, EDX, and ECX are to be free for use within a procedure or function, and need not be preserved.

[citation needed] As the name implies, these general-purpose registers usually hold temporary (volatile) information, that can be overwritten by any subroutine.

In other words, when the caller makes a procedure call, it can expect that those registers will hold the same value after the callee returns.

Thus, making it the callee's responsibility to both save (push at the start) and restore (pop accordingly) them before returning to the caller.

[23] When compiling for the x64 architecture in a Windows context (whether using Microsoft or non-Microsoft tools), stdcall, thiscall, cdecl, and fastcall all resolve to using this convention.

So when the called function is entered, the stack will be composed of (in ascending order) the return address, followed by the shadow space (32 bytes) followed by the fifth parameter.

The OpenVMS Calling Standard on x86-64 is based on the System V ABI with some extensions needed for backwards compatibility.

[28]: 27 If the callee wishes to use registers RBX, RSP, RBP, and R12–R15, it must restore their original values before returning control to the caller.

[28]: 55 Unlike the Microsoft calling convention, a shadow space is not provided; on function entry, the return address is adjacent to the seventh integer argument on the stack.