Question

I’m really only familiar with the C calling convention at this time (as described at Nasm – C calling convention). This same source also describes a Pascal style calling convention.

I am seeking to understand the various function calling conventions out there (description or sources)? Also what are the advantages or disadvantages of using one over another (if any)?

NOTE: Most interested in the x86 platform, but you’re welcome to describe others.

Answer 1

The big advantages to the C calling convention are:

1. handling of varargs. With the C calling convention, the caller pushes onto the stack and takes off — and the caller knows how many arguments they passed, since they’re doing the passing. Otherwise the callee has to figure it out somehow. They do anyway, of course, if they want to use the varargs, but why give them the chance to smash the stack if they get it wrong? Thus, languages that have the callee clean up tend not to support varargs.
2. your code is link-compatible with C (and most C++). You don’t hear about glibpascal much, do you? Didn’t think so.

Register based calling is useful too, and I think many C compilers do this to optimize rather than pushing integers/pointers on to the stack — but not for externally-visible functions in libraries, of course!

The whole point about conventions is that they’re conventional. It makes sense to follow the convention because then other people who obey the convention know what to expect. So really, don’t overthink it. This is a case where following the crowd yields tangible benefits.