Question

I’ve got libraries that do no expose any function except say “CreateObject”. Nevertheless all their functions are called indirectly, so I see in the perf report that up to 1.65% of time is spent in __i686.get_pc_thunk.bx. The functions (class methods) are called 160 million times and they are internal to the shared library, i.e. not exposed.

I wonder if it’s possible to compile internal methods without relocations – i.e. using relative offsets or something like that.

gcc is 4.5.2

UPDATE: Actually I think that was because of -O0 left in the makefile. So it’s not a big deal now, but I would still like to do the same with -O0, too, as it keeps less “garbage” for profiler. I wonder what is the -O2 “real” option that does this.

UPDATE2: hm, it wasn’t -O2, it was probably –dynamic-list that lower the pc_thunk performance hit a bit, but it’s still there… so not even sure if –dynamic-list really helps. Should hidden symbols still include indirection thunks, is it correct?

UPDATE3: I created a test project, for internal library function I set attribute visibility hidden, I compile with gcc 4.7 and -O2 and LTO enabled, I pass –dynamic-list to linker without the internal function in there, and nevertheless the call to get_pc_thunk is still there.

This is the code in the test shared library:

#include <stdio.h>

__attribute__((visibility("hidden"), noinline)) void lib1f2()
{
    puts("I should have PLT disabled");
}

void lib1f()
{
    puts("I'm lib1");
    lib1f2();
}

In gdb I still see thunk inside lib1f2.

What’s funny is that with -fwhole-program lib1f2 is inlined into the main executable but still contains this call to the thunk.

UPDATE4: OK I’m getting close (to realize me being dumb), the program (and code above) uses data even if it’s just a const string, so it needs GOT calls. So the question now is:

1. Still, can I avoid thunks for GOT?
2. (related) via, maybe, compiling without -fPIC – what will be drawbacks?

Answer 1

1) Still, can I avoid thunks for GOT?

I think not. Not on i686 at least. The problem is that code can automagically do relative jumps… or rather all the jumps on x86 are relative, beside indirect jump IIRC. On the other hand there is no way to index data relative to the current program counter. This problem is actually solved in x86_64, since there is a new instruction pointer relative addressing that can be used exactly for this cases.

Your test, compiled with gcc -fPIC -shared -O2 -flto

On 32 bit:

00000530 <lib1f2.2321>:
push   %ebx
call   52b <__x86.get_pc_thunk.bx>
add    $0x1aca,%ebx
sub    $0x18,%esp
lea    -0x1a67(%ebx),%eax
mov    %eax,(%esp)
call   3f0 <puts@plt>
add    $0x18,%esp
pop    %ebx
ret
00000560 <lib1f>:
push   %ebx
call   52b <__x86.get_pc_thunk.bx>
add    $0x1a9a,%ebx
sub    $0x18,%esp
lea    -0x1a4c(%ebx),%eax
mov    %eax,(%esp)
call   3f0 <puts@plt>
add    $0x18,%esp
pop    %ebx
jmp    530 <lib1f2.2321>
nop

On 64 bit

00000000000006b0 <lib1f2.2352>:
lea    0x2a(%rip),%rdi
jmpq   590 <puts@plt>

00000000000006c0 <lib1f>:
lea    0x35(%rip),%rdi
sub    $0x8,%rsp
callq  590 <puts@plt>
xor    %eax,%eax
add    $0x8,%rsp
jmp    6b0 <lib1f2.2352>

2) (related) via, maybe, compiling without -fPIC – what will be drawbacks?

Well, although it’s embarrassing I have to admit I’m slightly confused here. At first sight I would have said that a shared library bust be compiled with -fPIC. Instead, the following two commands both works

gcc -fPIC -m32 -shared -O2 -flto test.c -o test.so
gcc -m32 -shared -O2 -flto test.c -o test.so

In the non -fPIC case the code does also not need any call to get_pc_thunk. The trick is that the dynamic loader fixes the library code at runtime with the right address to the data.

This is a problem though, since you gained some speed avoiding the thunks, but you lost the ability to actually share the shared library since the operating system must create a new copy for every code page of the library which contains a relocation. On the other hand, when a GOT is used only the GOT page(s) must be duplicated, greatly reducing the memory footprint when many applications link to the same library.

Interestingly enough, in 64bit mode is not possible to compile a library in non pic mode, the following command fails

gcc -m64 -shared -O2 -flto test.c -o test.so

Still, since there is processor provided support for code relative addressing this is not a problem.