Question

I have a closed-source 3rd-party shared library I need to link against. Unfortunately, the creators of the 3rd-party library didn’t bother to limit which symbols are exported and exported all symbols. The 3rd-party library internally uses an incompatible version of a popular library that I am using in my code but exports conflicting symbols (google’s protobuf library). This leads to a runtime error when the protobuffer library version check finds the compile time and run time versions of the library are not compatible. I can solve the problem by reverting to an older version of protobufs 2.3 that matches the version used in the 3rd party library. However, protbuf 2.3 has performance issues that make it unusable slow for my application. I need a way to use protobuf 2.4 in my code and let the 3rd-party library use it’s own internal v 2.3.

Is there a way to generate a new version of the 3rd party library that doesn’t export the symbols from the protobuf v 2.3 library used internally given only the so file? If I had the source, it would be an easier problem. It seems that tools like objcopy and strip can’t actually modify the dynamic symbol table. The only idea I have so far is to create my own shim library that exports only the symbols I need by redirecting calls to the 3rd-party library (opened with dlopen perhaps?).

Is there a better solution?

Answer 1

I found a solution that works… I created a shim library that redirects calls to the third-party library allowing code outside the library to see protbuf v2.4 symbols while code inside the third-party library sees protobuf v2.3 symbols. This workaround was based on the idea posted here: http://www.linuxjournal.com/article/7795

I had to modify the dlopen flags to include RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND. The RTLD_LOCAL flag keeps symbols inside the third-party library from being seen outside the shim library (prevents symbols leaking out). The RTLD_DEEPBIND forces calls from inside the 3rd party library to see only the internal version of the symbols (keeps symbols from leaking in).

To be concrete, here is an example excerpt from my shim library.

#include <stdio.h>
#include <stdint.h>
#include <dlfcn.h>
#include "libhdfs/hdfs.h"

//#define PRINT_DEBUG_STUFF

// Helper function to retrieve a function pointer to a function from libMapRClient
// while isolating the symbols used internally from those already linked externaly
// to workaround symbol collision problem with the current version of libMapRClient.
void* GetFunc(const char* name){
  #ifdef PRINT_DEBUG_STUFF
    printf("redirecting %s\n", name);
  #endif
  void *handle;
  char *error;

  handle = dlopen("/opt/mapr/lib/libMapRClient.so", RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND);

  if (!handle) {
    fputs(dlerror(), stderr);
    exit(1);
  }
  void* fp = dlsym(handle, name);
  if ((error = dlerror()) != 0) {
    fprintf(stderr, "%s\n", error);
    exit(1);
  }
  return fp;
}


hdfsFS hdfsConnect(const char* host, tPort port) {
  typedef hdfsFS (*FP) (const char* host, tPort port);
  static FP ext = 0;
  if (!ext) {
    ext = (FP)GetFunc("hdfsConnect");
  }
  return ext(host, port);
}


int hdfsCloseFile(hdfsFS fs, hdfsFile file) {
  typedef int (*FP) (hdfsFS fs, hdfsFile file);
  static FP ext = 0;
  if (!ext) {
    ext = (FP)GetFunc("hdfsCloseFile");
  }
  return ext(fs, file);
}

… and so on for the other public API functions