Question

Here’s the scenario I’m having:

I’ve created a debootstrap ubuntu maverick (64-bit) environment. I placed it at /env/mav/ on my ubuntu (64-bit) lucid system. I can chroot into /env/mav and can utilize a maverick system perfectly.

I can even use the lucid programs fine outside the chrooted environment. That is /env/mav/bin/ls will run.

However, I noticed that if I modify LD_LIBRARY_PATH to be /env/mav/lib [1] [2]

every single program (both lucid and maverick) I run will crash instantly. (e.g. ls results in a segfault). kern.log shows:

segfault at 7fece284aa18 ip 00007fece284aa18 sp 00007fff32028158 error 15

However, clearly if I chroot into /env/mav, every program is running fine. And aren’t all libraries just being read from the jailed (/env/mav) /lib? So what is the difference between chroot and modifying LD_LIBRARY_PATH in this context?

Furthermore, if I:

mount -B /env /env/mav/env

and then chroot /env and then set LD_LIBRARY_PATH to be /env/mav/lib, everything still runs fine.

I’m at at a loss for what’s going internally here. Are there some ld internals stored somewhere? Does chroot do something magical?

[1] Use case is to run programs from the maverick environment bound correctly to maverick dynamically linked libraries outside the maverick jail.

[2] This is just an abridged example. In reality /usr/lib, etc. are all included. Including the maverick environment’s /lib “poisons” everything; there are no problems using the other maverick library directories.

Answer 1

LD_LIBRARY_PATH is the option of ld-linux.so program/library. This library is a dynamick linker. It’s path “/lib/ld-linux.so.2” is hardcoded in (almost) all dynamically linked programms in ubuntu, in ELF header (INTREP field). I mean that, when Linux kernel runs binary file, it know nothing about special meaning of LD_LIBRARY_PATH.

So, when you run

 LD_LIBRARY_PATH=/env/mav/ /env/mav/bin/ls

the root system’s /lib/ld-linux.so.2 will be used and then it will try to resolve dynamic libraries using $LD_LIBRARY_PATH env variable (you can see what’s goind using LD_DEBUG=all env variable)

And when you do a chroot, maverick’s /lib/ld-linux.so.2 will be used.

I think, there can be some incompability between host system’s ld-linux and guest(maverick) system’s libc.so (because ld-linux is part of glibc/eglibc package and it uses something from libc.so).

To test my assumption, try to run (bash syntax of env variable setting):

 LD_LIBRARY_PATH=/env/mav/ /env/mav/lib/ld-linux.so.2 /env/mav/bin/ls

Here I try to start guest’s ld-linux manually, in order to overwrite INTREP hardcoded path (Yes, this seems rather unusual to run an .so library, but this library is the very special case and allows for this syntax). If this command will work, my assumption may be good. If not, there are other explainations possible.