Question

On my Linux x86_64 host, I am trying to cross-compile some additional Python modules for my PowerPC target, specifically, greenlet, gevent, and gevent-websockets. Currently, I am stuck just trying to cross-build the greenlet module.

Using info from this site:

http://randomsplat.com/id5-cross-compiling-python-for-embedded-linux.html

I was able to cross-compile Python 2.7.2 using this setup for my build environment

# Undo variables for cross-compile environment
unset ROOT
unset SDKDIR
unset KLIBDIR
unset NFSDIR
unset CONFIG
unset CONFIGURED
unset ARCH
unset OS
unset TOOLCHAIN_BASE
unset TOOLCHAIN_BIN
unset CROSS_COMPILE
unset c
unset KERNEL_DIR
unset AS
unset LD
unset CC
unset AR
unset STRIP
unset SSTRIP
unset OBJCOPY
unset OBJDUMP
unset MAKE
unset CFLAGS

# Set cross-compile variables:
export TOOLCHAIN=/opt/freescale/usr/local/gcc-4.3.74-eglibc-2.8.74-dp-2/powerpc-none-linux-gnuspe/bin/powerpc-none-linux-gnuspe-
export CC=${TOOLCHAIN}gcc
export CXX=${TOOLCHAIN}g++
export AR=${TOOLCHAIN}ar
export RANLIB=${TOOLCHAIN}ranlib
export BLDSHARED="${TOOLCHAIN}gcc -shared"
export LDSHARED="${TOOLCHAIN}gcc -shared"
export RFS="../../ltib/rootfs"
export CFLAGS="-save-temps -Wall -I${RFS}/usr/include -I${RFS}/include/python2.7 -L${RFS}/usr/lib -L${RFS}/lib"
export LDFLAGS="-I${RFS}/usr/include -I${RFS}/include/python2.7 -L${RFS}/usr/lib -L${RFS}/lib"
export CROSS_COMPILE=ppc-linux
export CROSS_COMPILE_TARGET=yes
export HOSTARCH=ppc-linux
export BUILDARCH=x86_64-linux-gnu

Configuring my environment with the above script and then trying to build the greenlet module yields:

$ python ./setup.py build
running build
running build_ext
building 'greenlet' extension
creating build
creating build/temp.linux-x86_64-2.7
/opt/freescale/usr/local/gcc-4.3.74-eglibc-2.8.74-dp-2/powerpc-none-linux-gnuspe/bin/powerpc-none-linux-gnuspe-gcc -I../../../ltib/rootfs/usr/include -L../../../ltib/rootfs/usr/lib -L../../../ltib/rootfs/lib -fPIC -I/usr/include/python2.7 -c greenlet.c -o build/temp.linux-x86_64-2.7/greenlet.o
In file included from /usr/include/python2.7/Python.h:58,
                 from greenlet.h:8,
                 from greenlet.c:5:
/usr/include/python2.7/pyport.h:849:2: error: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."
error: command '/opt/freescale/usr/local/gcc-4.3.74-eglibc-2.8.74-dp-2/powerpc-none-linux-gnuspe/bin/powerpc-none-linux-gnuspe-gcc' failed with exit status 1

Why is setup.py pulling from /usr/include/python2.7 on my host system? I can’t find that dir on my target. How can I create it for my target?

Any suggestions?

Thanks!

Trevor

UPDATE #1:

My relative references to my host’s copy of the target’s rootfs were incorrect. Correcting it and rerunning yields:

$ python ./setup.py build
running build
running build_ext
building 'greenlet' extension
creating build
creating build/temp.linux-x86_64-2.7
/opt/freescale/usr/local/gcc-4.3.74-eglibc-2.8.74-dp-2/powerpc-none-linux-gnuspe/bin/powerpc-none-linux-gnuspe-gcc -save-temps -Wall -I../../ltib/rootfs/usr/include -I../../ltib/rootfs/include/python2.7 -L../../ltib/rootfs/usr/lib -L../../ltib/rootfs/lib -fPIC -I/usr/include/python2.7 -c greenlet.c -o build/temp.linux-x86_64-2.7/greenlet.o
greenlet.s: Assembler messages:
greenlet.s:832: Error: syntax error; found `(' but expected `,'
greenlet.s:832: Error: junk at end of line: `(31),1'
error: command '/opt/freescale/usr/local/gcc-4.3.74-eglibc-2.8.74-dp-2/powerpc-none-linux-gnuspe/bin/powerpc-none-linux-gnuspe-gcc' failed with exit status 1

At least it is finding more of my target’s include libraries, but now I am really stumped! 🙁

Any more suggestions?

Thanks!

UPDATE #2:

By adding the -save-temps flag to the compiler (updated error above), I was able to save and examine the intermediate assembler code that was mentioned in the above error message. The broken lines are:

#APP
 # 52 "platform/switch_ppc_linux.h" 1
    mr 8(31), 1
 # 0 "" 2

The MR (move register) op is fairly simple, accepting only 2 args (mr to-reg, from-reg). I don’t know how the parenthesis with additional register number were added. FWIW, here is the referenced macro in the above header file:

#define STACK_REFPLUS 1

#ifdef SLP_EVAL

#define STACK_MAGIC 3

/* !!!!WARNING!!!! need to add "r31" in the next line if this header file
 * is meant to be compiled non-dynamically!
 */
#define REGS_TO_SAVE "r13", "r14", "r15", "r16", "r17", "r18", "r19", "r20", \
       "r21", "r22", "r23", "r24", "r25", "r26", "r27", "r28", "r29", \
       "cr2", "cr3", "cr4"
static int
slp_switch(void)
{
    register int *stackref, stsizediff;
    __asm__ volatile ("" : : : REGS_TO_SAVE);
    __asm__ ("mr %0, 1" : "=g" (stackref) : );
    {
        SLP_SAVE_STATE(stackref, stsizediff);
        __asm__ volatile (
            "mr 11, %0\n"
            "add 1, 1, 11\n"
            "add 30, 30, 11\n"
            : /* no outputs */
            : "g" (stsizediff)
            : "11"
            );
        SLP_RESTORE_STATE();
    }
    __asm__ volatile ("" : : : REGS_TO_SAVE);
    return 0;
}

#endif

I am starting to wonder if this is a bug in the compiler, because the macro seems simple enough! Any suggestions? … Thanks!

Answer 1

Maybe you should have asked a new question, because there really are (at least) two completely independent problems here. But, looking at your second problem:

__asm__ ("mr %0, 1" : "=g" (stackref) : );

This is wrong. I’ll explain why below, but first, the following change will likely fix it:

__asm__ ("mr %0, 1" : "=r" (stackref) : );

You may also need to change the "g" (stsizediff) below to "r" (stsizediff).

So, what’s wrong with the existing version? First, look at how stackref is defined:

register int *stackref, stsizediff;

The register is a hint to the compiler, saying you think it might make things faster or better if it allocated a register for stackref instead of using a stack location, not a requirement. If stackref ends up in R12, great; if it ends up 8 bytes into the stack frame, that’s fine too. Either is perfectly legal, as long as it doesn’t violate any constraints.

So, what constraints are there on stackref? The only one is in that asm block quoted above. You’ve got "=g" (stackref) as an output operand. The = means it’s a write-only constraint, and the g means it must be in a register, memory location, or immediate value.

So the compiler’s not doing anything wrong. It allocates stackref 8 bytes off the stack, which matches the constraint (that’s a memory location), then it substitutes that value in for the "%0", and you get:

mr 8(31), 1

Nothing wrong with that—until you try to assemble it, and the assembler notices that you’re trying to use 8(31) with an opcode that only takes registers. But the problem isn’t the compiler, or the assembler, it’s the code. You asked it to use stackref as an operand for mr, and didn’t force stackref to be a register, so you got what you asked for.

Anyway, changing the "=g" to "=r" changes the constraint from “any register, memory location, or immediate value” to “any general register”. That means the compiler has to put stackref in a general register. Or, if it can’t for some reason, it will fail and tell you why, instead of generating assembly that won’t assemble.

So, why did this work for the original author? Well, he probably got lucky, and stackref got allocated to, say, R12, instead of 8 bytes into the stack frame, so he ended up with mr 12, 1, which assembles just fine.

Or, one more possibility. Looking at the git tree, it looks like the code was developed on Mac OS X, then ported to AIX (by people who are primarily Mac developers) a decade ago, then copied verbatim from AIX to linux (even leaving the description “Port for AIX on PowerPC”), and not touched significantly since. Both OS X and AIX only had gcc 3 back then. So, maybe that’s the reason it worked for everyone at the time, and isn’t working for you. And maybe just getting an older cross-compiler will solve your problem. But I’d try fixing the code first.