Question

Can I use the assembly routines for Serpent encryption in the link below written for 32-bit x86 from a 64-bit program on an x86-64 machine? That is, without launching a separate 32-bit process for it? If not, does anyone have a pointer to an optimized implementation of Serpent that works in both 32 and 64 bit (LGPL is OK but cannot use GPL since it’s a commercial project)?
http://gladman.plushost.co.uk/oldsite/cryptography_technology/serpent/serpent.asm

Answer 1

You will need to convert the portions of the code that transfer the results and data to/from memory to use 64-bit address registers. Also, stack manipulation code will need to use the 64-bit stack registers. Other than that, it’s likely to work without major changes.