Question

I use CMake to create build scripts (Makefiles + VS solutions) for my projects. As best practice I create the build scripts in a separate folder (out of source). I build the projects in the same folder.

This works fine for compiled programs but I can’t find an adequate solution for my Python scripts as these have no build step that would copy (build) them to the build folder.

Looking for creative solutions….

Requirements:

• All executables should be available in the build folder post build (I consider *.py files to be executable
• Python scripts should be easily managed using an IDE (spyder, eclipse, etc)
• Source folder with python scripts is in Git repository. Build folder is not.
• C++ compiled python modules should reside next to relevant python scripts

So far I considered two options:

• Copy scripts to build folder when running CMake – Need to run CMake for every change in python files (IDE unfriendly). Can cause confusion: which copy of the sources to edit?
• Create links to source folder in build folder – Multi platform mess. Problem deploying compiled c++ python modules next to the scripts without polluting source folder.

I hope this is clear enough.

Answer 1

Eventually I found a solution which involves creating symbolic links to the python sources and other related files that are not compiled but are necessary in the build environment. To allow mixing built modules with the symbolic links I used real folders instead of symbolic links.

This way:

• There is one copy of the python scripts
• It can be run/edited seamlessly from the binary folder

Utility function:

function (create_symlinks)
    # Do nothing if building in-source
    if (${CMAKE_CURRENT_BINARY_DIR} STREQUAL ${CMAKE_CURRENT_SOURCE_DIR})
        return()
    endif()

    foreach (path_file ${ARGN})
        get_filename_component(folder ${path_file} PATH)

        # Create REAL folder
        file(MAKE_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/${folder}")

        # Delete symlink if it exists
        file(REMOVE "${CMAKE_CURRENT_BINARY_DIR}/${path_file}")

        # Get OS dependent path to use in `execute_process`
        file(TO_NATIVE_PATH "${CMAKE_CURRENT_BINARY_DIR}/${path_file}" link)
        file(TO_NATIVE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/${path_file}" target)

        if (UNIX)
            set(command ln -s ${target} ${link})
        else()
            set(command cmd.exe /c mklink ${link} ${target})
        endif()

        execute_process(COMMAND ${command} 
                        RESULT_VARIABLE result
                        ERROR_VARIABLE output)

        if (NOT ${result} EQUAL 0)
            message(FATAL_ERROR "Could not create symbolic link for: ${target} --> ${output}")
        endif()

    endforeach(path_file)
endfunction(create_symlinks)

Usage for a python module (inside CMakeLists.txt):

# Do not omit !!!RELATIVE!!!
file(GLOB_RECURSE files RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} *.py *.dat *.xml)
create_symlinks(${files})

Usage:

cd src_dir
mkdir build_dir
cd build_dir
cmake ..

IMPORTANT:

• When adding new files don’t forget to run cmake
• On Windows mklink support only some Windows versions
• On Windows mklink can be run as Administrator only. A workaround can be found here.
• Use relative paths when calling create_symlinks as this is how the directory structure is reconstructed in the binary folder.