How to Build Cmake Externalproject While Configurating Main One

How to build cmake ExternalProject while configurating main one?

You may use cmake call within execute_process for configure and build CMake project, which contains ExternalProject:

other_project/CMakeLists.txt:

project(other_project)
include(ExternalProject)

ExternalProject_Add(<project_name> <options...>)

CMakeLists.txt:

# Configure external project
execute_process(
    COMMAND ${CMAKE_COMMAND} ${CMAKE_SOURCE_DIR}/other_project
    WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/other_project
)

# Build external project
execute_process(
    COMMAND ${CMAKE_COMMAND} --build ${CMAKE_BINARY_DIR}/other_project
)

Such a way other_project will be configured and built in directory ${CMAKE_BINARY_DIR}/other_project. If you do not disable installation in ExternalProject_Add call, then it will performed when building other_project.

Normally, you want some options to ExternalProject, like SOURCE_DIR, BINARY_DIR, INSTALL_DIR, to be deduced from variables in the main project. You have two ways for achive that:

1. Create CMakeLists.txt for other_project with configure_file, called from main project (before execute_process command).

2. Pass variables from main project as -D parameters to ${CMAKE_COMMAND}.

---

Having separated execute_process calls for sequential COMMANDS is important. Otherwise, if use single execute_process with several COMMANDS, these commands will be just "piped" (executed concurrently but with output of the first command being treated as input for the second).

Force building external project (with buildtools) before main project with CMake

After you define your main library/executable with add_library/add_executable, set gsl_local as a dependency of your project using the add_dependencies command (link).

add_dependencies(moosebin gsl_local)

Note that "moosebin" here is the name of the target you create with add_library or add_executable, which is not necessarily the same as what you define with project().

Installing an ExternalProject with CMake

Command flow install(TARGETS) installs targets, which are created with add_executable or add_library commands. For install concrete files, you need to use command flow install(FILES).

Instead of installing selected files from subproject's build directory, you may install subproject as is. For that you can use

make DESTDIR=<...> install

command as INSTALL option of ExeternalProject_Add. This command will install whole subproject into directory given as DESTDIR parameter for make (more information about this parameter can be found here).

Then you may use install(DIRECTORY) command for install all subproject's files at once:

# It is better to use binary directory for download or build 3d-party project 
set(sphinxbase_SOURCE_DIR ${CMAKE_CURRENT_BINARY_DIR}/lib/sphinxbase)
# This will be used as DESTDIR on subproject's `make install`.
set(sphinxbase_DESTDIR ${CMAKE_CURRENT_BINARY_DIR}/lib/sphinxbase_install)

ExternalProject_Add(
    sphinxbase
    GIT_REPOSITORY      "https://github.com/cmusphinx/sphinxbase.git"
    GIT_TAG             "e34b1c632392276101ed16e8a05862e43f038a7c"
    SOURCE_DIR          ${sphinxbase_SOURCE_DIR}
    # Specify installation prefix for configure.sh (autogen.sh).
    CONFIGURE_COMMAND   ./autogen.sh --prefix=${CMAKE_INSTALL_PREFIX}
    BUILD_COMMAND       ${MAKE}
    UPDATE_COMMAND      ""
    # Fake installation: copy installed files into DESTDIR.
    INSTALL_COMMAND     make DESTDIR=${sphinxbase_DESTDIR} install
    ...
)
# Actually install subproject.
install(
    DIRECTORY ${sphinxbase_DESTDIR}/
    DESTINATION "/"
    USE_SOURCE_PERMISSIONS # Remain permissions (rwx) for installed files
)

Note, that root directory is used as DESTINATION. This is because files under sphinxbase_DESTDIR directory already located in accordance to CMAKE_INSTALL_PREFIX, which is passed as --prefix option to configure.sh (autogen.sh in the given case).

If you know, that subprojects installs its files only under installation prefix (given by --prefix), you can make main project installation more packaging-friendly:

install(
    DIRECTORY ${sphinxbase_DESTDIR}/${CMAKE_INSTALL_PREFIX}/
    DESTINATION "."
    USE_SOURCE_PERMISSIONS
)

Because now relative path is used for DESTINATION option, the project will correctly support packaging. For example, make DESTDIR=<...> install will work for main project too.

---

Note, that even if ExternalProject_Add is used for build CMake subproject, targets created in this subroject are not seen by the main project.

Build external library only once with CMake

As @Tsyvarev said, in your case ExternalProject_Add is better than add_subdirectory. add_subdirectory is good when you want project to be essential part of your build system because target it creates can be used in in the right-hand-side of the target_link_libraries() command while target created by ExternalProject_Add cannot.

Here is the approach I used in one of my projects. You try to find required library and build it only if it was not found. I use INTERFACE library to turn FOO_EXTERNAL into a target acceptable by target_link_libraries().

add_library(foo INTERFACE)
find_package(foo ${FOO_VER})
if(NOT foo_FOUND)
    include(ExternalProject)
    include(GNUInstallDirs)
    ExternalProject_Add(FOO_EXTERNAL
                    SOURCE_DIR "${FOO_SOURCE_DIR}"
                    BINARY_DIR "${FOO_BINARY_DIR}"
                    INSTALL_DIR "${FOO_INSTALL_DIR}"
                    CMAKE_ARGS "-DCMAKE_C_FLAGS=${CMAKE_C_FLAGS}"
                               "-DCMAKE_CXX_FLAGS=${CMAKE_CXX_FLAGS}"
                               "-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}"

                               "-DCMAKE_INSTALL_PREFIX=${FOO_INSTALL_DIR}"
                    )

    add_dependencies(foo FOO_EXTERNAL)
    set(foo_LIBRARY
            "${FOO_INSTALL_DIR}/${CMAKE_INSTALL_LIBDIR}/${CMAKE_STATIC_LIBRARY_PREFIX}foo${CMAKE_STATIC_LIBRARY_SUFFIX}")
    set(foo_INCLUDE_DIR "${FOO_INSTALL_DIR}/include")
endif()

target_link_libraries(foo INTERFACE ${foo_LIBRARY})
target_include_directories(foo INTERFACE ${foo_INCLUDE_DIR})

CMake - linking to library downloaded from ExternalProject_add()

When you're using ExternalProject_Add, you can't use find_package, since there's nothing to find when CMake runs to configure the outer project.

So, if library locations vary by platform you will need conditional logic based on your platform. (I don't know protobuf's libraries or structure here, so this is just an example, but it should get you headed in the right direction...) Something like this:

if(WIN32)
  set(PROTOBUF_LIB_DIR "${MYPROJ_SOURCE_DIR}/dependencies/win"
  set(prefix "")
  set(suffix ".lib")
elseif(APPLE)
  set(PROTOBUF_LIB_DIR "${MYPROJ_SOURCE_DIR}/dependencies/mac"
  set(prefix "lib")
  set(suffix ".a")
else()
  set(PROTOBUF_LIB_DIR "${MYPROJ_SOURCE_DIR}/dependencies/linux"
  set(prefix "lib")
  set(suffix ".a")
endif()

set(PROTOBUF_LIBRARIES
  "${PROTOBUF_LIB_DIR}/${prefix}protobufLib1${suffix}"
  "${PROTOBUF_LIB_DIR}/${prefix}protobufLib2${suffix}")

Granted, this is less convenient than using find_package. If you can use a pre-built/pre-installed package, you should, so that you can use find_package. If you must build the other package from source code as part of your project, though, ExternalProject_Add is useful, even though it is unable to abstract away all the details for you.

ExternalProject_Add autogen project prevent configure on rebuild

The only way I know is to move the steps to build the external project into a script called by add_command. And then add an dependency between your project and the external project's output/lib/binary.

• Pro: Avoids unnecessary rebuilds
• Pro: Avoids unexpected updates of the external lib
• Con: Does not rebuild in case the external repository was updated (and this is required)
• Con: Additional work to write such a script

BTW: Within my project, we do it the hard way. But we do not want any unexpected updates of external libs.

How to build executable with a compiler from ExternalProject

Short of reverse engineering CMake's enable_language implementation modules to add your own for Prolog, you can always go the route of add_custom_command and ask your compiler to generate object files, which can be included as sources in the add_executable command.

If you add some detail to the question about how the Prolog compiler works, I can help you write the custom commands.

CMake declare dependency of function on ExternalProject_Add

I did start modifying my code based on comment posted by Tsyvarev:

CMake functions are executed at configure stage, so you need to build the external project at configure stage too.

While I trusted that his proposed solution would work, I was slightly uncomfortable and kept thinking that there has to be a more elegant solution. I consulted with a colleague and came up with a better solution which is as simple as the following diff (which removes ${flatbuffers_GENERATED_CXX}).

- add_dependencies(my_framework flatbuffers ${flatbuffers_GENERATED_CXX})
+ add_dependencies(my_framework flatbuffers)

we reviewed that the problem with the code in question is that as is, CMake reads add_dependencies(my_framework flatbuffers ${flatbuffers_GENERATED_CXX}) and understands that it needs ${flatbuffers_GENERATED_CXX} as target to build my_framework so it proceeds with running the function. But there is no way for it to understand that the function depends on the external project. Now if we remove the explicit dependency declaration of ${flatbuffers_GENERATED_CXX}, CMake defers running the function to after resolving other dependencies (flatbuffers target) which will effectively download and build the external project prior to running the project.

---

Related Topics