Creating packages that work well with package managers

Table of Contents

For packages to work well with package managers, there are a few considerations:

But first some definitions:

1. Directories

Placing files in these directories will

  • Help packages coexist and not cause ambiguities
  • Help tools find things the package provides
  • Make packages easy to load by module files or other tools

1.1. bin

This is where executables of the package should go.

Files in this directory are used by executing them and loading a package should add this directory to the list PATH.

1.2. etc

1.3. etc/profile.d/${PACKAGE_NAME}.sh

This is where I put files that would need to be sourced to use the package.

Putting files in appropriate directories reduces the need for users to have to source a file to use our package.

1.4. include

This is where headers for C and C++ should be placed.

They are used by various build systems in various way that each end up causing -I ${PREFIX}/include to be added to compilation commands.

1.4.1. include/${PACKAGE_NAME}

To prevent ambiguities with similarly named headers, a subdirectory of include named like the package can be used.

I recommend still only adding -I ${PREFIX}/include to compilations rather than -I ${PREFIX}/include/${PACKAGE_NAME}. If we are using this technique to prevent ambiguities with names, then we would also not want code to be ambiguous. The include #include <${PACKAGE_NAME}/ambiguous_name.h> is less ambiguous than #include <ambiguous_name.h>

1.5. lib

This directorie's primary purpose is to hold shared and static library files.

These libraries are used by build systems in various ways that usually end up causing -L ${PREFIX}/lib and -lNAME to linking commands. CMake does its own search for library files and provides the result (the full path to the library file) to the linker.

Loading the package for runtime can be done by adding this directory to LD_LIBRARY_PATH.

1.5.1. lib/cmake

When doing find_package(${PACKAGE_NAME}), CMake will look in $X/lib/cmake for a file named ${PACKAGE_NAME}Config.cmake for each X in CMAKE_PREFIX_PATH.

This is one of many choices. The file can also be named ${PACKAGE_NAME@u}-config.cmake and can also be placed in a dozen other locations relative to directories X in CMAKE_INSTALL_PREFIX.

See section Build/CMake/Config Files for more details.

1.6. share

1.6.1. share/man

This directory contains manpages.

The manpages can be made available to the user in various ways.

The way I suggest is to simply add ${PREFIX}/bin to PATH.

1.6.2. share/bash-completion/completions

Put files in here that define completions for a command or shell function. The name of the file should be _ls, _ls.bash, ls, or ls.bash for definding the completion of a command named ls.

A file defining completion functions for many commands can be reused by placing symbolic links for each other command whose completion is provided by this file.

1.6.3. share/${PACKAGE_NAME}

Other static files used by the package should go in this directory (technically what should go in here is more precise than "everything else" but if you don't know where to put something, put it here.

For packages to coexist, if the package provides more that one file, all those files should go in a subdirectory of share named like the package.

This might seem redundant if you install packages each in their own PREFIX. For example, I might install the package named repos in the prefix $HOME/my-packages/repos and it will contain the directory $HOME/my-packages/repos/share/repos. It might seem weird to have the name of the package twice but this important for the package to be able to coexist with other packages.

It is also good to do it this way because then we follow FHS guidelines.

1.6.4. share/modulefiles

The initialization of lmod and the other one adds /usr/share/modulefiles to MODULE_PATH therefore it makes sense to put module files for things in our package in this directory.

2. Build

Files placed according to the above guidelines will be easily usable, all the things it provides will be available to users without fuss and without fail.

These guidelines will help make build systems of packages integrate well with package managers that build from source.

2.1. Git repository information

For packages to be distributable with source tarballs, there should be a way to disable anything that depends on the git repository.

2.2. Git submodules

There should be a way to disable the use of dependencies vendored with the main package. This allows the package manager to handle dependencies.

2.3. CMake

2.3.1. CMAKE_INSTALL_PREFIX

Tools may choose to do run cmake SRC_DIR -DCMAKE_INSTALL_PREFIX=... followed by make install or cmake -S SRC_DIR -B BUILD_DIR followed by cmake --install BUILD_DIR --prefix ${PREFIX}.

It is essential that both be supported if using cpack (make package) because it works by using cmake --install BUILD_DIR --prefix STAGING_AREA and making a tarball of STAGING_AREA.

Both should be supported. This means that evaluation of CMAKE_INSTALL_PREFIX should be deferred to the moment of installation.

The following evaluates CMAKE_INSTALL_PREFIX at configure time and does not support cmake --install BUILD_DIR --prefix ${PREFIX}:

# Doesn't work
install(CODE "message(STATUS \"installing into ${CMAKE_INSTALL_PREFIX}\")")

This is because CMake works by creating cmake_install.cmake at configuration time that is then run as a CMake script at install time. The evaluation at configure time causes the value of CMAKE_INSTALL_PREFIX from configure time to become hardcoded into this script:

# cmake_install.cmake
message(STATUS "installing into \usr\local"

The fix is very simple: just defer the evaluation until install time. This version (with a \ in front of the dollar sign:

# Works
install(CODE "message(STATUS \"installing into \${CMAKE_INSTALL_PREFIX}\")")

works because it the evaluation of ${CMAKE_INSTALL_PREFIX} will happen when cmake_install.cmake is run (instead of happening during its creation):

# cmake_install.cmake
message(STATUS "installing into ${CMAKE_INSTALL_PREFIX}")

In other install() commands, the value of CMAKE_INSTALL_PREFIX should not be used. Let CMake handle that. The following:

# don't do this
install(TARGETS my_exec DESTINATION ${CMAKE_INSTALL_PREFIX}/bin)
install(FILES something DESTINATION ${CMAKE_INSTALL_PREFIX}/share/${PACKAGE_NAME}/something)

is redundant because when the argument of DESTINATION is a relative path, it is interepreted relative to the installation directory.

It also breaks cpack, and prevents users and package managers from using cmake --install BUILD --prefix ${PREFIX}.

We should do this instead:

# do this
install(TARGETS my_exec DESTINATION bin)
install(FILES something DESTINATION share/${PACKAGE_NAME}/something)

2.3.2. Config files

Starting with ${PROJECT_NAME}Config.cmake.in containing

<<TODO>>

and list_of_targets containing a list of some or all of the project's targets, the code below define a name ${export_set_name} for a set of installed targets and the calls to configure_package_config_file and write_basic_package_version_file create $PREFIX/cmake/${PACKAGE_NAME}Config.cmake and $PREFIX/cmake/${PACKAGE_NAME}ConfigVersion.cmake

include(GNUInstallDirs) # Defines CONFIG_INSTALL_DIR as lib/cmake
include(CMakePackageConfigHelpers)

# Add `EXPORT ${export_set_name}` to `install(TARGETS` command
# to define ${export_set_name} and associate it with these targets.
install(TARGETS ${list_of_targets} EXPORT ${export_set_name})
# An `install(EXPORT ${export_set_name} ...)` tells CMake to generate files
# containing all the information on the targets in the previously created "export
# set" for client # projects built with CMake to consume.
install(
    EXPORT ${export_set_name}
    NAMESPACE ${PROJECT_NAME}::
    DESTINATION ${CONFIG_INSTALL_DIR}
)

# This creates =${PACKAGE_NAME}Config.cmake= which is the entry point from a
# =find_package= call.
configure_package_config_file(
    "${PROJECT_NAME}Config.cmake.in"
    "${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake"
    INSTALL_DESTINATION "${CONFIG_INSTALL_DIR}"
    PATH_VARS CMAKE_INSTALL_INCLUDEDIR CMAKE_INSTALL_LIBDIR
)
# Creates =${PACKAGE_NAME}ConfigVersion.cmake=
write_basic_package_version_file(
    "${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}ConfigVersion.cmake"
    COMPATIBILITY SameMajorVersion
)

Note that CMAKE_INSTALL_PREFIX should not be used in the argument to INSTALL_DESTINATION. This is for reasons described in the previous section about deferring evaluation of CMAKE_INSTALL_PREFIX until install time.

3. Avoid files that need to be sourced to use the package

Using the above directories lessens the need for a package to provide a file that users source to use the package.

For example, instead of defining completions in a file that needs to be sourced to use the package, we can place them in share/bash-completion/completions and name them appropriately and add ${PREFIX}/share to XDG_DATA_DIRS.

A good reason not providing commands as shell functions is that it add the requirement that users source a file to use our package.

It happens that providing commands as shell function is a necessity. Two examples of this are env-diff and vc (view-command).

4. Module file

Packages using the directories in the way described above can use the following modulefile:

prepend-path CMAKE_PREFIX_PATH @CMAKE_INSTALL_PREFIX@
prepend-path PATH @CMAKE_INSTALL_PREFIX@/bin
prepend-path LD_LIBRARY_PATH @CMAKE_INSTALL_PREFIX/lib
prepend-path XDG_DATA_DIRS @CMAKE_INSTALL_PREFIX@/share
source-sh @CMAKE_INSTALL_PREFIX@/etc/profile.d/@PROJECT_NAME@.sh

To illustrate, I have used the @ syntax used for CMake's configure_file() command but a project using something else a simple sed 's|<PREFIX>|...|g' can do the trick to replace occurences of <PREFIX> by the install destination given at install time.

5. Package loading tool

and a package loading tool like

load-package(){
    prefix=$1
    export CMAKE_PREFIX_PATH=${prefix}${CMAKE_PREFIX_PATH:+:${CMAKE_PREFIX_PATH}}
    export PATH=${prefix}/bin${PATH:+:${PATH}}
    export LD_LIBRARY_PATH=${prefix}/bin${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
    export XDG_DATA_DIRS=${prefix}/share{XDG_DATA_DIRS:+:${XDG_DATA_DIRS}}
    local restore_nullglob=$(shopt -p nullglob)
    shopt -s nullglob
    for f in ${prefix}/etc/profile.d/*.sh ${prefix}/etc/profile.d/*.bash ; do
        if [[ -f ${f} ]] ; then
            source $f
        fi
    done
    case ${MANPATH} in
        :*|*::*|*:|"") : good ;;
        *) MANPATH=:${MANPATH} ;;
    esac
}

6. Table

Directory Purpose Loading with evironment
bin Public executables Add ${PREFIX}/bin to PATH
etc Configuration files  
etc/profile.d Files that should be sourced at load time This is just a thing I do
include Header files Don't use C_INCLUDE_PATH
lib Shared and static libraries Add ${PREFIX}/bin to LD_LIBRARY_PATH
libexec/${PACKAGE_NAME} Private executables Users should not have direct access to these
share/man Man pages Add ${PREFIX}/bin to PATH, unset MANPATH
share/${PACKAGE_NAME} Other static files (1)  
share/bash-completion/completions Bash completion files Add ${PREFIX}/share to XDG_DATA_DIRS (2)
share/modulefiles Module files Add ${PREFIX}/share/modulefiles to MODULEPATH (3)
  • (1) What FSH says to put in share/${PACKAGE_NAME} is more precise than "everything that doesn't go in the other directories" but doing this is a good starting point.
  • (2) This is not the only way to make completion files available for lazy loading by bash-completion.
  • (3) Not FSH