Creating packages that work well with package managers
Table of Contents
For packages to work well with package managers, there are a few considerations:
- Placing files in the right places
- Setting up the build so that the package manager can easily build it.
But first some definitions:
PREFIX: The root directory where we install a package after building it.PACKAGE_NAME: The name of the package usually in kebab case.- "loading": Modifications made to the environment to make the package available for use by users. Not all packages are made available this way. If they are, I will call this "loading".
- "coexisting": A package is able to coexist with other package if they do not cause conflicts or ambiguities.
1. Directories
Placing files in these directories will
- Help packages coexist and not cause ambiguities
- Help tools find things the package provides
- Make packages easy to load by module files or other tools
1.1. bin
This is where executables of the package should go.
Files in this directory are used by executing them and loading a package should
add this directory to the list PATH.
1.2. etc
1.3. etc/profile.d/${PACKAGE_NAME}.sh
This is where I put files that would need to be sourced to use the package.
Putting files in appropriate directories reduces the need for users to have to source a file to use our package.
1.4. include
This is where headers for C and C++ should be placed.
They are used by various build systems in various way that each end up causing
-I ${PREFIX}/include to be added to compilation commands.
1.4.1. include/${PACKAGE_NAME}
To prevent ambiguities with similarly named headers, a subdirectory of include
named like the package can be used.
I recommend still only adding -I ${PREFIX}/include to compilations rather than
-I ${PREFIX}/include/${PACKAGE_NAME}. If we are using this technique to prevent
ambiguities with names, then we would also not want code to be ambiguous.
The include #include <${PACKAGE_NAME}/ambiguous_name.h> is less ambiguous than
#include <ambiguous_name.h>
1.5. lib
This directorie's primary purpose is to hold shared and static library files.
These libraries are used by build systems in various ways that usually end up
causing -L ${PREFIX}/lib and -lNAME to linking commands. CMake does its own
search for library files and provides the result (the full path to the library
file) to the linker.
Loading the package for runtime can be done by adding this directory to
LD_LIBRARY_PATH.
1.5.1. lib/cmake
When doing find_package(${PACKAGE_NAME}), CMake will look in $X/lib/cmake for a
file named ${PACKAGE_NAME}Config.cmake for each X in CMAKE_PREFIX_PATH.
This is one of many choices. The file can also be named
${PACKAGE_NAME@u}-config.cmake and can also be placed in a dozen other
locations relative to directories X in CMAKE_INSTALL_PREFIX.
See section Build/CMake/Config Files for more details.
1.6. share
1.6.1. share/man
This directory contains manpages.
The manpages can be made available to the user in various ways.
The way I suggest is to simply add ${PREFIX}/bin to PATH.
1.6.2. share/bash-completion/completions
Put files in here that define completions for a command or shell function. The
name of the file should be _ls, _ls.bash, ls, or ls.bash for definding
the completion of a command named ls.
A file defining completion functions for many commands can be reused by placing symbolic links for each other command whose completion is provided by this file.
1.6.3. share/${PACKAGE_NAME}
Other static files used by the package should go in this directory (technically what should go in here is more precise than "everything else" but if you don't know where to put something, put it here.
For packages to coexist, if the package provides more that one file, all those
files should go in a subdirectory of share named like the package.
This might seem redundant if you install packages each in their own PREFIX.
For example, I might install the package named repos in the prefix
$HOME/my-packages/repos and it will contain the directory
$HOME/my-packages/repos/share/repos. It might seem weird to have the name of
the package twice but this important for the package to be able to coexist with
other packages.
It is also good to do it this way because then we follow FHS guidelines.
1.6.4. share/modulefiles
The initialization of lmod and the other one adds /usr/share/modulefiles to
MODULE_PATH therefore it makes sense to put module files for things in our
package in this directory.
2. Build
Files placed according to the above guidelines will be easily usable, all the things it provides will be available to users without fuss and without fail.
These guidelines will help make build systems of packages integrate well with package managers that build from source.
2.1. Git repository information
For packages to be distributable with source tarballs, there should be a way to disable anything that depends on the git repository.
2.2. Git submodules
There should be a way to disable the use of dependencies vendored with the main package. This allows the package manager to handle dependencies.
2.3. CMake
2.3.1. CMAKE_INSTALL_PREFIX
Tools may choose to do run cmake SRC_DIR -DCMAKE_INSTALL_PREFIX=... followed
by make install or cmake -S SRC_DIR -B BUILD_DIR followed by cmake
--install BUILD_DIR --prefix ${PREFIX}.
It is essential that both be supported if using cpack (make package) because
it works by using cmake --install BUILD_DIR --prefix STAGING_AREA and making a
tarball of STAGING_AREA.
Both should be supported. This means that evaluation of CMAKE_INSTALL_PREFIX
should be deferred to the moment of installation.
The following evaluates CMAKE_INSTALL_PREFIX at configure time and does not
support cmake --install BUILD_DIR --prefix ${PREFIX}:
# Doesn't work install(CODE "message(STATUS \"installing into ${CMAKE_INSTALL_PREFIX}\")")
This is because CMake works by creating cmake_install.cmake at configuration
time that is then run as a CMake script at install time. The evaluation at
configure time causes the value of CMAKE_INSTALL_PREFIX from configure time to
become hardcoded into this script:
# cmake_install.cmake message(STATUS "installing into \usr\local"
The fix is very simple: just defer the evaluation until install time. This
version (with a \ in front of the dollar sign:
# Works install(CODE "message(STATUS \"installing into \${CMAKE_INSTALL_PREFIX}\")")
works because it the evaluation of ${CMAKE_INSTALL_PREFIX} will happen when
cmake_install.cmake is run (instead of happening during its creation):
# cmake_install.cmake message(STATUS "installing into ${CMAKE_INSTALL_PREFIX}")
In other install() commands, the value of CMAKE_INSTALL_PREFIX should not be
used. Let CMake handle that. The following:
# don't do this install(TARGETS my_exec DESTINATION ${CMAKE_INSTALL_PREFIX}/bin) install(FILES something DESTINATION ${CMAKE_INSTALL_PREFIX}/share/${PACKAGE_NAME}/something)
is redundant because when the argument of DESTINATION is a relative path, it
is interepreted relative to the installation directory.
It also breaks cpack, and prevents users and package managers from using
cmake --install BUILD --prefix ${PREFIX}.
We should do this instead:
# do this install(TARGETS my_exec DESTINATION bin) install(FILES something DESTINATION share/${PACKAGE_NAME}/something)
2.3.2. Config files
Starting with ${PROJECT_NAME}Config.cmake.in containing
<<TODO>>
and list_of_targets containing a list of some or all of the project's targets,
the code below define a name ${export_set_name} for a set of installed targets
and the calls to configure_package_config_file and
write_basic_package_version_file create
$PREFIX/cmake/${PACKAGE_NAME}Config.cmake and $PREFIX/cmake/${PACKAGE_NAME}ConfigVersion.cmake
include(GNUInstallDirs) # Defines CONFIG_INSTALL_DIR as lib/cmake include(CMakePackageConfigHelpers) # Add `EXPORT ${export_set_name}` to `install(TARGETS` command # to define ${export_set_name} and associate it with these targets. install(TARGETS ${list_of_targets} EXPORT ${export_set_name}) # An `install(EXPORT ${export_set_name} ...)` tells CMake to generate files # containing all the information on the targets in the previously created "export # set" for client # projects built with CMake to consume. install( EXPORT ${export_set_name} NAMESPACE ${PROJECT_NAME}:: DESTINATION ${CONFIG_INSTALL_DIR} ) # This creates =${PACKAGE_NAME}Config.cmake= which is the entry point from a # =find_package= call. configure_package_config_file( "${PROJECT_NAME}Config.cmake.in" "${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake" INSTALL_DESTINATION "${CONFIG_INSTALL_DIR}" PATH_VARS CMAKE_INSTALL_INCLUDEDIR CMAKE_INSTALL_LIBDIR ) # Creates =${PACKAGE_NAME}ConfigVersion.cmake= write_basic_package_version_file( "${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}ConfigVersion.cmake" COMPATIBILITY SameMajorVersion )
Note that CMAKE_INSTALL_PREFIX should not be used in the argument to
INSTALL_DESTINATION. This is for reasons described in the previous section
about deferring evaluation of CMAKE_INSTALL_PREFIX until install time.
3. Avoid files that need to be sourced to use the package
Using the above directories lessens the need for a package to provide a file that users source to use the package.
For example, instead of defining completions in a file that needs to be sourced
to use the package, we can place them in share/bash-completion/completions and
name them appropriately and add ${PREFIX}/share to XDG_DATA_DIRS.
A good reason not providing commands as shell functions is that it add the requirement that users source a file to use our package.
It happens that providing commands as shell function is a necessity. Two
examples of this are env-diff and vc (view-command).
4. Module file
Packages using the directories in the way described above can use the following modulefile:
prepend-path CMAKE_PREFIX_PATH @CMAKE_INSTALL_PREFIX@
prepend-path PATH @CMAKE_INSTALL_PREFIX@/bin
prepend-path LD_LIBRARY_PATH @CMAKE_INSTALL_PREFIX/lib
prepend-path XDG_DATA_DIRS @CMAKE_INSTALL_PREFIX@/share
source-sh @CMAKE_INSTALL_PREFIX@/etc/profile.d/@PROJECT_NAME@.sh
To illustrate, I have used the @ syntax used for CMake's configure_file()
command but a project using something else a simple sed 's|<PREFIX>|...|g' can
do the trick to replace occurences of <PREFIX> by the install destination
given at install time.
5. Package loading tool
and a package loading tool like
load-package(){ prefix=$1 export CMAKE_PREFIX_PATH=${prefix}${CMAKE_PREFIX_PATH:+:${CMAKE_PREFIX_PATH}} export PATH=${prefix}/bin${PATH:+:${PATH}} export LD_LIBRARY_PATH=${prefix}/bin${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} export XDG_DATA_DIRS=${prefix}/share{XDG_DATA_DIRS:+:${XDG_DATA_DIRS}} local restore_nullglob=$(shopt -p nullglob) shopt -s nullglob for f in ${prefix}/etc/profile.d/*.sh ${prefix}/etc/profile.d/*.bash ; do if [[ -f ${f} ]] ; then source $f fi done case ${MANPATH} in :*|*::*|*:|"") : good ;; *) MANPATH=:${MANPATH} ;; esac }
6. Table
| Directory | Purpose | Loading with evironment |
|---|---|---|
bin |
Public executables | Add ${PREFIX}/bin to PATH |
etc |
Configuration files | |
etc/profile.d |
Files that should be sourced at load time | This is just a thing I do |
include |
Header files | Don't use C_INCLUDE_PATH |
lib |
Shared and static libraries | Add ${PREFIX}/bin to LD_LIBRARY_PATH |
libexec/${PACKAGE_NAME} |
Private executables | Users should not have direct access to these |
share/man |
Man pages | Add ${PREFIX}/bin to PATH, unset MANPATH |
share/${PACKAGE_NAME} |
Other static files (1) | |
share/bash-completion/completions |
Bash completion files | Add ${PREFIX}/share to XDG_DATA_DIRS (2) |
share/modulefiles |
Module files | Add ${PREFIX}/share/modulefiles to MODULEPATH (3) |
- (1) What FSH says to put in
share/${PACKAGE_NAME}is more precise than "everything that doesn't go in the other directories" but doing this is a good starting point. - (2) This is not the only way to make completion files available for lazy
loading by
bash-completion. - (3) Not FSH