== C and C++ projects

Although Waf is language neutral, it is commonly used for C and C++ projects. This chapter describes the waf tools used for this purpose.

=== Common script for C/C++ applications

The c/c++ builds consist in transforming source files into object files, and to assemble the object files at the end. In theory a single programming language should be sufficient for writing any application, but in practice this does not scale:

. Applications may be divided in dynamic or static libraries
. Additional files may enter in the link step (libraries, object files)
. Source files may be generated by other compilers
. Differnt platforms require different processing rules (manifest files on windows, etc)

The construction of c/c++ applications can be quite complicated, and several measures must be taken to ensure coherent interaction with new compilation rules. The canonical code for a task generator building a c/c++ application is the following:

[source,python]
---------------
top = '.'
out = 'build'

def set_options(opt):
	opt.tool_options('compiler_cc')

def configure(conf):
	conf.check_tool('compiler_cc') <1>

def build(bld):
	t = bld(
		features     = ['cc', 'cprogram'], <2>
		source       = 'main.c', <3>
		target       = 'appname', <4>
		install_path = '${SOME_PATH}/bin', <5>
		vnum         = '1.2.3', <6>
		includes     = ['.'], <7>
		defines      = ['LINUX=1', 'BIDULE'], <8>
		ccflags      = ['-O2', '-Wall'], <9>
		lib          = ['m'], <10>
		libpath      = ['/usr/lib'],
		linkflags    = ['-g'],
	)
	t.rpath          = ['/usr/lib']
---------------

<1> Load the c/c++ support routines (such as the features below) and try to find a suitable compiler for the build. The support for c++ is loaded by 'compiler_cxx'
<2> Task generator declaration; each element in the list represent a feature; it is possible to add several languages at once ('ocaml and c++' for example), but the one of 'cstaticlib, cshlib or cprogram' must be chosen.
<3> List of source, it may be either a python list, or a string containing the file names separated with spaces. This list may contain file names of different extensions to make hybrid applications.
<4> Target name, it is concerted to the name of the binary 'name.so' or 'name.exe' depending on the platform and the features.
<5> Installation directory, this is where to install the library or program produced. The '$\{}' expression is a reference to a variable to be extracted from 'tgen.env'. By default it is set to '$\{PREFIX}/bin' for programs and '$\{PREFIX}/lib' for libraries. To disable the installation, set it to 'None'.
<6> Version number for shared libraries. It is not used for programs or static libraries.
<7> List of include paths, it may be either a python list, or a string containing the paths separated by spaces. The paths are used for both the command-line and for finding the implicit dependencies (headers). In general, include paths must be relative to the wscript folder and given explicitly.
<8> Command-line defines: list of defines to add to the command-line with the '-D' prefix. To reduce the size of the command-line, it is possible to use a configuration header, see the following section for more details.
<9> Command-line compilation flags, for the c++ language the attribute is called 'cxxflags'
<10> Shared libraries may be given directly (use 'staticlib' and 'staticlibpath' for static libraries)

Additional pararameters may be added from a task generator reference. The next section describes a technique to gather the conditions into the configuration section.

=== Include processing

==== Execution path and flags

Include paths are used by the c/c++ compilers for finding the headers. When one header changes, the files are recompiled automatically. For example on a project having the following structure:

[source,shishell]
---------------
$ tree
.
|-- foo.h
|-- src
|   |-- main.c
|   `-- wscript
`-- wscript
---------------

The file 'src/wscript' will contain the following code:

[source,python]
---------------
def build(bld):
    bld(
        features = 'cc cprogram',
        source   = 'main.c',
        target   = 'myapp',
        includes = '.. .')
---------------

The command-line (output by `waf -v`) will have the following form:

[source,shishell]
---------------
cc -Idefault -I.. -Idefault/src -I../src ../src/main.c -c -o default/src/main_1.o
---------------

Because commands are executed from the build directory, the folders have been converted to include flags in the following way:

[source,shishell]
---------------
.. -> -I..      -Idefault
.  -> -I../src  -Idefault/src
---------------

There are the important points to remember:

. The includes are always given relative to the directory containing the wscript file
. The includes add both the source directory and the corresponding build directory for the task generator variant
. Commands are executed from the build directory, so the include paths must be converted
. System include paths should be defined during the configuration and added to CPPPATH variables (uselib)

==== The Waf preprocessor

Waf uses a preprocessor written in Python for adding the dependencies on the headers. A simple parser looking at #include statements would miss constructs such as:

[source,c]
---------------
#define mymacro "foo.h"
#include mymacro
---------------

Using the compiler for finding the dependencies would not work for applications requiring file preprocessing such as Qt. For Qt, special include files having the '.moc' extension must be detected by the build system and produced ahead of time. The c compiler could not parse such files.

[source,c]
---------------
#include "foo.moc"
---------------

Since system headers are not tracked by default, the waf preprocessor may miss dependencies written in the following form:

[source,c]
---------------
#if SOMEMACRO
	/* an include in the project */
	#include "foo.h"
#endif
---------------

To write portable code and to ease debugging, it is strongly recommended to put all the conditions used within a project into a 'config.h' file.

[source,python]
---------------
def configure(conf):
	conf.check(
		fragment='int main() { return 0; }\n',
		define_name='FOO',
		mandatory=1)
	conf.write_config_header('config.h')
---------------

For performance reasons, the dependencies are not added onto system headers by default. The following code may be used to enable this behaviour:

[source,python]
---------------
import preproc
preproc.go_absolute = True
---------------

==== Debugging dependencies

The Waf preprocessor contains a specific debugging zone:

[source,shishell]
---------------
$ waf --zones=preproc
---------------

To display the dependencies obtained or missed, use the following:

[source,shishell]
---------------
$ waf --zones=deps

23:53:21 deps deps for src:///comp/waf/demos/qt4/src/window.cpp: <1>
  [src:///comp/waf/demos/qt4/src/window.h, bld:///comp/waf/demos/qt4/src/window.moc]; <2>
  unresolved ['QtGui', 'QGLWidget', 'QWidget'] <3>
---------------

<1> File being preprocessed
<2> Headers found
<3> System headers discarded

The dependency computation is performed only when the files are not up-to-date, so these commands will display something only when there is a file to compile.


=== Library interaction (uselib)

==== Local libraries

To link a library against another one created in the same Waf project, the attribute 'uselib_local' may be used. The include paths, the link path and the library name are automatically exported, and the dependent binary is recompiled when the library changes:

[source,python]
---------------
def build(bld):
	staticlib = bld(
		features       = 'cc cstaticlib', <1>
		source         = 'test_staticlib.c',
		target         = 'teststaticlib',
		export_incdirs = '.') <2>

	main = bld(
		features       = 'cc cprogram', <3>
		source         = 'main.c',
		target         = 'test_c_program',
		includes       = '.',
		uselib_local   = 'teststaticlib') <4>
---------------

<1> A static library
<2> Include paths to export for use with uselib_local (include paths are not added automatically). These folders are taken relatively to the current target.
<3> A program using the static library declared previously
<4> A list of references to existing libraries declared in the project (either a python list or a string containing the names space-separated)

The library does not need to actually declare a target. Header-only libraries may be created to add common include paths to several targets:

[source,python]
---------------
def build(bld):
	bld(
		uselib         = '',
		export_incdirs = '.',
		name           = 'common_includes')

	main = bld(
		features       = 'cc cprogram',
		source         = 'main.c',
		target         = 'some_app',
		uselib_local   = 'common_includes')
---------------

NOTE: Include paths are only propagated when 'export_incdirs' is provided. The include paths are relative to the path of the task generator that declared them.

NOTE: The local shared libraries and the uselib variables are propagated transitively, but the static libraries are not.

==== System libraries

To link an application against various 'system libraries', several compilation flags and link flags must be given at once. To reduce the maintenance, a system called 'uselib' can be used to give all the flags at the same time:

[source,python]
---------------
def configure(conf):
	conf.env.CPPPATH_TEST = ['/usr/include'] <1>

	if sys.platform != win32: <2>
		conf.env.CCDEFINES_TEST = ['TEST']
		conf.env.CCFLAGS_TEST   = ['-O0'] <3>
		conf.env.LIB_TEST       = 'test'
		conf.env.LIBPATH_TEST   = ['/usr/lib'] <4>
		conf.env.LINKFLAGS_TEST = ['-g']
		conf.env.CPPPATH_TEST   = ['/opt/gnome/include']

def build(bld):
	mylib = bld(
		features = 'cc cstaticlib',
		source   = 'test_staticlib.c',
		target   = 'teststaticlib',
		uselib   = 'TEST') <5>

	if mylib.env.CC_NAME == 'gcc':
		mylib.cxxflags = ['-O2'] <6>
---------------

<1> For portability reasons, it is recommended to use CPPPATH instead of giving flags of the form -I/include. Note that the CPPPATH use used by both c and c++
<2> Variables may be left undefined in platform-specific settings, yet the build scripts will remain identical.
<3> Declare a few variables during the configuration, the variables follow the convention VAR_NAME
<4> Values should be declared as lists excepts for LIB and STATICLIB
<5> Add all the VAR_NAME corresponding to the uselib NAME, which is 'TEST' in this example
<6> 'Model to avoid': setting the flags and checking for the configuration must be performed in the configuration section

The variables used for c/c++ are the following:

.Uselib variables and task generator attributes for c/c++
[options="header"]
|=================
|Uselib variable | Attribute | Usage
|LIB      |lib      | list of sharedlibrary names to use, without prefix or extension
|STATICLIB|staticlib| list of static library names to use, without prefix or extension
|LIBPATH  |libpath  | list of search path for shared and static libraries
|RPATH    |rpath    | list of paths to hard-code into the binary during linking time
|CXXFLAGS |cxxflags | flags to use during the compilation of c++ files
|CCFLAGS  |ccflags  | flags to use during the compilation of c files
|CPPPATH  |includes | include paths
|CXXDEPS  |         | a variable to trigger c++ file recompilations when it changes
|CCDEPS   |         | same as above, for c
|CCDEFINES,CXXDEFINES|defines| list of defines in the form ['key=value', 'key', ...]
|FRAMEWORK|framework| list of frameworks to use, requires the waf tool 'osx'
|FRAMEWORKPATH|frameworkpath| list of framework paths to use, requires the waf tool 'osx'
|=================


The variables may be left empty for later use, and will not cause errors. During the development, the configuration cache files (for example, default.cache.py) may be modified from a text editor to try different configurations without forcing a whole project reconfiguration. The files affected will be rebuilt however.

==== Specific settings by object files

In some cases, it is necessary to re-use object files generated by another task generator to avoid recompilations. This is similar to copy-pasting code, so it is discouraged in general. Another use for this is to enable some compilation flags for specific files. Although it is not exactly part of the library prossing, the attribute "add_objects" can be used for this purpose:

[source,python]
---------------
def build(bld):
	some_objects = bld(
		features       = 'cc', <1>
		source         = 'test.c',
		ccflags        = '-O3',
		target         = 'my_objs')

	main = bld(
		features       = 'cc cprogram',
		source         = 'main.c',
		ccflags        = '-O2', <2>
		target         = 'test_c_program',
		add_objects    = 'my_objs') <3>
---------------

<1> Files will be compiled in c mode, but no program or library will be produced
<2> Different compilation flags may be used
<3> The objects will be added automatically in the link stage

=== Configuration helpers

==== Configuration tests

The method 'check' is used to detect parameters using a small build project. The main parameters are the following

. msg: title of the test to execute
. okmsg: message to display when the test succeeds
. errmsg: message to display when the test fails
. mandatory: when true, raise a configuration exception if the test fails
. env: environment to use for the build (conf.env is used by default)
. compile_mode: 'cc' or 'cxx'
. define_name: add a define for the configuration header when the test succeeds (in most cases it is calculated automatically)

Besides the main parameters, the attributes from c/c++ task generators may be used. Here is a concrete example:

[source,python]
---------------
def configure(conf):

	conf.check(header_name='time.h', compile_mode='cc') <1>
	conf.check_cc(function_name='printf', header_name="stdio.h", mandatory=True) <2>
	conf.check_cc(fragment='int main() {2+2==4;}\n', define_name="boobah") <3>
	conf.check_cc(lib='m', ccflags='-Wall', defines=['var=foo', 'x=y'],
		uselib_store='M') <4>
	conf.check_cxx(lib='linux', uselib='M', cxxflags='-O2') <5>

	conf.check_cc(fragment='''
			#include <stdio.h>
			int main() { printf("4"); return 0; } ''',
		define_name = "booeah",
		execute     = True,
		define_ret  = True,
		msg         = "Checking for something") <6>

	conf.write_config_header('config.h') <7>
---------------

<1> Try to compile a program using the configuration header time.h, if present on the system, if the test is successful, the define HAVE_TIME_H will be added
<2> Try to compile a program with the function printf, adding the header stdio.h (the header_name may be a list of additional headers). The parameter mandatory will make the test raise an exception if it fails. Note that convenience methods 'check_cc' and 'check_cxx' only define the compilation mode 'compile_mode'
<3> Try to compile a piece of code, and if the test is successful, define the name boobah
<4> Modifications made to the task generator environment are not stored. When the test is successful and when the attribute uselib_store is provided, the names lib, cflags and defines will be converted into uselib variables LIB_M, CCFLAGS_M and DEFINE_M and the flag values are added to the configuration environment.
<5> Try to compile a simple c program against a library called 'linux', and reuse the previous parameters for libm (uselib)
<6> Execute a simple program, collect the output, and put it in a define when successful
<7> After all the tests are executed, write a configuration header in the build directory (optional). The configuration header is used to limit the size of the command-line.

Here is an example of a 'config.h' produced with the previous test code:

[source,c]
---------------
/* Configuration header created by Waf - do not edit */
#ifndef _CONFIG_H_WAF
#define _CONFIG_H_WAF

#define HAVE_PRINTF 1
#define HAVE_TIME_H 1
#define boobah 1
#define booeah "4"

#endif /* _CONFIG_H_WAF */
---------------

The file `default.cache.py` will contain the following variables:

[source,python]
---------------
CCDEFINES_M = ['var=foo', 'x=y']
CXXDEFINES_M = ['var=foo', 'x=y']
CXXFLAGS_M = ['-Wall']
CCFLAGS_M = ['-Wall']
LIB_M = ['m']
boobah = 1
booeah = '4'
defines = {'booeah': '"4"', 'boobah': 1, 'HAVE_TIME_H': 1, 'HAVE_PRINTF': 1}
dep_files = ['config.h']
waf_config_files = ['/compilation/waf/demos/adv/build/default/config.h']
---------------

NOTE: The methods 'conf.check' all use task generators internally. This means that the attributes 'includes', 'defines', 'cxxflags' may be used (not all shown here).

NOTE: The attribute 'compile_mode' may also contain user-defined task generator features.

==== Creating configuration headers

Adding lots of command-line define values increases the size of the command-line and conceals the useful information (differences). Some projects use headers which are generated during the configuration, they are not modified during the build and they are not installed or redistributed. This system is useful for huge projects, and has been made popular by autoconf-based projects.

Writing configuration headers can be performed using the following methods:

[source,python]
---------------
def configure(conf):
	conf.define('NOLIBF', 1)
	conf.undefine('NOLIBF')
	conf.define('LIBF', 1)
	conf.define('LIBF_VERSION', '1.0.2')
	conf.write_config_header('config.h')
---------------

The code snipped will produce the following 'config.h' in the build directory:

[source,shishell]
---------------
build/
|-- c4che
|   |-- build.config.py
|   `-- default.cache.py
|-- config.log
`-- default
    `-- config.h
---------------

The contents of the config.h for this example are

[source,c]
---------------
/* Configuration header created by Waf - do not edit */
#ifndef _CONFIG_H_WAF
#define _CONFIG_H_WAF

/* #undef NOLIBF */
#define LIBF 1
#define LIBF_VERSION "1.0.2"

#endif /* _CONFIG_H_WAF */
---------------


==== Pkg-config

Instead of duplicating the configuration detection in all dependent projects, configuration files may be written when libraries are installed. To ease the interaction with build systems based on Make (cannot query databases or apis), small applications have been created for reading the cache files and to interpret the parameters (with names traditionally ending in '-config'): http://pkg-config.freedesktop.org/wiki/[pkg-config], wx-config, sdl-config, etc.

The method 'check_cfg' is provided to ease the interaction with these applications. Here are a few examples:

[source,python]
---------------
def configure(conf):
	conf.check_cfg(atleast_pkgconfig_version='0.0.0') <1>
	pango_version = conf.check_cfg(modversion='pango', mandatory=True) <2>

	conf.check_cfg(package='pango') <3>
	conf.check_cfg(package='pango', uselib_store='MYPANGO', mandatory=True) <4>

	conf.check_cfg(package='pango',
		args='"pango >= 1.0.0" "pango < 9.9.9" --cflags --libs', <5>
		msg="Checking for pango 1.0.0", mandatory=True) <6>

	conf.check_cfg(path='sdl-config', args='--cflags --libs',
		package='', uselib_store='SDL') <7>
	conf.check_cfg(path='mpicc', args='--showme:compile --showme:link',
		package='', uselib_store='OPEN_MPI')
---------------

<1> Check for the pkg-config version
<2> Retrieve the module version for a package. The returned object is a string containing the version number or an empty string in case of any errors. If there were no errors, 'PANGO_VERSION' is defined, can be overridden with the attribute "uselib_store='MYPANGO'".
<3> Obtain the flags for a package and assign them to the uselib PANGO (calculated automatically from the package name)
<4> Store the flags to the uselib variable 'MYPANGO', and raise a configuration error if the package cannot be found
<5> Use pkg-config clauses to ensure a particular version number. The arguments and the command are joined into a string and executed by the shell.
<6> Display a custom message on the output.
<7> Obtain the flags for a different configuration program (sdl-config). The example is applicable for other configuration programs such as wx-config, pcre-config, etc

Due to the amount of flags, the lack of standards between config applications, and to the output changing for different operating systems (-I for gcc, /I for msvc), the output of pkg-config is parsed, and the variables for the corresponding uselib are set in a go. The function 'parse_flags(line, uselib, env)' in the Waf module config_c.py performs the output analysis.

The outputs are written in the build directory into the file 'config.log':

[source,shishell]
---------------
# project cpp_test (0.0.1) configured on Fri Feb 26 20:51:39 2010 by
# waf 1.5.18 (abi 7, python 20602f0 on linux2)
# using /home/waf/bin/waf configure
#
Checking for pkg-config version >= 0.0.0
pkg-config --errors-to-stdout --print-errors --atleast-pkgconfig-version=0.0.0

pkg-config --errors-to-stdout --print-errors --modversion pango

Checking for pango
pkg-config --errors-to-stdout --print-errors  pango

Checking for pango
pkg-config --errors-to-stdout --print-errors  pango

Checking for pango 1.0.0
pkg-config --errors-to-stdout --print-errors "pango >= 1.0.0" --cflags --libs pango

Checking for sdl-config
sdl-config --cflags --libs

Checking for mpicc
mpicc --showme:compile --showme:link
---------------

After such a configuration, the configuration set contents will be similar to the following:

[source,python]
---------------
CCFLAGS_OPEN_MPI = ['-pthread']
CPPPATH_OPEN_MPI = ['/usr/lib64/mpi/gcc/openmpi/include/openmpi']
CPPPATH_PANGO = ['/usr/include/pango-1.0', '/usr/include/glib-2.0']
CXXFLAGS_OPEN_MPI = ['-pthread']
HAVE_MYPANGO = 1
HAVE_OPEN_MPI = 1
HAVE_PANGO = 1
LIB_PANGO = ['pango-1.0', 'gobject-2.0', 'gmodule-2.0', 'glib-2.0']
LINKFLAGS_OPEN_MPI = ['-pthread']
PANGO_VERSION = '1.25.3'
PREFIX = '/usr/local'
defines = {'HAVE_OPEN_MPI': 1, 'HAVE_PANGO': 1,
	'PANGO_VERSION': '"1.25.3"', 'HAVE_MYPANGO': 1}
---------------

