Butter is a plug-out for generating a build system for (at the moment) C++ projects. It can produce build systems in three styles: Boost jam, standard jam and gnu make. For people who have not used jam before, butter allows you to try it out before learning a new build-file syntax.
The genpro tool is okay for single-target projects which use Qt, but limited if you have a medium sized project with several dynamic libraries and executables. I myself use bouml for a project that has (currently) 8 or so libraries, 8 executables and 10 or so plugins. Thankfully I found the boost jam build tool which makes building such a project a breeze. However I found it frustrating that I could define all the build target information into bouml but could not write/convert that to anything. Hence the butter project was conceived (because butter goes well with jam!).
After writing the tool for boost jam files it was extended to create standard jam and gnu
make files. One of my original goals for adding make files was to allow me to build tarballs
for projects, possibly with autoconf configure scripts that could be built by people
that did not have boost jam. The current make style requires a make program that allows
immediately assigned variables allowing constructs such as A:=$(A) -B -C, which result in
a cyclic dependency in standard make. Perhaps someone more intelligent can come up with a
make writer that doesn't have this limitation. I am also hoping that some Java user may like
to contribute a writer class for an ant style.
This is the first beta release of the butter project. It is fully functional and a very useful tool, however not every situation has been tested. This could lead to complete failures of the tool (the plugout starts then "disappears") or any errors that are reported may not make sense. Please let me know your experiences of using this tool and any suggestions or feedback.
jam does not have an independent
install target (it performs installation as part of the default build) butter defines
install targets only for the release variant, requiring bjam variant=release to
build and install the project.
jam based build systems.
buildfile, makefile.sys,
local.jam or Jamrules have special meaning.
The standard jam program has the limitation that include
directories and compilation flags are set globally for each compilation
directory. This obviously means that the same set of
compilation flags and include directories are seen by all targets
build from a single directory and is a serious limitation if you want
two targets in the same directory to have conflicting flags. Note this
limitation is not true for the linker flags which are always target
specific. The plugout does attempt to merge the information from each
target to remove duplicate information.
===Make limitations===[make.limit]
The jam program manages dependencies for the entire project and correctly
rebuilds all affected targets in the correct order. The make program can manage
dependencies for targets in a single directory but not across directories.
Also, the way butter plugout writes makefiles means the targets will be
build in an arbitrary order. This means that you may need to
run make several times before all targets are updated correctly.
For example, take the case of target A being dependent on target B
in different directories and where B is build after A. The first make
run will have an error for building A as it cannot find B. This is good
as you can see that you must run make again. However, if you make a change
to B the next make run will rebuild B but not A as the old
version of B will be seen when make checks target A. Furthermore
there will be no error message. The only sure indication that all targets are
updated is when a make run performs no actions.
The generated build systems should produce equivalent targets. These are not necessarily identical and will not necessarily be located in the same place. Additionally, you can use target types that are specific to a particular build system style.
In this initial release of butter there is no install target type. It is intended that future releases will add this capability with particular emphasis on producing systems designed for easy integration with the autoconf system.
People are free to contribute additional build system styles and make comments to improve the existing styles.
Current development.
The butter project contains the butter files and properties needed to create a
build system for building butter itself. However, to create your first butter version
you use the genpro tool on the butter.plugout artifact to create a qmake pro
file. You will need to set the artifacts stereotype to executable before running
genpro. Before you use butter on this project you need to reset the `butter.plugout``
artifact's stereotype to source so that it is ignored.
To use the butter tool, place the executable in the bouml directory containing the other tools. Then in the Tools->tools settings dialog select the Others tab and add something like the following:
| Executable | display | Prj |
|---|---|---|
| butter | Create build system | X |
The butter plugout always operates on the entire project so it does not care where it is started. You can therefore add it to virtually any location. I activate it for the Prj as shown above as well as the next five columns (all the yellow folder icons) and under the artifact column.
The following example uses the butter bouml project as an example. In this projct we have five sub-packages:
..
bouml and src location as ../src/bouml.
bouml and src location as ../src/bouml, the same as the API BASE
package.
butter and src location is ../src/butter.
TIP I set the generation settings root directory to be /blah/blah/include and
then each package as header = name, source = ../src/name. When combined with the
#include : with root relative path you can create projects with source separated from
header files and need only include one -I include path. This is particularly useful
when you want to create a library that other people may use as you can easily package up the
header files and require library clients to use only one -I include path for your
library.
The butter system needs to be told where the base of the build directory tree is in the filesystem. The default location is the root directory as specified in the generation settings dialogue. If all locations defined in the project will not be sub-directories of this then you must indicate the base location. You use a user defined butter base property on the project package to define the relative path between the bouml root directory and the build base. Obviously if no such property is defined the generation settings root directory is used as the build base directory.
The generation settings root directory is [my-project]/include with each package header
as [package] and package source directory as ../src/[package]. I therefore need to
specify the build base directory as the parent of the root directory with a user property
on the base package:
butter base : ..
We also need to define which style of build system to use. This is done by defining a
butter base user property on the base package. If this is not defined then the
boost jam style is the default (it is also used whenever the style name is not
recognised).
butter style : make
When using the boost jam style we can also define where we want the object files and targets
to be build using the butter build-dir property:
butter build-dir : build
The standard bouml plug-out project uses a single directory for all the header and source files.
I have modified this to use different directories for headers and source and also placed the
headers in a sub-directory of the generation root. I have also defined
#include : with root relative path generation option. For automatically generated includes
I need not do anything more, however many includes are manually defined in the API BASE and
API USER packages. I therefore need to add include/bouml as an include directory for
all targets.
In addition I will be using definitions and settings specific to the C++ system. I therefore
need to define the -DWITHCPP flag for all targets. Lastly, I have a multi-directory
project but only want a single application so I therefore want to build all libraries targets as
static. To do this I set the following user properties on the base package:
butter flags : -DWITHCPP butter include : include/bouml butter type : static
The bouml API uses the QT3 library and this is used as the basis library for the butter
project. I therefore need to define the include and linker options for this library. These could
be added to the project level options above, but then they would apply to all targets. A more
flexible way is to define a library artifact to act as a placeholder for external library in the
same way classes defined as external act as placeholders. In the External package I define
an artifact called qt which I set to have the library stereotype. I then mark it as an
external library by defining the butter project user property:
butter project : qt3
I define the include and link options by setting the library description to:
${butter_generic}
HDR=${QTDIR}/include
LINK=-L${QTDIR}/lib -lqt-mt -lXext -lX11 -lm
The ${butter_generic} is a section marker indicating the following description contains
information that is used for all styles. The HDR= and LINK= obviously (also FLAGS=)
define the include directories and link flags. Note that the include directories are the specified
as the directory names themselves.
Using the ${butter_generic} marker gives butter information to translate into
each style. However it has limitations and so providing definitions on a per-style basis
is possible. A simple example is wanting to use the compile option configuration program
supplied with many libraries, in this case the standard method will not work for boost jam
so an alternative for that must be used.
${butter_generic}
FLAGS=`pkg-config qt-mt --cflags`
LINK=`pkg-config qt-mt --libs`
${butter_boost}
# This may not work, but given as an example
.qtcompileflags = [ string.join [ string.words [ SHELL "pkg-config qt-mt --cflags-only-other" ] ] : " " ] ;
.qtincprefix = [ MATCH "-I\(.*\)" : [ string.words [ SHELL "pkg-config qt-mt --cflags-only-I" ] ] ] ;
.qtlibprefix = [ MATCH "-L\(.*\)" : string.join [ string.words [ SHELL "pkg-config qt-mt --libs-only-L" ] ] ] ;
.qtlinkflags = [ string.join [ MATCH "-lqt-mt|(.*)" : [ string.words [ SHELL "pkg-config qt-mt --libs-only-l --libs-only-other" ] ] ] : " " ] ;
lib qt-mt : <name>qt3 : : <include>$(.qtincprefix)
<library-path>$(.qtlibprefix)
<cflags>$(.qtcompileflags)
<cxxflags>$(.qtcompileflags)
<linkflags>$(.qtlinkflags)
<allow>qt-mt ;
A feature of the library is that a target can only contain source artifacts from within the
same filesystem directory. The butter project has two main source directories, src/bouml
and src/butter. Here we will create a library for the bouml API classes in src/bouml and
the plug-out executable in src/butter.
In package API USER I create an artifact bouml with stereotype library. The artifact
is associated with all the API USER and API BASE artifacts as well as the qt library
artifact. The default library type was defined as static in the base project and I do not want
to override that here, so this library is defined.
In the butter package I create an artifact butter with stereotype executable. This
is associated with the artifacts for butter classes and the ''Main'' source. Additionally it
is associated to both the qt and the bouml libraries.
In addition to creating build files butter will also write any artifacts with the document stereotype. I therefore create a README document artifact in the Build package.
The boost version of jam understands the concept of build variants. This is not present in the
other systems so we need a way to define settings specific to each style. Here I want to set
some options that are only used in the release variant (debug is the default). To do
this I create an artifact called buildfile in the Build package. When creating the build
system butter searches the packages at each build location for an artifact with document
stereotype and buildfile name. This is included as-is into the top of the build file for
created by butter if no section headers are present, otherwise the appropriate section
is included. Here the buildfile description may contain the following. Note the
use of section headers ${butter_boost} etc to specify parts of the description to
be used in each build style.
${butter_boost}
using gcc : : : <cxxflags>"-std=c++98"
<linkflags>"-rdynamic -lz"
<optimization>speed:<cxxflags>"-march=native -mfpmath=sse -msse3" ;
import string ;
project butter : requirements <variant>release:<cxxflags>"-DNO_LOG -DNO_DBC" ;
${butter_standard}
# Build file for the butter project.
#
COMP = GNU ;
GUILIB = QT3 ;
# Signal use of the OPTIM flags instead of the DEBUG flags for release builds.
# I added "-DNO_LOG -DNO_DBC" to the optimisation flags in Jamrules
# DEBUG = OPTIM ;
DEBUG = DEBUG ;
${butter_make}
# Uncomment the following for release builds.
# USERFLAGS :=$(USERFLAGS) -DNO_LOG -DNO_DBC
We have now defined our basic build system. We can no select any item that has the butter tool defined and select it. Note that butter always navigates to and runs from the base package regardless of where it is started.
Running the tool will give a result similar to the following, except here we have run it before so it only writes any changed documents.
Please wait: building target list for project butter. Skipping unchanged build file : /home/finnerty/Office/Projects/butter/src/butter/makefile Writing build file : /home/finnerty/Office/Projects/butter/src/bouml/makefile Skipping unchanged build file : /home/finnerty/Office/Projects/butter/src/makefile Skipping unchanged build file : /home/finnerty/Office/Projects/butter/makefile Writing changed document : /home/finnerty/Office/Projects/butter/README.t2t Skipping unchanged document : /home/finnerty/Office/Projects/butter/makefile.sys
Note at the bottom the document makefile.sys. If you now look at the Build package
an artifact called makefile.sys has been created. This contains the framework methods
used in the make style build system. If we change the style to standard we would
get a Jamrules artifact appearing. Note that the Build package is not special, the
program searches for artifacts with these names and uses whatever is found. If an artifact
is not found then one with the default implementation is added to the first Deployment
View at the project base, creating the Deployment View if necessary.
Once created these files can be edited to add extra features and details. If you make a mistake and the build system stops working, you can simply rename the current artifact and a new default implementation will be generated. You can then re-add any changes from the previous version (that work!).
All the created build systems attempt to ensure dependencies are met. In this regard jam and
boost jam are much better as they parse the entire build project and order builds to ensure
dependencies are satisfied. The make system handles this on a directory-by-directory basis, but
across directories is not handled (see make limitations). Therefore to build
the project we can use jam or bjam once, but must run make -k several times.
As our project continues we may add classes, change class (and artifact) names and so on. This should be no problem, we simply rerun butter to update the build system. Butter attempts to provide enough flexibility that you can encapsulate all build information into the bouml project. This could be a great help when sharing the project with other users and allows you to version (CVS, subversion etc) everything in the project using only the data in the bouml project directory.
The basic purpose of the butter project is to write non-source information from the bouml project into the filesystem. The major operation is transforming the target artifact objects into some sort of build file, in a similar way to the genpro plugout. Additionally it combines all the targets to generate a single set of build files for creating every target in the project.
The second major operation is to write document artifacts into the
filesystem. This is required by the standard and make styles so they
can write their global definition files into the bouml project, where the user
can subsequently edit them if necessary. It also has the advantage that text
files, such as README, can be stored in the bouml project.
These two operations combine to allow every aspect of a project to be saved within the bouml project.
The butter plugout uses the top-level package names as the project name. I
also uses the user property "butter version" or "version" as the version number.
However, the most significant place this is used is the VERSIONDIR variable
in the build settings file that will appear as document artifacts
in the bouml project. Once created these files are not altered by the butter
project. This allows you to add extra information to these files but will
not reflect changes to the version number of project name. Therefore the
VERSIONDIR will need to be manually changed.
In bouml you can specify the same filesystem directory as the location to write
source files for several different packages. Furthermore, these filesystem
directories can be located at an arbitrary distance from each other. To handle
this situation butter introduces the concept of a location, which basically
represents a filesystem directory. Internally a location will be assigned to
every filesystem directory in the project. A location may therefore contain
several packages, targets, documents and/or child locations. After processing
the bouml project, locations without any targets, documents and/or child locations
are discarded and every remaining location will be converted into a build file.
What this means for users
buildfile artifact in one
package will also be seen by the other packages at that location.
A central concept used in butter is that of a build target. These are defined to be UML artifact objects with the library or executable stereotype and are translated into the obvious build targets. Alternative build targets (eg python-extension) are supported by setting a property on a library artifact.
A target is defined using the Associated artifacts of the target artifact. Users should note the following considerations:
cannot find source file: XXXX errors.
In an analogous way to using external classes, you can create an artifact for an external library. The include, link and flags definitions for this artifact are then automatically added to internal targets that reference the external target.
All build targets are added to the default invocation of the build tool. In
addition an install target is supported. How the install target varies
with the build style. For the make style an install target is added
and must be called manually, for boost the installation occurs automatically
for release variant builds.
One goal of the butter plugout is to facilitate project delivery as well
as building. We assume that the project packaging is likely to be managed
by the GNU autoconf or similar tool. To facilitate using such a tool each
style contains a set of variables for each installation target directory used
by autoconf. The names of these variables can be found by looking at one of
the special build files. For example the default variable used
for a library target is LIBDIR and for an executable target it
is BINDIR. The install location can be changed by setting the
butter install property of the target to the name of one of the other
variables.
Additionally, if you set the butter install property on a document
artifact the document will be set added to the install target.
The butter plugout does not directly support installation of files
generated by the documentation plugout. This is because these files are
primarily developer documentation rather than user documentation. It can
be supported indirectly by defining a document artifact with the name
of the root-directory of the generated documentation and setting the
butter document property to nowrite and the butter install
location to somewhere, such as DOCDIR.
Let us assume that you have the text for a man page written in txt2tags markup
in document artifact butter.t2t and also you have a man build rule
for converting the document to a man page. To convert the artifact to a man
page and have it installed we create a library artifact butter.1 and
associate butter.t2t to it. We then set the following properties on the new
library artifact:
| butter other | man |
| butter install | MAN1 |
The butter plug-out uses three main mechanisms for encoding build information into the the bouml project: user-defined Properties, Descriptions and, description of specially named Artifacts. In general properties are used for information that is used regardless of the build style. Descriptions are used for information that may need to be different for different styles but is generally present in each style. Specially named artifacts are used to include information specific to each style which may not be present in every style.
boost. The other alternatives are standard (for standard jam) and
make. If the name of the style is not recognised you get an error message
and the default style is used.
[real-root]/include and
set butter base to .. (I also set the src directories to "../src/[name]").
boost style.
===Artifact properties===[art.prop]
Those properties with an (*) may be inherited from parent objects up to and including the project, with local definitions overriding more distant definitions. Properties with an (@) when used on the project package are added to every build target, the project level properties are combined with that of the target. Properties with an (#) may also be set in the target's description see Artifact Descriptions (note that this does not apply for a package or project).
-I will be prepended for those systems
that require it.
BINDIR and LIBDIR install locations. To install in a different location
set this property to one of the other install directory names. If
you do not want the target installed set it to NONE.
boost style, [name] is used to group external projects which
have the same external project name.
$(NAME_IN_UPPERCASE) used as the target build tool.
The definition for what these do must be defined by the user or
must be already present in the build systems.
using [name] ;
include [name].jam ;
include [name].make
The butter plugout can use the description from target artifacts
to store generic or style-specific build information. It uses section
markers that look like the replacement markers used in bouml, namely
${butter_generic} for the section containing generic information.
Each style has a section marker defined as ${butter_ + style name + }.
Firstly the description is divided into sections by the markers and the active section is defined by the following rules:
Once a section has been discovered it is searched for lines containing the following definition labels.
HDR= FLAGS= LINK=
which are converted to include, flag and link definitions.
For external library targets if the definition labels are present then the rest of the section is ignored. If none of the labels are present then the section is written as-is into the build file.
Artifacts with the document stereotype and with the name
buildfile are considered to contain header text in their
descriptions for the
build file at the current location. It uses the same active section
rules as for target artifact descriptions. Any
active section is written as-is into the beginning of the
build file for this location.
If several buildfile documents are present for a single
filesystem location then each is written into the single build
file in an unspecified order.
Each styles uses a file containing global definitions located at the top-level project directory. An artifact with the content of such a file is created in the bouml project the first time butter is run for that styles. Once created these named artifacts will be reused on later butter runs, allowing the user to edit the content if desired. Note that if one of these files exist in the project directory already it will be overwritten by the butter plugout.
The names of these build files are:
| style | filename |
|---|---|
make |
makefile.sys |
standard |
Jamrules |
boost |
local.jam |
The content of these files, particularly makefile.sys, is essential for the correct operation of the generated build system. Therefore, care should be taken when editing these artifacts in bouml. If something does go wrong then simply renaming the artifact in bouml will cause a fresh default version to be created.
The default contents of these files (as generated by butter) are intended only as starting points. It is recommended that you edit these to match your needs. Because butter will use artifacts with these names it is possible to add these artifacts to a template bouml project with your other tailered settings.