Bering-uClibc 5.x - Developer Guide - Compiling Source Code
|Compiling Source Code|
|Prev||Bering-uClibc 5.x - Developer Guide||Next|
- 1 Introduction
- 2 Buildtool usage
- 3 Creating buildtool sources/packages
What is buildtool?
Buildtool is a set of Perl scripts and Perl modules to build LRP files (LEAF Packages) from source. Everything in the build process is automated, from extracting the sources and applying patches to configuring and building the source and creating the LRP Package.
In very early LEAF releases it was usual to compile the sources and build the Packages manually. This could be error-prone, and where upstream source upgrades happened regularly it was tedious to have to repeat builds for new versions manually.
The current practice is for every Package to be generated automatically according to simple configuration definitions. This means that no special knowledge is required to re-build a particular Package, because all of the steps are captured in the configuration files. Some effort is required to prepare the initial buildtool configuration but many Packages are fairly "standard" and it is often possible to adapt the configuration from a similar package quite quickly.
usage: ./buildtool.pl [option] command [pkgname|srcname] [...] commands: describe [pkgname|srcname] shows descriptionlines of package list [sourced|built] shows a list of built/sourced packages and sources dumpenv [pkgname|srcname] dump the environment of buildtool and package source [pkgname|srcname] downloads, unpacks and patches the wanted package/source build [pkgname|srcname] the same as source, but builds and installs sources/packages also pkglist [pkgname|srcname] create a list with all dependencies for the package given or all if no name given buildclean [pkgname|srcname] removes everything that is outside the source dir srcclean [pkgname|srcname] same as buildclean + call make srcclean remove [pkgname|srcname] same as buildclean + remove everything from dldir distclean remove everything maketar make a tar for distribution options: -v just print version and exit -f allows you to force the command even if the internal state of buildtool states it has nothing to do -O Do not override default Server entries with the ones found in package/source buildtool config -D Download nothing, use files in Source dir (useful for devel) -d Only to be used in conjunction with the "source" target. Only download files, don't invoke the source action on buildtool.mk -t toolchain Build using the specified toolchain (or actually build that toolchain if srcname = "toolchain") Value of "toolchain" is e.g. i486-unknown-linux-uclibc
Getting information about the sources that can be built
Buildtool can only build sources/packages that it has a configuration for (see the next section on how to create such a configuration). To get a listing of the sources that are configured for buildtool type:
This will give you a list (and a short description) of all sources/packages that can be built. An example output looks like this:
./buildtool.pl describe The following packages and sources are available: Sources: --------------------------------------------------------- openssh openssh tcp_wrappers Wietse Venema's TCP wrappers library helloworld classical example openssl Secure Sockets Layer and cryptography libraries and tools zlib zLib shared libraries linux kernel source only package Packages: --------------------------------------------------------- toolchain this holds everything to build cross build env kernel kernel pseudo package
The difference between sources and packages
You might be a bit confused why in the above output there are Sources and Packages. The main reason for having sources and packages separately is that there are some sources out there that produce programs used in several packages. Or in other words, it sometimes makes sense to just build the sources (like util-linux for example) only once and create several packages out of the resulting binaries. (Although the developer team did not strictly obey this idea in the past.)
Getting information about sources and packages that already has been built
shows a list of already build packages/sources.
Downloading the sources for a package
./buildtool.pl source somepackageName
will cause the sources of somepackageName to be "downloaded" (and before that, all its dependencies, if there are any that haven't been downloaded yet).
Upstream sources are now copied into the Bering-uClibc 5.x Git repository, since this avoids problems when upstream download sites go off-line and makes it easier to distribute copies of all the source code as required for compliance with some Open Source licenses.
As a result, these sources are "downloaded" (from the Internet) when the Sourceforge-hosted Git repository is "cloned", but an important further step is to create symbolic links to the "local repository" (directory
repo/) from the
source/ directory, so this is what now happens when sources are "downloaded".
After that, the makefile
buildtool.mk for this source is executed for target source. This has the effect that the sources will be extracted, and patches (if there are any) are applied.
If all this is completed without error, the sources of somepackageName are ready to be compiled.
Compiling the sources for a package
./buildtool.pl build somepackageName
will cause buildtool to actually compile the source and install it to the
build/ directory (and all its dependencies). After this is completed successfully, all binaries required for the lrp package (linked against uClibc) reside in the
Note that [toolchain] in the example above will be replaced with the toolchain you choosed in
conf/buildtool.conf as toolchain. By default it points to i486-unknown-linux-uclibc.
Creating buildtool sources/packages
For each package, five to six files control how it's built:
- This file defines which files to include - usually the servers configuration file
conf/servers.cfg, local definitions in
conf/sources.localand all configuration files in the directory
- Defines the servers where the files for the packages are downloaded from. The most used "server" location is localrepo, which points to your local copy of LEAF's git repository. That way you can build the toolchain and the packages even if an upstream site is not available. Therefor, and because it's a requirement of Sourceforge, when hosting LEAF, that all sources have to be available, you usually don't have to touch this file. Instead add the sources along with the other files needed to build a package into the git repository.
conf/sources.ddirectory holds the configuration files for sources and packages. For each package/source a new file with the package/source name in
conf/sources.dwith the extension cfg has to be created. The configuration file defines the Server (referring to a server defined in
conf/servers.cfg) where all files belonging to a package (or source) can be downloaded from (usually localrepo), the Directory on that server, the Revision (usually HEAD) a short Description of the package (source) and the dependencies for that package (source). All other files that are part of a package and should be downloaded from somewhere have to be explicitly named in the
buildtool.cfgwhich is specific to the package.
- New since Bering-uClibc 4.x there is a fourth file,
conf/sources.local, which is "included" at the end of
conf/sources.cfgand is used to define local sources/packages, typically those being developed and debugged and not yet ready to be loaded into the master repository.
In addition there are two more package specific files:
- (package specific - will be downloaded with the information from
conf/sources.d/packagename.cfg). Defines where to get the sources/patches and also has the definitions for the package itself (used by buildpacket).
- (package specific - will be downloaded with the information from
buildtool.cfg). Makefile for actually building the sources.
Step by step guide to creating a simple config
This section should give you a quick glance at how the configuration for a specific source is created (but it won't show the hours of fighting with some source because it refuses to build - you will experience that soon enough when you create your own package. I don't want to discourage you, but cross-compiling often is more difficult than building on the target system). For a more complete overview, please see the following sections of this chapter.
For this exercise, we will create a package called hdsupp.lrp, which contains utilities for preparing a harddisk or Compact Flash disk for running under Bering uClibc. This package will contain
Because there is no combined tarball for
mkfs.msdos, we will have to create 3 different source definitions for buildtool. We will start with syslinux, since that is the easiest one.
By the time you read this, hdsupp will already be part of the buildtool checkout. This means that the settings for syslinux, util-linux and dosfstools in
conf/sources.d will already be there. You can decide to either just skip that section, or (for the sake of learning something) delete that part of
conf/sources.d and enter the information again manually. The following sections will assume that buildtool knows nothing about syslinux, util-linux and dosfstools.
Creating the source definition for syslinux, util-linux, dosfstools
First of all we need to create the files
conf/sources.d/dosfstools.cfg containing a source definition for syslinux, util-linux-ng and dosfstools.
We start with
<Source syslinux> Server = localrepo Directory = syslinux Description = Syslinux boot-loader plus an MBR <Requires> Name = toolchain </Requires> </Source>
<Source util-linux> Server = localrepo Directory = util-linux-ng Description = fdisk and losetup <Requires> Name = toolchain </Requires> </Source>
<Source dosfstools> Server = localrepo Directory = dosfstools Description = provides mkfs.dos <Requires> Name = toolchain </Requires> </Source>
For now, we'll just pretend that the
buildtool.cfg file will be in the leaf Git repository. In this section, we will create everything needed for syslinux - the other sources will be taken care of later.
Next, we need to create a directory called
will do the job. This directory will contain
buildtool.mk and the syslinux sources.
Create a new file
repo/syslinux/buildtool.cfg and enter the following information.
<File buildtool.mk> Server = localrepo Directory = syslinux </File> <File syslinux-4.03.tar.bz2> Server = localrepo envname = SYSLINUX_SOURCE </File>
The two server definitions tell buildtool which server it should download the files from. In almost every case it is "localrepo" which refers to your local copy of the Bering-uClibc 5.x Git repository. Next we define the file definitions themselves.
envname parameter - with this definition in the cfg-file, buildtool will set an environment variable (named
SYSLINUX_SOURCE in this case) containing the filename of the syslinux sources (
syslinux-4.03.tar.bz2 in our example). This saves us from having to hardcode any version-specific information into the makefile. Specifying
envname is not mandatory, but it keeps things simple for upgrading to a new version of a source.
Creating the makefile for syslinux
Obviously, this guide will not be able to teach you how to write makefiles, or what to do when there are errors during build. We'll simply focus on creating a makefile that produces the binaries and conforms to what buildtool expects.
Create a new file
repo/syslinux/buildtool.mk and enter the following contents:
############################################################# # buildtool makefile for syslinux ############################################################# SYSLINUX_DIR:=$(shell $(BT_TGZ_GETDIRNAME) $(SYSLINUX_SOURCE) 2>/dev/null ) SYSLINUX_TARGET_DIR:=$(BT_BUILD_DIR)/syslinux $(SYSLINUX_DIR)/.source: $(BT_SETUP_BUILDDIR) -v $(SYSLINUX_SOURCE) touch $(SYSLINUX_DIR)/.source $(SYSLINUX_DIR)/.build: $(SYSLINUX_DIR)/.source mkdir -p $(SYSLINUX_TARGET_DIR)/usr/bin mkdir -p $(SYSLINUX_TARGET_DIR)/usr/share/syslinux $(MAKE) $(MAKEOPTS) CC=$(TARGET_CC) -C $(SYSLINUX_DIR) installer cp -a $(SYSLINUX_DIR)/core/pxelinux.0 $(SYSLINUX_TARGET_DIR)/usr/share/syslinux/ cp -a $(SYSLINUX_DIR)/core/isolinux.bin $(SYSLINUX_TARGET_DIR)/usr/share/syslinux/ cp -a $(SYSLINUX_DIR)/gpxe/gpxelinux.0 $(SYSLINUX_TARGET_DIR)/usr/share/syslinux/ cp -a $(SYSLINUX_DIR)/mtools/syslinux $(SYSLINUX_TARGET_DIR)/usr/bin/ cp -a $(SYSLINUX_DIR)/mbr/mbr.bin $(SYSLINUX_TARGET_DIR)/usr/share/syslinux/ -$(BT_STRIP) $(BT_STRIP_BINOPTS) $(SYSLINUX_TARGET_DIR)/usr/bin/* cp -a $(SYSLINUX_TARGET_DIR)/* $(BT_STAGING_DIR) touch $(SYSLINUX_DIR)/.build source: $(SYSLINUX_DIR)/.source build: $(SYSLINUX_DIR)/.build clean: -rm $(SYSLINUX_DIR)/.build rm -rf $(SYSLINUX_TARGET_DIR) $(MAKE) -C $(SYSLINUX_DIR) clean srcclean: rm -rf $(SYSLINUX_DIR)
As with all makefiles, make sure that the delimiter in front of the commands (like $(MAKE) -C $(SOURCE_DIR) clean above) is a tab and not spaces.
Creating the source definition for fdisk
On Debian, fdisk (and also mkfs.minix, which we will use for the hdsupp package as well) is part of the util-linux-ng package. So, we use the sources that are luckily already provided in the Bering-uClibc local repository.
Create a directory
repo/util-linux-ng/ and a new file
repo/util-linux-ng/buildtool.cfg and enter the following information.