Difference between revisions of "Bering-uClibc 5.x - Developer Guide - Compiling Source Code"

From bering-uClibc
Jump to: navigation, search
(Compiling the sources for a package)
(Added Best Practice Notes section - other developers please expand! :-))
 
(29 intermediate revisions by 2 users not shown)
Line 7: Line 7:
 
|}
 
|}
 
----
 
----
 
  
 
==Introduction==
 
==Introduction==
Line 104: Line 103:
  
 
If all this is completed without error, the sources of <span class="emphasis">''somepackageName''</span> are ready to be compiled.
 
If all this is completed without error, the sources of <span class="emphasis">''somepackageName''</span> are ready to be compiled.
 
  
 
===Compiling the sources for a package===
 
===Compiling the sources for a package===
Line 116: Line 114:
  
  
<!--
 
 
==Creating buildtool sources/packages==
 
==Creating buildtool sources/packages==
 +
===Overview===
 +
For each package, five to six files control how it's built:
 +
; <span class="term">conf/sources.cfg</span>
 +
: This file defines which files to include - usually the servers configuration file <code class="filename">conf/servers.cfg</code>, local definitions in <code class="filename">conf/sources.local</code> and all configuration files in the directory <code class="filename">conf/sources.d</code>.
 +
; <span class="term">conf/servers.cfg</span>
 +
: Defines the servers where the files for the packages are downloaded from. The most used "server" location is <tt>localrepo</tt>, which points to your local copy of LEAF's git repository. That way you can build the toolchain and the packages even if an upstream site is not available. Therefor, and because it's a requirement of Sourceforge, when hosting LEAF, that all sources have to be available, you usually don't have to touch this file. Instead add the sources along with the other files needed to build a package into the git repository.
 +
; <span class="term">conf/sources.d/[package|source].cfg</span>
 +
: The <code class="filename">conf/sources.d</code> directory holds the configuration files for sources and packages. For each package/source a new file with the package/source name in <code class="filename">conf/sources.d</code> with the extension <tt>cfg</tt> has to be created. The configuration file defines the Server (referring to a server defined in <code class="filename">conf/servers.cfg</code>) where all files belonging to a package (or source) can be downloaded from (usually <tt>localrepo</tt>), the Directory on that server, the Revision (usually <tt>HEAD</tt>) a short Description of the package (source) and the dependencies for that package (source). All other files that are part of a package and should be downloaded from somewhere have to be explicitly named in the <code class="filename">buildtool.cfg</code> which is specific to the package.
 +
; <span class="term">conf/sources.local</span>
 +
: New since [[Bering-uClibc 4.x]] there is a fourth file, <code class="filename">conf/sources.local</code>, which is "included" at the end of <code class="filename">conf/sources.cfg</code> and is used to define local sources/packages, typically those being developed and debugged and not yet ready to be loaded into the master repository.
  
===Overview===
+
In addition there are two more '''package specific''' files:  
For each package, 3 files control how it's built:
+
; <span class="term">conf/sources.cfg</span>
+
: Defines the servers that the config files for the packages are downloaded from, and the packages/sources themselves as well as the dependencies of a given package. All other files that are part of a package and should be downloaded from somewhere have to be explicitly named in the <code class="filename">buildtool.cfg</code> which is specific to the package. The file <code class="filename">conf/sources.cfg</code> is the same for all packages/sources, and you can find it in the <code class="filename">conf/</code> subdir in your Buildtool root directory.
+
: New with [[Bering-uClibc 4.x]] is <code class="filename">conf/sources.local</code> which is "included" at the end of <code class="filename">conf/sources.cfg</code> and is used to define local sources/packages, typically those being developed and debugged and not yet ready to be loaded into the master repository.
+
 
; <span class="term">buildtool.cfg</span>
 
; <span class="term">buildtool.cfg</span>
: (package specific - will be downloaded with the information from <code class="filename">conf/sources.cfg</code>). Defines where to get the sources/patches and also has the definitions for the package itself (used by buildpacket).
+
: (package specific - will be downloaded with the information from <code class="filename">conf/sources.d/<tt>packagename</tt>.cfg</code>). Defines where to get the sources/patches and also has the definitions for the package itself (used by buildpacket).
 
; <span class="term">buildtool.mk</span>
 
; <span class="term">buildtool.mk</span>
 
: (package specific - will be downloaded with the information from <code class="filename">buildtool.cfg</code>). Makefile for actually building the sources.
 
: (package specific - will be downloaded with the information from <code class="filename">buildtool.cfg</code>). Makefile for actually building the sources.
Line 136: Line 139:
 
Because there is no combined tarball for <code class="filename">syslinux</code>, <code class="filename">fdisk</code> and <code class="filename">mkfs.msdos</code>, we will have to create 3 different source definitions for buildtool. We will start with syslinux, since that is the easiest one.
 
Because there is no combined tarball for <code class="filename">syslinux</code>, <code class="filename">fdisk</code> and <code class="filename">mkfs.msdos</code>, we will have to create 3 different source definitions for buildtool. We will start with syslinux, since that is the easiest one.
  
By the time you read this, hdsupp will already be part of the buildtool checkout. This means that the settings for syslinux, util-linux and dosfstools in <code class="filename">conf/sources.cfg</code> will already be there. You can decide to either just skip that section, or (for the sake of learning something) delete that part of <code class="filename">conf/sources.cfg</code> and enter the information again manually. The following sections will assume that buildtool knows nothing about syslinux, util-linux and dosfstools.
+
By the time you read this, hdsupp will already be part of the buildtool checkout. This means that the settings for syslinux, util-linux and dosfstools in <code class="filename">conf/sources.d</code> will already be there. You can decide to either just skip that section, or (for the sake of learning something) delete that part of <code class="filename">conf/sources.d</code> and enter the information again manually. The following sections will assume that buildtool knows nothing about syslinux, util-linux and dosfstools.
  
 
====Creating the source definition for syslinux, util-linux, dosfstools====
 
====Creating the source definition for syslinux, util-linux, dosfstools====
First of all we need to edit <code class="filename">conf/sources.cfg</code> to contain a source definition for syslinux, util-linux, dosfstools.
+
First of all we need to create the files <code class="filename">conf/sources.d/syslinux.cfg</code>, <code class="filename">conf/sources.d/util-linux-ng.cfg</code> and <code class="filename">conf/sources.d/dosfstools.cfg</code> containing a source definition for syslinux, util-linux-ng and dosfstools.
  
 +
We start with <code class="filename">conf/sources.d/syslinux.cfg</code>
 
  &lt;Source syslinux&gt;
 
  &lt;Source syslinux&gt;
 
   Server = localrepo
 
   Server = localrepo
Line 146: Line 150:
 
   Description = Syslinux boot-loader plus an MBR
 
   Description = Syslinux boot-loader plus an MBR
 
   &lt;Requires&gt;
 
   &lt;Requires&gt;
     Name = buildenv
+
     Name = util-linux-ng
 +
    Name = toolchain
 
   &lt;/Requires&gt;
 
   &lt;/Requires&gt;
 
  &lt;/Source&gt;
 
  &lt;/Source&gt;
+
 
 +
followed by <code class="filename">conf/sources.d/util-linux-ng.cfg</code>
 
  &lt;Source util-linux&gt;
 
  &lt;Source util-linux&gt;
 
   Server = localrepo
 
   Server = localrepo
   Directory = util-linux
+
   Directory = util-linux-ng
   Description = provides fdisk and mkfs.minix
+
   Description = fdisk and losetup
 
   &lt;Requires&gt;
 
   &lt;Requires&gt;
     Name = buildenv
+
     Name = toolchain
 +
    Name = kernel
 
   &lt;/Requires&gt;
 
   &lt;/Requires&gt;
 
  &lt;/Source&gt;
 
  &lt;/Source&gt;
+
 
 +
and finally <code class="filename">conf/sources.d/dosfstools.cfg</code>
 
  &lt;Source dosfstools&gt;
 
  &lt;Source dosfstools&gt;
 
   Server = localrepo
 
   Server = localrepo
 
   Directory = dosfstools
 
   Directory = dosfstools
 +
  Description = provides mkfs.dos
 
   &lt;Requires&gt;
 
   &lt;Requires&gt;
     Name = buildenv
+
     Name = toolchain
   &lt;/Requires&gt;  Description = provides mkfs.dos
+
    Name = libiconv
 +
   &lt;/Requires&gt;   
 
  &lt;/Source&gt;
 
  &lt;/Source&gt;
  
 
For now, we'll just pretend that the <code class="filename">buildtool.cfg</code> file will be in the leaf Git repository. In this section, we will create everything needed for syslinux - the other sources will be taken care of later.
 
For now, we'll just pretend that the <code class="filename">buildtool.cfg</code> file will be in the leaf Git repository. In this section, we will create everything needed for syslinux - the other sources will be taken care of later.
  
Next, we need to create a directory called <code class="filename">syslinux</code> in <code class="filename">source</code>. Doing
+
Next, we need to create a directory called <code class="filename">syslinux</code> in <code class="filename">repo</code>. Doing
  
 
  mkdir repo/syslinux
 
  mkdir repo/syslinux
Line 182: Line 192:
 
  &lt;/File&gt;
 
  &lt;/File&gt;
 
   
 
   
  &lt;File syslinux-4.03.tar.bz2&gt;
+
  &lt;File syslinux-4.07.tar.xz&gt;
 
   Server = localrepo
 
   Server = localrepo
   envname = SOURCE_TGZ
+
   envname = SYSLINUX_SOURCE
 
  &lt;/File&gt;
 
  &lt;/File&gt;
  
 
The two server definitions tell buildtool which server it should download the files from.
 
The two server definitions tell buildtool which server it should download the files from.
In almost every case it is "<tt>localrepo</tt>" which refers to your local copy of the [[Bering-uClibc 4.x]] Git repository.
+
In almost every case it is "<tt>localrepo</tt>" which refers to your local copy of the [[Bering-uClibc 5.x]] Git repository.
Next we define the file definitiones themselves.
+
Next we define the file definitions themselves.
 
+
Note the <code class="filename">envname</code> parameter - with this definition in the cfg-file, buildtool will set an environment variable (named <code class="filename">SOURCE_TGZ</code> in this case) containing the filename of the syslinux sources (<code class="filename">syslinux-4.03.tar.bz2</code> in our example). This saves us from having to hardcode any version-specific information into the makefile. Specifying <code class="filename">envname</code> is not mandatory, but it keeps things simple for upgrading to a new version of a source.
+
  
 +
Note the <code class="filename">envname</code> parameter - with this definition in the cfg-file, buildtool will set an environment variable (named <code class="filename">SYSLINUX_SOURCE</code> in this case) containing the filename of the syslinux sources (<code class="filename">syslinux-4.07.tar.xz</code> in our example). This saves us from having to hardcode any version-specific information into the makefile. Specifying <code class="filename">envname</code> is not mandatory, but it keeps things simple for upgrading to a new version of a source.
  
 
====Creating the makefile for syslinux====
 
====Creating the makefile for syslinux====
 
Obviously, this guide will not be able to teach you how to write makefiles, or what to do when there are errors during build. We'll simply focus on creating a makefile that produces the binaries and conforms to what buildtool expects.
 
Obviously, this guide will not be able to teach you how to write makefiles, or what to do when there are errors during build. We'll simply focus on creating a makefile that produces the binaries and conforms to what buildtool expects.
  
Create a new file <code class="filename">source/syslinux/buildtool.mk</code> and enter the following contents:
+
Create a new file <code class="filename">repo/syslinux/buildtool.mk</code> and enter the following contents:
  
  <nowiki> #############################################################
+
  #############################################################
#
+
 
  # buildtool makefile for syslinux
 
  # buildtool makefile for syslinux
#
 
 
  #############################################################
 
  #############################################################
 
include $(MASTERMAKEFILE)
 
 
   
 
   
  SOURCE_DIR:=$(shell $(BT_TGZ_GETDIRNAME) $(SOURCE_TGZ) 2>/dev/null )
+
  SYSLINUX_DIR:=$(shell $(BT_TGZ_GETDIRNAME) $(SYSLINUX_SOURCE) 2>/dev/null )
  ifeq ($(SOURCE_DIR),)
+
  SYSLINUX_TARGET_DIR:=$(BT_BUILD_DIR)/syslinux
  SOURCE_DIR:=$(shell cat DIRNAME)
+
  endif
+
$(SYSLINUX_DIR)/.source:
  TARGET_DIR:=$(BT_BUILD_DIR)/syslinux
+
$(BT_SETUP_BUILDDIR) -v $(SYSLINUX_SOURCE)
 +
touch $(SYSLINUX_DIR)/.source
 +
   
 +
$(SYSLINUX_DIR)/.build: $(SYSLINUX_DIR)/.source
 +
mkdir -p $(SYSLINUX_TARGET_DIR)/usr/bin
 +
mkdir -p $(SYSLINUX_TARGET_DIR)/usr/share/syslinux
 +
$(MAKE) $(MAKEOPTS) CC=$(TARGET_CC) -C $(SYSLINUX_DIR) installer
 +
cp -a $(SYSLINUX_DIR)/core/pxelinux.0 $(SYSLINUX_TARGET_DIR)/usr/share/syslinux/
 +
cp -a $(SYSLINUX_DIR)/core/isolinux.bin $(SYSLINUX_TARGET_DIR)/usr/share/syslinux/
 +
cp -a $(SYSLINUX_DIR)/gpxe/gpxelinux.0 $(SYSLINUX_TARGET_DIR)/usr/share/syslinux/
 +
cp -a $(SYSLINUX_DIR)/mtools/syslinux $(SYSLINUX_TARGET_DIR)/usr/bin/
 +
cp -a $(SYSLINUX_DIR)/mbr/mbr.bin $(SYSLINUX_TARGET_DIR)/usr/share/syslinux/
 +
-$(BT_STRIP) $(BT_STRIP_BINOPTS) $(SYSLINUX_TARGET_DIR)/usr/bin/*
 +
cp -a $(SYSLINUX_TARGET_DIR)/* $(BT_STAGING_DIR)
 +
touch $(SYSLINUX_DIR)/.build
 +
   
 +
  source: $(SYSLINUX_DIR)/.source
 +
 +
build: $(SYSLINUX_DIR)/.build
 
   
 
   
source:
 
  bzcat $(SOURCE_TGZ) |  tar -xvf -
 
  echo $(SOURCE_DIR) > DIRNAME
 
  -mkdir $(TARGET_DIR)
 
 
build:
 
  export LANG=en_US  # Hack for Redhat 9
 
  export CC=$(TARGET_CC)
 
  $(MAKE) -C $(SOURCE_DIR) syslinux
 
  -$(BT_STRIP) $(BT_STRIP_BINOPTS) $(SOURCE_DIR)/syslinux
 
  $(MAKE) -C $(SOURCE_DIR) INSTALLROOT=$(TARGET_DIR) install
 
  cp $(SOURCE_DIR)/mbr.bin $(TARGET_DIR)/usr/bin/mbr.bin
 
  mkdir -p $(BT_STAGING_DIR)/usr/bin
 
  -cp $(TARGET_DIR)/usr/bin/syslinux $(BT_STAGING_DIR)/usr/sbin/
 
  -cp $(TARGET_DIR)/usr/bin/mbr.bin $(BT_STAGING_DIR)/usr/sbin/
 
 
 
  clean:
 
  clean:
  rm -rf $(TARGET_DIR)
+
-rm $(SYSLINUX_DIR)/.build
  $(MAKE) -C $(SOURCE_DIR) clean</nowiki>
+
rm -rf $(SYSLINUX_TARGET_DIR)
 +
$(MAKE) -C $(SYSLINUX_DIR) clean
 +
 +
srcclean:
 +
rm -rf $(SYSLINUX_DIR)
  
 
As with all makefiles, make sure that the delimiter in front of the commands (like <span class="command">'''$(MAKE) -C $(SOURCE_DIR) clean'''</span> above) is a <span class="emphasis">''tab''</span> and not <span class="emphasis">''spaces''</span>.
 
As with all makefiles, make sure that the delimiter in front of the commands (like <span class="command">'''$(MAKE) -C $(SOURCE_DIR) clean'''</span> above) is a <span class="emphasis">''tab''</span> and not <span class="emphasis">''spaces''</span>.
  
 
====Creating the source definition for fdisk====
 
====Creating the source definition for fdisk====
On Debian, fdisk (and also mkfs.minix, which we will use for the hdsupp package as well) is part of the util-linux package. So, we use the sources that are luckily already provided in the bering uClibc CVS (no need to pretend they're in CVS this time).
+
On Debian, fdisk (and also mkfs.minix, which we will use for the hdsupp package as well) is part of the util-linux-ng package. So, we use the sources that are luckily already provided in the Bering-uClibc local repository.
  
Create a directory <code class="filename">source/util-linux/</code> and a new file <code class="filename">source/util-linux/buildtool.cfg</code> and enter the following information.
+
Create a directory <code class="filename">repo/util-linux-ng/</code> and a new file <code class="filename">repo/util-linux-ng/buildtool.cfg</code> and enter the following information:
  
&lt;Server cvs-sourceforge&gt;
 
  Type = viewcvs
 
  Name = leaf.cvs.sourceforge.net
 
  Serverpath = /leaf/src/bering-uclibc/apps
 
&lt;/Server&gt;
 
 
 
  &lt;File buildtool.mk&gt;
 
  &lt;File buildtool.mk&gt;
   Server = cvs-sourceforge
+
   Server = localrepo
 
   Revision = HEAD
 
   Revision = HEAD
   Directory = util-linux
+
   Directory = util-linux-ng
 
  &lt;/File&gt;
 
  &lt;/File&gt;
 
   
 
   
  &lt;File util-linux_2.11n.orig.tar.gz&gt;
+
  &lt;File util-linux-ng-2.18.tar.bz2&gt;
   Server = cvs-sourceforge
+
   Server = localrepo
   envname = UTIL_LINUX_SOURCE
+
  Revision = HEAD
 +
  Directory = util-linux-ng
 +
   envname = UTIL_LINUX_NG_SOURCE
 
  &lt;/File&gt;
 
  &lt;/File&gt;
 
   
 
   
&lt;File util-linux_2.11n-4.diff.gz&gt;So
 
  Server = cvs-sourceforge
 
  envname = UTIL_LINUX_PATCH1
 
&lt;/File&gt;
 
 
 
====The makefile for util-linux====
 
====The makefile for util-linux====
As with syslinux, we need to create a makefile. This time the source target will also include applying a patch
+
As with syslinux, we need to create a makefile. This time we also configure the build options.
  
  <nowiki> #############################################################
+
  <nowiki>  
#
+
#############################################################
# util-linux
+
#
#
+
# util-linux (+loop-AES)
#############################################################
+
#
 +
#############################################################
  
include $(MASTERMAKEFILE)
+
UTIL_LINUX_DIR:=$(shell $(BT_TGZ_GETDIRNAME) $(UTIL_LINUX_NG_SOURCE) 2>/dev/null )
+
UTIL_LINUX_DIR:=$(shell $(BT_TGZ_GETDIRNAME) $(UTIL_LINUX_SOURCE) 2>/dev/null )
+
ifeq ($(UTIL_LINUX_DIR),)
+
UTIL_LINUX_DIR:=$(shell cat DIRNAME)
+
endif
+
+
UTIL_LINUX_TARGET_DIR:=$(BT_BUILD_DIR)/util-linux
+
export CC=$(TARGET_CC)
+
export CFLAGS=-I$(BT_STAGING_DIR)/include
+
export LDFLAGS=-L$(BT_STAGING_DIR)/lib -L$(BT_STAGING_DIR)/usr/lib
+
  
source:
+
UTIL_LINUX_TARGET_DIR:=$(BT_BUILD_DIR)/util-linux-ng
  zcat $(UTIL_LINUX_SOURCE) |  tar -xvf -
+
OPT="-pipe -fomit-frame-pointer $(BT_COPT_FLAGS)"
  echo $(UTIL_LINUX_DIR) > DIRNAME
+
export $OPT
  zcat $(UTIL_LINUX_PATCH1) | patch -d $(UTIL_LINUX_DIR) -p1
+
  cd $(UTIL_LINUX_DIR) &amp;&amp; ./configure
+
  perl -i -p -e 's,HAVE_SLANG=yes,HAVE_SLANG=no,' $(UTIL_LINUX_DIR)/MCONFIG
+
  perl -i -p -e 's,LIBSLANG=-lslang,LIBSLANG=,' $(UTIL_LINUX_DIR)/MCONFIG
+
  -mkdir $(UTIL_LINUX_TARGET_DIR)
+
  
build:
+
LOOPAES_DIR:=loop-AES-v3.1d
  $(MAKE) CC=$(TARGET_CC) -C $(UTIL_LINUX_DIR) misc-utils
+
  $(MAKE) CC=$(TARGET_CC) -C $(UTIL_LINUX_DIR) disk-utils
+
  $(MAKE) CC=$(TARGET_CC) -C $(UTIL_LINUX_DIR) fdisk fdisk
+
  -$(BT_STRIP) $(BT_STRIP_BINOPTS) $(UTIL_LINUX_DIR)/fdisk/fdisk
+
  -$(BT_STRIP) $(BT_STRIP_BINOPTS) $(UTIL_LINUX_DIR)/disk-utils/mkfs.minix
+
  -cp $(UTIL_LINUX_DIR)/fdisk/fdisk $(UTIL_LINUX_TARGET_DIR)/
+
  -cp $(UTIL_LINUX_DIR)/disk-utils/mkfs.minix $(UTIL_LINUX_TARGET_DIR)/
+
  mkdir -p $(BT_STAGING_DIR)/sbin
+
  -cp $(UTIL_LINUX_DIR)/fdisk/fdisk $(BT_STAGING_DIR)/sbin/
+
  -cp $(UTIL_LINUX_DIR)/disk-utils/mkfs.minix $(BT_STAGING_DIR)/sbin/
+
  
clean:
+
$(UTIL_LINUX_DIR)/.source:
  rm -rf $(UTIL_LINUX_TARGET_DIR)
+
$(BT_SETUP_BUILDDIR) -v $(UTIL_LINUX_NG_SOURCE)
  $(MAKE) -C $(UTIL_LINUX_DIR) clean</nowiki>
+
touch $(UTIL_LINUX_DIR)/.source
 +
 
 +
$(UTIL_LINUX_DIR)/.configured: $(UTIL_LINUX_DIR)/.source
 +
(cd $(UTIL_LINUX_DIR) ; ./configure \
 +
--host=$(GNU_TARGET_NAME) \
 +
--build=$(GNU_BUILD_NAME) \
 +
--disable-nls --without-ncurses);
 +
touch $(UTIL_LINUX_DIR)/.configured
 +
 
 +
$(UTIL_LINUX_DIR)/.build: $(UTIL_LINUX_DIR)/.configured
 +
mkdir -p $(UTIL_LINUX_TARGET_DIR)/sbin
 +
mkdir -p $(UTIL_LINUX_TARGET_DIR)/lib
 +
$(MAKE) $(MAKEOPTS) -C $(UTIL_LINUX_DIR)/fdisk fdisk
 +
$(MAKE) $(MAKEOPTS) -C $(UTIL_LINUX_DIR)/mount losetup
 +
$(MAKE) $(MAKEOPTS) -C $(UTIL_LINUX_DIR)/misc-utils blkid findfs
 +
cp -a $(UTIL_LINUX_DIR)/fdisk/.libs/fdisk $(UTIL_LINUX_TARGET_DIR)/sbin/
 +
cp -a $(UTIL_LINUX_DIR)/mount/losetup $(UTIL_LINUX_TARGET_DIR)/sbin/
 +
cp -a $(UTIL_LINUX_DIR)/misc-utils/.libs/findfs $(UTIL_LINUX_TARGET_DIR)/sbin/
 +
cp -a $(UTIL_LINUX_DIR)/misc-utils/.libs/blkid $(UTIL_LINUX_TARGET_DIR)/sbin/
 +
cp -a $(UTIL_LINUX_DIR)/shlibs/blkid/src/.libs/libblkid.* $(UTIL_LINUX_TARGET_DIR)/lib/
 +
rm -f $(UTIL_LINUX_TARGET_DIR)/lib/libblkid.la
 +
cp -a $(UTIL_LINUX_DIR)/shlibs/blkid/src/libblkid.la $(UTIL_LINUX_TARGET_DIR)/lib/
 +
cp -a $(UTIL_LINUX_DIR)/shlibs/uuid/src/.libs/libuuid.* $(UTIL_LINUX_TARGET_DIR)/lib/
 +
rm -f $(UTIL_LINUX_TARGET_DIR)/lib/libuuid.la
 +
cp -a $(UTIL_LINUX_DIR)/shlibs/uuid/src/libuuid.la $(UTIL_LINUX_TARGET_DIR)/lib/
 +
perl -i -p -e "s,^libdir=.*$$,libdir='$(BT_STAGING_DIR)/lib\'," $(UTIL_LINUX_TARGET_DIR)/lib/*.la
 +
-$(BT_STRIP) $(BT_STRIP_BINOPTS) $(UTIL_LINUX_TARGET_DIR)/sbin/*
 +
-$(BT_STRIP) $(BT_STRIP_LIBOPTS) $(UTIL_LINUX_TARGET_DIR)/lib/*
 +
cp -a $(UTIL_LINUX_TARGET_DIR)/* $(BT_STAGING_DIR)
 +
touch $(UTIL_LINUX_DIR)/.build
 +
 
 +
source: $(UTIL_LINUX_DIR)/.source
 +
 
 +
build: $(UTIL_LINUX_DIR)/.build
  
This one shows where things get a bit trickier - for some reason, the configure-script that comes with util-linux insisted on building cfdisk (which needs ncurses). So, after running configure, we remove the definitions that cause this from MCONFIG (this is what the two calls to Perl do).
+
clean:
 +
-rm $(UTIL_LINUX_DIR)/.build
 +
-rm $(LOOPAES_DIR)/.build
 +
rm -rf $(UTIL_LINUX_TARGET_DIR)
 +
$(MAKE) -C $(UTIL_LINUX_DIR) clean
 +
 
 +
srcclean:
 +
rm -rf $(UTIL_LINUX_TARGET_DIR)
 +
rm -rf $(UTIL_LINUX_DIR)
 +
</nowiki>
 +
 
 +
This one shows where things get a bit trickier - we have to replace the libdir in repo/util_linux_ng/lib/*.la with the lib directory of our toolchain (this is what the call to Perl do).
  
 
====Source definition for dosfstools====
 
====Source definition for dosfstools====
Dosfstools provides mkfs.msdos. Again we need to create a directory for the sources/configs first. So, create a directory <code class="filename">source/dosfstools</code> and a new file <code class="filename">source/dosfstools/buildtool.cfg</code> with the following content:
+
Dosfstools provides mkfs.msdos. Again we need to create a directory for the sources/configs first. So, create a directory <code class="filename">repo/dosfstools</code> and a new file <code class="filename">repo/dosfstools/buildtool.cfg</code> with the following content:
 
+
&lt;Server cvs-sourceforge&gt;
+
  Type = viewcvs
+
  Name = leaf.cvs.sourceforge.net
+
  Serverpath = /leaf/src/bering-uclibc/apps
+
&lt;/Server&gt;
+
+
&lt;Server debian&gt;
+
  Type = http
+
  Name = ftp.debian.org
+
  Serverpath = debian/pool/main/d/dosfstools
+
&lt;/Server&gt;
+
+
 
  &lt;File buildtool.mk&gt;
 
  &lt;File buildtool.mk&gt;
   Server = cvs-sourceforge
+
   Server = localrepo
 
   Revision = HEAD
 
   Revision = HEAD
 
   Directory = dosfstools
 
   Directory = dosfstools
 
  &lt;/File&gt;
 
  &lt;/File&gt;
 
   
 
   
  &lt;File dosfstools_2.9.orig.tar.gz&gt;
+
  &lt;File dosfstools-3.0.26.tar.xz&gt;
   Server = debian
+
   Server = localrepo
 +
  Directory = dosfstools
 
   envname = DOSFSTOOLS_SOURCE
 
   envname = DOSFSTOOLS_SOURCE
 
  &lt;/File&gt;
 
  &lt;/File&gt;
 
   
 
   
&lt;File dosfstools_2.9-1.diff.gz&gt;
+
Pretty much like the definitions before.  
  Server = debian
+
  envname = DOSFSTOOLS_PATCH1
+
&lt;/File&gt;
+
 
+
Pretty much like the definitions before. Don't let yourself be fooled - despite the name ftp.debian.org, we can access this server using the http protocol.
+
  
 
====Makefile for dosfstools====
 
====Makefile for dosfstools====
 
As one would expect the makefile for dosfstools looks just like the other ones.
 
As one would expect the makefile for dosfstools looks just like the other ones.
  
  <nowiki> #############################################################
+
  <nowiki>#############################################################
#
+
# dosfstools
# dosfstools
+
#############################################################
#
+
#############################################################
+
  
include $(MASTERMAKEFILE)
+
DOSFSTOOLS_DIR:=$(CURDIR)/$(shell $(BT_TGZ_GETDIRNAME) $(DOSFSTOOLS_SOURCE) 2>/dev/null)
+
DOSFSTOOLS_TARGET_DIR:=$(BT_BUILD_DIR)/dosfstools
DOSFSTOOLS_DIR:=$(shell $(BT_TGZ_GETDIRNAME) $(DOSFSTOOLS_SOURCE) 2>/dev/null )
+
export CC=$(TARGET_CC)
ifeq ($(DOSFSTOOLS_DIR),)
+
DOSFSTOOLS_DIR:=$(shell cat DIRNAME)
+
endif
+
+
DOSFSTOOLS_TARGET_DIR:=$(BT_BUILD_DIR)/dosfstools
+
export CC=$(TARGET_CC)
+
  
source:
+
$(DOSFSTOOLS_DIR)/.source:
  zcat $(DOSFSTOOLS_SOURCE) |  tar -xvf -
+
zcat $(DOSFSTOOLS_SOURCE) |  tar -xvf -
  echo $(DOSFSTOOLS_DIR) > DIRNAME
+
touch $(DOSFSTOOLS_DIR)/.source
  zcat $(DOSFSTOOLS_PATCH1) | patch -d $(DOSFSTOOLS_DIR) -p1
+
  perl -i -p -e 's,CC\s*=\s*gcc,#CC = gcc,' $(DOSFSTOOLS_DIR)/Makefile
+
  perl -i -p -e 's,PREFIX\s*=.*,PREFIX=$(DOSFSTOOLS_TARGET_DIR),' \
+
    $(DOSFSTOOLS_DIR)/Makefile
+
  -mkdir $(DOSFSTOOLS_TARGET_DIR)
+
  
build:
+
$(DOSFSTOOLS_DIR)/.build: $(DOSFSTOOLS_DIR)/.source
  export PREFIX=$(DOSFSTOOLS_TARGET_DIR)
+
mkdir -p $(DOSFSTOOLS_TARGET_DIR)
  $(MAKE) CC=$(TARGET_CC) -C $(DOSFSTOOLS_DIR)
+
export PREFIX=$(DOSFSTOOLS_TARGET_DIR)
  $(MAKE) CC=$(TARGET_CC) -C $(DOSFSTOOLS_DIR) install
+
$(MAKE) $(MAKEOPTS) -C $(DOSFSTOOLS_DIR)
  -$(BT_STRIP) $(BT_STRIP_BINOPTS) $(DOSFSTOOLS_TARGET_DIR)/sbin/mkdosfs
+
$(MAKE) -C $(DOSFSTOOLS_DIR) install DESTDIR=$(DOSFSTOOLS_TARGET_DIR) PREFIX=/
  mkdir -p $(BT_STAGING_DIR)/sbin
+
-$(BT_STRIP) $(BT_STRIP_BINOPTS) $(DOSFSTOOLS_TARGET_DIR)/sbin/*
  -cp $(DOSFSTOOLS_TARGET_DIR)/sbin/mkfs.msdos $(BT_STAGING_DIR)/sbin/
+
-rm -rf $(DOSFSTOOLS_TARGET_DIR)/share
 +
cp -a $(DOSFSTOOLS_TARGET_DIR)/* $(BT_STAGING_DIR)/
 +
touch $(DOSFSTOOLS_DIR)/.build
  
clean:
+
source: $(DOSFSTOOLS_DIR)/.source
  rm -rf $(DOSFSTOOLS_TARGET_DIR)
+
 
  $(MAKE) -C $(DOSFSTOOLS_DIR) clean</nowiki>
+
build: $(DOSFSTOOLS_DIR)/.build
 +
 
 +
clean:
 +
rm -rf $(DOSFSTOOLS_TARGET_DIR)
 +
-rm $(DOSFSTOOLS_DIR)/.build
 +
$(MAKE) -C $(DOSFSTOOLS_DIR) clean
 +
 
 +
srcclean:
 +
rm -rf $(DOSFSTOOLS_DIR)</nowiki>
  
 
====Finishing up hdsupp====
 
====Finishing up hdsupp====
Finally - almost there. Hdsupp will be a "wrapper package" - it doesn't really compile anything, it just makes sure that all sources it needs have been compiled, copies the binaries to the staging dir (this part could also be done in the individual .mk files) and creates the lrp package.
+
Finally - almost there. Hdsupp will be a "wrapper package" - it doesn't really compile anything, it just makes shure that all sources it needs have been compiled, copies the binaries to the staging dir (this part could also be done in the individual .mk files) and creates the LRP Package.
  
In the beginning of this guide, we haven't added a definition for hdsupp to <code class="filename">conf/sources.cfg</code>. We'll change that now. Hdsupp is special, since one of its functions is to make sure all of the sources have been compiled already - it has <span class="emphasis">''dependencies''</span>. In buildtool, we use the <span class="command">'''required'''</span> tag to indicate dependencies. This tag lists all sources that must be built before hdsupp can be built. Notice that hdsupp is not specified as a source, but as a package (at this point, it makes no difference to buildtool, but it most likely will in the future - only packages will be able to produce lrp-files).
+
In the beginning of this guide, we haven't added a definition for hdsupp to <code class="filename">conf/sources.d</code>. We'll change that now. Hdsupp is special, since one of its functions is to make sure all of the sources have been compiled already - it has <span class="emphasis">''dependencies''</span>. In buildtool, we use the <span class="command">'''required'''</span> tag to indicate dependencies. This tag lists all sources that must be built before hdsupp can be built. Notice that hdsupp is not specified as a source, but as a package (at this point, it makes no difference to buildtool, but it most likely will in the future - only packages will be able to produce LRP files).
  
So, enter the following to <code class="filename">conf/sources.cfg</code>:
+
So, enter the following to <code class="filename">conf/sources.d/hdsupp.cfg</code>:
  
 
  &lt;Package hdsupp&gt;
 
  &lt;Package hdsupp&gt;
   Server = cvs-sourceforge
+
   Server = localrepo
 
   Revision = HEAD
 
   Revision = HEAD
 
   Directory = hdsupp
 
   Directory = hdsupp
   Description = Package providing tools for harddrives/CF disks
+
   Description = HD-Support metapackage containing the tools needed to make a harddisc or compact flash bootable
 
   &lt;Requires&gt;
 
   &lt;Requires&gt;
 
     Name = syslinux
 
     Name = syslinux
     Name = util-linux
+
     Name = util-linux-ng
 
     Name = dosfstools
 
     Name = dosfstools
 +
    Name = e2fsprogs
 +
    Name = hdparm
 
   &lt;/Requires&gt;
 
   &lt;/Requires&gt;
 
  &lt;/Package&gt;
 
  &lt;/Package&gt;
 +
 +
''Note that the hdsupp Package requires also e2fsprogs and hdparm in the Bering-uClibc 5.x version, whereas at the time the guide was originally written syslinux, util-linux and dosfstools has been all needed. So to finish the package you need also create Package definitions for e2fsprogs and hdparm.''
 +
  
 
=====Hdsupp package definition=====
 
=====Hdsupp package definition=====
As before we need to create a directory for hdsupp containing buildtool.cfg and buildtool.mk. Create a new directory <code class="filename">source/hdsupp</code>, a file <code class="filename">buildtool.cfg</code> and enter the following information to <code class="filename">buildtool.cfg</code>:
+
As before we need to create a directory for hdsupp containing buildtool.cfg and buildtool.mk. Create a new directory <code class="filename">repo/hdsupp</code>, a file <code class="filename">buildtool.cfg</code> and enter the following information to <code class="filename">buildtool.cfg</code>:
  
&lt;Server cvs-sourceforge&gt;
 
  Type = viewcvs
 
  Name = leaf.cvs.sourceforge.net
 
  Serverpath = /leaf/src/bering-uclibc/apps
 
&lt;/Server&gt;
 
 
 
  &lt;File buildtool.mk&gt;
 
  &lt;File buildtool.mk&gt;
   Server = cvs-sourceforge
+
   Server = localrepo
 
   Revision = HEAD
 
   Revision = HEAD
 
   Directory = hdsupp
 
   Directory = hdsupp
Line 419: Line 420:
 
As you can see, this one is extremely simple. Since hdsupp doesn't provide any sources, we only need to specify buildtool.mk.
 
As you can see, this one is extremely simple. Since hdsupp doesn't provide any sources, we only need to specify buildtool.mk.
  
This is not all we need to add here - there's more to get <code class="filename">buildpacket</code> to work. But this will be taken care of in the "Step by step" section of the buildpacket chapter. (TODO add link)
+
This is not all we need to add here - there's more to get <code class="filename">buildpacket.pl</code> to work. But this will be taken care of in the "Step by step" section of the buildpacket chapter [https://sourceforge.net/apps/mediawiki/leaf/index.php?title=Bering-uClibc_5.x_-_Developer_Guide_-_Building_a_Package buildpacket chapter ].
  
 
=====Hdsupp makefile=====
 
=====Hdsupp makefile=====
 
You may be wondering why we need a makefile at all, since hdsupp does not provide any sources. You are right - we don't <span class="emphasis">''need''</span> a makefile. But to keep buildtool happy (and to make sure that buildpacket can verify that hdsupp has actually been built), we will provide one that simply does nothing.
 
You may be wondering why we need a makefile at all, since hdsupp does not provide any sources. You are right - we don't <span class="emphasis">''need''</span> a makefile. But to keep buildtool happy (and to make sure that buildpacket can verify that hdsupp has actually been built), we will provide one that simply does nothing.
  
As before, create a new file called <code class="filename">buildtool.mk</code> in <code class="filename">sources/hdsupp</code> and enter the following contents:
+
As before, create a new file called <code class="filename">buildtool.mk</code> in <code class="filename">repo/hdsupp</code> and enter the following contents:
  
 
  <nowiki> #############################################################
 
  <nowiki> #############################################################
Line 443: Line 444:
 
   # nothing to be done</nowiki>
 
   # nothing to be done</nowiki>
  
That's it. You can now enter ./buildtool.pl build hdsupp and (hopefully) buildtool will download and compile everything needed. If something goes wrong, have a look at the buildtool-logfile (usually the error-messages are pretty clear about what's wrong).
+
That's it. You can now enter  
 +
./buildtool.pl build hdsupp  
 +
and (hopefully) buildtool will download and compile everything needed. If something goes wrong, have a look at the buildtool-logfile (usually the error-messages are pretty clear about what's wrong).
  
 +
 +
==Best Practice Notes==
 +
The following are the preferred approaches for various aspects of buildtool usage.
 +
 +
# Only specify the version of an upstream source package in one place.
 +
#* Set a variable called <tt>''package''_version</tt> on the first line of <code class="filename">buildtool.cfg</code>
 +
#* Use this variable to specify the name of the source <tt>.tar.gz</tt> or equivalent file within <code class="filename">buildtool.cfg</code>
 +
#* Use this variable to specify the <tt>Version</tt> for each Package defined in <code class="filename">buildtool.cfg</code>
 +
#* Set an environment variable called <tt>SOURCE</tt> to reference the source <tt>.tar.gz</tt> or equivalent file within <code class="filename">buildtool.cfg</code>
 +
#* Use this <tt>SOURCE</tt> environment variable within <code class="filename">buildtool.mk</code>
 +
#* Use <tt>$(BT_TGZ_GETDIRNAME)</tt> to extract the source directory name and version from the <tt>SOURCE</tt> file within <code class="filename">buildtool.mk</code>
 +
# Use the upstream source package's <tt>make install</tt> target to copy the run-time files to the <tt>build/</tt> directory.
 +
#* Write the  contents of <code class="filename">buildtool.mk</code> so as to compile and link the source code in the Package directory under <tt>source/</tt> and then run <tt>make install</tt> to copy the run-time files into the Package directory under <tt>build/</tt>.
 +
#* Add further lines to <code class="filename">buildtool.mk</code> to selectively copy required files under <tt>$(BT_STAGING_DIR)</tt>
 +
 +
 +
tbd
 +
<!--
 
===Buildtool reference===
 
===Buildtool reference===
  
Line 528: Line 549:
  
 
====buildtool.mk====
 
====buildtool.mk====
<code class="filename">buildtool.mk</code> is the makefile that will be used to extract the source archive, apply the patches and compile the source. It must include the "mastermakefile" <code class="filename">make/MasterInclude.mk</code>. Luckily, the environment variable MASTERMAKEFILE contains the file name of this file (including the full path). In short, every buildtool.mk file should start with:
+
<code class="filename">buildtool.mk</code> is the makefile that will be used to extract the source archive, apply the patches and compile the source.  
 
+
include $(MASTERMAKEFILE)
+
  
 
=====targets=====
 
=====targets=====
Line 665: Line 684:
 
and everything should work fine
 
and everything should work fine
 
-->
 
-->
 +
  
 
----
 
----

Latest revision as of 13:37, 31 December 2014

Compiling Source Code
Prev Bering-uClibc 5.x - Developer Guide Next

Introduction

What is buildtool?

Buildtool is a set of Perl scripts and Perl modules to build LRP files (LEAF Packages) from source. Everything in the build process is automated, from extracting the sources and applying patches to configuring and building the source and creating the LRP Package.

Why bother?

In very early LEAF releases it was usual to compile the sources and build the Packages manually. This could be error-prone, and where upstream source upgrades happened regularly it was tedious to have to repeat builds for new versions manually.

The current practice is for every Package to be generated automatically according to simple configuration definitions. This means that no special knowledge is required to re-build a particular Package, because all of the steps are captured in the configuration files. Some effort is required to prepare the initial buildtool configuration but many Packages are fairly "standard" and it is often possible to adapt the configuration from a similar package quite quickly.


Buildtool usage

Options

usage: ./buildtool.pl [option] command [pkgname|srcname] [...]

commands:
describe [pkgname|srcname]       shows descriptionlines of package
list [sourced|built]             shows a list of built/sourced packages and sources
dumpenv [pkgname|srcname]        dump the environment of buildtool and package
source [pkgname|srcname]         downloads, unpacks and patches
                                the wanted package/source
build [pkgname|srcname]          the same as source, but builds
                                and installs sources/packages also
pkglist [pkgname|srcname]        create a list with all dependencies
                                for the package given or all if no name given
buildclean [pkgname|srcname]     removes everything that is outside
                                the source dir
srcclean [pkgname|srcname]       same as buildclean + call make srcclean
remove [pkgname|srcname]         same as buildclean + remove everything from dldir
distclean                        remove everything
maketar                          make a tar for distribution
 
options:
-v                               just print version and exit
-f                               allows you to force the command even if the internal
                                state of buildtool states it has nothing to do
-O                               Do not override default Server entries with the ones
                                found in package/source buildtool config
-D                               Download nothing, use files in Source dir (useful for devel)
-d                               Only to be used in conjunction with the "source" target. 
                                Only download files, don't invoke the source action on buildtool.mk
-t toolchain                     Build using the specified toolchain
                                (or actually build that toolchain if srcname = "toolchain")
                                Value of "toolchain" is e.g. i486-unknown-linux-uclibc

Getting information about the sources that can be built

Buildtool can only build sources/packages that it has a configuration for (see the next section on how to create such a configuration). To get a listing of the sources that are configured for buildtool type:

 ./buildtool.pl describe

This will give you a list (and a short description) of all sources/packages that can be built. An example output looks like this:

 ./buildtool.pl describe

 The following packages and sources are available:

 Sources:
 ---------------------------------------------------------
 openssh          openssh
 tcp_wrappers     Wietse Venema's TCP wrappers library
 helloworld       classical example
 openssl          Secure Sockets Layer and cryptography
                  libraries and tools
 zlib             zLib shared libraries
 linux            kernel source only package

 Packages:
 ---------------------------------------------------------
 toolchain         this holds everything to build cross build env
 kernel           kernel pseudo package

The difference between sources and packages

You might be a bit confused why in the above output there are Sources and Packages. The main reason for having sources and packages separately is that there are some sources out there that produce programs used in several packages. Or in other words, it sometimes makes sense to just build the sources (like util-linux for example) only once and create several packages out of the resulting binaries. (Although the developer team did not strictly obey this idea in the past.)

Getting information about sources and packages that already has been built

The command

./buildtool.pl list

shows a list of already build packages/sources.

Downloading the sources for a package

Running

./buildtool.pl source somepackageName

will cause the sources of somepackageName to be "downloaded" (and before that, all its dependencies, if there are any that haven't been downloaded yet). Upstream sources are now copied into the Bering-uClibc 5.x Git repository, since this avoids problems when upstream download sites go off-line and makes it easier to distribute copies of all the source code as required for compliance with some Open Source licenses. As a result, these sources are "downloaded" (from the Internet) when the Sourceforge-hosted Git repository is "cloned", but an important further step is to create symbolic links to the "local repository" (directory repo/) from the source/ directory, so this is what now happens when sources are "downloaded".

After that, the makefile buildtool.mk for this source is executed for target source. This has the effect that the sources will be extracted, and patches (if there are any) are applied.

If all this is completed without error, the sources of somepackageName are ready to be compiled.

Compiling the sources for a package

Running

./buildtool.pl build somepackageName

will cause buildtool to actually compile the source and install it to the build/ directory (and all its dependencies). After this is completed successfully, all binaries required for the lrp package (linked against uClibc) reside in the build/[toolchain] directory.

Note that [toolchain] in the example above will be replaced with the toolchain you choosed in conf/buildtool.conf as toolchain. By default it points to i486-unknown-linux-uclibc.


Creating buildtool sources/packages

Overview

For each package, five to six files control how it's built:

conf/sources.cfg
This file defines which files to include - usually the servers configuration file conf/servers.cfg, local definitions in conf/sources.local and all configuration files in the directory conf/sources.d.
conf/servers.cfg
Defines the servers where the files for the packages are downloaded from. The most used "server" location is localrepo, which points to your local copy of LEAF's git repository. That way you can build the toolchain and the packages even if an upstream site is not available. Therefor, and because it's a requirement of Sourceforge, when hosting LEAF, that all sources have to be available, you usually don't have to touch this file. Instead add the sources along with the other files needed to build a package into the git repository.
conf/sources.d/[package|source].cfg
The conf/sources.d directory holds the configuration files for sources and packages. For each package/source a new file with the package/source name in conf/sources.d with the extension cfg has to be created. The configuration file defines the Server (referring to a server defined in conf/servers.cfg) where all files belonging to a package (or source) can be downloaded from (usually localrepo), the Directory on that server, the Revision (usually HEAD) a short Description of the package (source) and the dependencies for that package (source). All other files that are part of a package and should be downloaded from somewhere have to be explicitly named in the buildtool.cfg which is specific to the package.
conf/sources.local
New since Bering-uClibc 4.x there is a fourth file, conf/sources.local, which is "included" at the end of conf/sources.cfg and is used to define local sources/packages, typically those being developed and debugged and not yet ready to be loaded into the master repository.

In addition there are two more package specific files:

buildtool.cfg
(package specific - will be downloaded with the information from conf/sources.d/packagename.cfg). Defines where to get the sources/patches and also has the definitions for the package itself (used by buildpacket).
buildtool.mk
(package specific - will be downloaded with the information from buildtool.cfg). Makefile for actually building the sources.

Step by step guide to creating a simple config

This section should give you a quick glance at how the configuration for a specific source is created (but it won't show the hours of fighting with some source because it refuses to build - you will experience that soon enough when you create your own package. I don't want to discourage you, but cross-compiling often is more difficult than building on the target system). For a more complete overview, please see the following sections of this chapter.

For this exercise, we will create a package called hdsupp.lrp, which contains utilities for preparing a harddisk or Compact Flash disk for running under Bering uClibc. This package will contain fdisk, syslinux and mkfs.msdos.

Because there is no combined tarball for syslinux, fdisk and mkfs.msdos, we will have to create 3 different source definitions for buildtool. We will start with syslinux, since that is the easiest one.

By the time you read this, hdsupp will already be part of the buildtool checkout. This means that the settings for syslinux, util-linux and dosfstools in conf/sources.d will already be there. You can decide to either just skip that section, or (for the sake of learning something) delete that part of conf/sources.d and enter the information again manually. The following sections will assume that buildtool knows nothing about syslinux, util-linux and dosfstools.

Creating the source definition for syslinux, util-linux, dosfstools

First of all we need to create the files conf/sources.d/syslinux.cfg, conf/sources.d/util-linux-ng.cfg and conf/sources.d/dosfstools.cfg containing a source definition for syslinux, util-linux-ng and dosfstools.

We start with conf/sources.d/syslinux.cfg

<Source syslinux>
  Server = localrepo
  Directory = syslinux
  Description = Syslinux boot-loader plus an MBR
  <Requires>
    Name = util-linux-ng
    Name = toolchain
  </Requires>
</Source>

followed by conf/sources.d/util-linux-ng.cfg

<Source util-linux>
  Server = localrepo
  Directory = util-linux-ng
  Description = fdisk and losetup
  <Requires>
    Name = toolchain
    Name = kernel
  </Requires>
</Source>

and finally conf/sources.d/dosfstools.cfg

<Source dosfstools>
  Server = localrepo
  Directory = dosfstools
  Description = provides mkfs.dos
  <Requires>
    Name = toolchain
    Name = libiconv
  </Requires>  
</Source>

For now, we'll just pretend that the buildtool.cfg file will be in the leaf Git repository. In this section, we will create everything needed for syslinux - the other sources will be taken care of later.

Next, we need to create a directory called syslinux in repo. Doing

mkdir repo/syslinux

will do the job. This directory will contain buildtool.cfg, buildtool.mk and the syslinux sources.

Create a new file repo/syslinux/buildtool.cfg and enter the following information.

<File buildtool.mk>
  Server = localrepo
  Directory = syslinux
</File>

<File syslinux-4.07.tar.xz>
  Server = localrepo
  envname = SYSLINUX_SOURCE
</File>

The two server definitions tell buildtool which server it should download the files from. In almost every case it is "localrepo" which refers to your local copy of the Bering-uClibc 5.x Git repository. Next we define the file definitions themselves.

Note the envname parameter - with this definition in the cfg-file, buildtool will set an environment variable (named SYSLINUX_SOURCE in this case) containing the filename of the syslinux sources (syslinux-4.07.tar.xz in our example). This saves us from having to hardcode any version-specific information into the makefile. Specifying envname is not mandatory, but it keeps things simple for upgrading to a new version of a source.

Creating the makefile for syslinux

Obviously, this guide will not be able to teach you how to write makefiles, or what to do when there are errors during build. We'll simply focus on creating a makefile that produces the binaries and conforms to what buildtool expects.

Create a new file repo/syslinux/buildtool.mk and enter the following contents:

#############################################################
# buildtool makefile for syslinux
#############################################################

SYSLINUX_DIR:=$(shell $(BT_TGZ_GETDIRNAME) $(SYSLINUX_SOURCE) 2>/dev/null )
SYSLINUX_TARGET_DIR:=$(BT_BUILD_DIR)/syslinux

$(SYSLINUX_DIR)/.source:
	$(BT_SETUP_BUILDDIR) -v $(SYSLINUX_SOURCE)
	touch $(SYSLINUX_DIR)/.source

$(SYSLINUX_DIR)/.build: $(SYSLINUX_DIR)/.source
	mkdir -p $(SYSLINUX_TARGET_DIR)/usr/bin
	mkdir -p $(SYSLINUX_TARGET_DIR)/usr/share/syslinux
	$(MAKE) $(MAKEOPTS) CC=$(TARGET_CC) -C $(SYSLINUX_DIR) installer
	cp -a $(SYSLINUX_DIR)/core/pxelinux.0 $(SYSLINUX_TARGET_DIR)/usr/share/syslinux/
	cp -a $(SYSLINUX_DIR)/core/isolinux.bin $(SYSLINUX_TARGET_DIR)/usr/share/syslinux/
	cp -a $(SYSLINUX_DIR)/gpxe/gpxelinux.0 $(SYSLINUX_TARGET_DIR)/usr/share/syslinux/
	cp -a $(SYSLINUX_DIR)/mtools/syslinux $(SYSLINUX_TARGET_DIR)/usr/bin/
	cp -a $(SYSLINUX_DIR)/mbr/mbr.bin $(SYSLINUX_TARGET_DIR)/usr/share/syslinux/
	-$(BT_STRIP) $(BT_STRIP_BINOPTS) $(SYSLINUX_TARGET_DIR)/usr/bin/*
	cp -a $(SYSLINUX_TARGET_DIR)/* $(BT_STAGING_DIR)
	touch $(SYSLINUX_DIR)/.build

source: $(SYSLINUX_DIR)/.source

build: $(SYSLINUX_DIR)/.build

clean:
	-rm $(SYSLINUX_DIR)/.build
	rm -rf $(SYSLINUX_TARGET_DIR)
	$(MAKE) -C $(SYSLINUX_DIR) clean

srcclean:
	rm -rf $(SYSLINUX_DIR)

As with all makefiles, make sure that the delimiter in front of the commands (like $(MAKE) -C $(SOURCE_DIR) clean above) is a tab and not spaces.

Creating the source definition for fdisk

On Debian, fdisk (and also mkfs.minix, which we will use for the hdsupp package as well) is part of the util-linux-ng package. So, we use the sources that are luckily already provided in the Bering-uClibc local repository.

Create a directory repo/util-linux-ng/ and a new file repo/util-linux-ng/buildtool.cfg and enter the following information:

<File buildtool.mk>
  Server = localrepo
  Revision = HEAD
  Directory = util-linux-ng
</File>

<File util-linux-ng-2.18.tar.bz2>
  Server = localrepo
  Revision = HEAD
  Directory = util-linux-ng
  envname = UTIL_LINUX_NG_SOURCE
</File>

The makefile for util-linux

As with syslinux, we need to create a makefile. This time we also configure the build options.

 
#############################################################
#
# util-linux (+loop-AES)
#
#############################################################

UTIL_LINUX_DIR:=$(shell $(BT_TGZ_GETDIRNAME) $(UTIL_LINUX_NG_SOURCE) 2>/dev/null )

UTIL_LINUX_TARGET_DIR:=$(BT_BUILD_DIR)/util-linux-ng
OPT="-pipe -fomit-frame-pointer $(BT_COPT_FLAGS)"
export $OPT

LOOPAES_DIR:=loop-AES-v3.1d

$(UTIL_LINUX_DIR)/.source:
	$(BT_SETUP_BUILDDIR) -v $(UTIL_LINUX_NG_SOURCE) 
	touch $(UTIL_LINUX_DIR)/.source

$(UTIL_LINUX_DIR)/.configured: $(UTIL_LINUX_DIR)/.source
	(cd $(UTIL_LINUX_DIR) ; ./configure \
		--host=$(GNU_TARGET_NAME) \
		--build=$(GNU_BUILD_NAME) \
		--disable-nls --without-ncurses);
	touch $(UTIL_LINUX_DIR)/.configured

$(UTIL_LINUX_DIR)/.build: $(UTIL_LINUX_DIR)/.configured
	mkdir -p $(UTIL_LINUX_TARGET_DIR)/sbin
	mkdir -p $(UTIL_LINUX_TARGET_DIR)/lib
	$(MAKE) $(MAKEOPTS) -C $(UTIL_LINUX_DIR)/fdisk fdisk
	$(MAKE) $(MAKEOPTS) -C $(UTIL_LINUX_DIR)/mount losetup
	$(MAKE) $(MAKEOPTS) -C $(UTIL_LINUX_DIR)/misc-utils blkid findfs
	cp -a $(UTIL_LINUX_DIR)/fdisk/.libs/fdisk $(UTIL_LINUX_TARGET_DIR)/sbin/
	cp -a $(UTIL_LINUX_DIR)/mount/losetup $(UTIL_LINUX_TARGET_DIR)/sbin/
	cp -a $(UTIL_LINUX_DIR)/misc-utils/.libs/findfs $(UTIL_LINUX_TARGET_DIR)/sbin/
	cp -a $(UTIL_LINUX_DIR)/misc-utils/.libs/blkid $(UTIL_LINUX_TARGET_DIR)/sbin/
	cp -a $(UTIL_LINUX_DIR)/shlibs/blkid/src/.libs/libblkid.* $(UTIL_LINUX_TARGET_DIR)/lib/
	rm -f $(UTIL_LINUX_TARGET_DIR)/lib/libblkid.la
	cp -a $(UTIL_LINUX_DIR)/shlibs/blkid/src/libblkid.la $(UTIL_LINUX_TARGET_DIR)/lib/
	cp -a $(UTIL_LINUX_DIR)/shlibs/uuid/src/.libs/libuuid.* $(UTIL_LINUX_TARGET_DIR)/lib/
	rm -f $(UTIL_LINUX_TARGET_DIR)/lib/libuuid.la
	cp -a $(UTIL_LINUX_DIR)/shlibs/uuid/src/libuuid.la $(UTIL_LINUX_TARGET_DIR)/lib/
	perl -i -p -e "s,^libdir=.*$$,libdir='$(BT_STAGING_DIR)/lib\'," $(UTIL_LINUX_TARGET_DIR)/lib/*.la
	-$(BT_STRIP) $(BT_STRIP_BINOPTS) $(UTIL_LINUX_TARGET_DIR)/sbin/*
	-$(BT_STRIP) $(BT_STRIP_LIBOPTS) $(UTIL_LINUX_TARGET_DIR)/lib/*
	cp -a $(UTIL_LINUX_TARGET_DIR)/* $(BT_STAGING_DIR)
	touch $(UTIL_LINUX_DIR)/.build

source: $(UTIL_LINUX_DIR)/.source

build: $(UTIL_LINUX_DIR)/.build

clean:
	-rm $(UTIL_LINUX_DIR)/.build
	-rm $(LOOPAES_DIR)/.build
	rm -rf $(UTIL_LINUX_TARGET_DIR)
	$(MAKE) -C $(UTIL_LINUX_DIR) clean

srcclean:
	rm -rf $(UTIL_LINUX_TARGET_DIR)
	rm -rf $(UTIL_LINUX_DIR)
 

This one shows where things get a bit trickier - we have to replace the libdir in repo/util_linux_ng/lib/*.la with the lib directory of our toolchain (this is what the call to Perl do).

Source definition for dosfstools

Dosfstools provides mkfs.msdos. Again we need to create a directory for the sources/configs first. So, create a directory repo/dosfstools and a new file repo/dosfstools/buildtool.cfg with the following content:

<File buildtool.mk>
  Server = localrepo
  Revision = HEAD
  Directory = dosfstools
</File>

<File dosfstools-3.0.26.tar.xz>
  Server = localrepo
  Directory = dosfstools
  envname = DOSFSTOOLS_SOURCE
</File>

Pretty much like the definitions before.

Makefile for dosfstools

As one would expect the makefile for dosfstools looks just like the other ones.

#############################################################
# dosfstools
#############################################################

DOSFSTOOLS_DIR:=$(CURDIR)/$(shell $(BT_TGZ_GETDIRNAME) $(DOSFSTOOLS_SOURCE) 2>/dev/null)
DOSFSTOOLS_TARGET_DIR:=$(BT_BUILD_DIR)/dosfstools
export CC=$(TARGET_CC)

$(DOSFSTOOLS_DIR)/.source:
	zcat $(DOSFSTOOLS_SOURCE) |  tar -xvf -
	touch $(DOSFSTOOLS_DIR)/.source

$(DOSFSTOOLS_DIR)/.build: $(DOSFSTOOLS_DIR)/.source
	mkdir -p $(DOSFSTOOLS_TARGET_DIR)
	export PREFIX=$(DOSFSTOOLS_TARGET_DIR)
	$(MAKE) $(MAKEOPTS) -C $(DOSFSTOOLS_DIR)
	$(MAKE) -C $(DOSFSTOOLS_DIR) install DESTDIR=$(DOSFSTOOLS_TARGET_DIR) PREFIX=/
	-$(BT_STRIP) $(BT_STRIP_BINOPTS) $(DOSFSTOOLS_TARGET_DIR)/sbin/*
	-rm -rf $(DOSFSTOOLS_TARGET_DIR)/share
	cp -a $(DOSFSTOOLS_TARGET_DIR)/* $(BT_STAGING_DIR)/
	touch $(DOSFSTOOLS_DIR)/.build

source: $(DOSFSTOOLS_DIR)/.source

build: $(DOSFSTOOLS_DIR)/.build

clean:
	rm -rf $(DOSFSTOOLS_TARGET_DIR)
	-rm $(DOSFSTOOLS_DIR)/.build
	$(MAKE) -C $(DOSFSTOOLS_DIR) clean

srcclean:
	rm -rf $(DOSFSTOOLS_DIR)

Finishing up hdsupp

Finally - almost there. Hdsupp will be a "wrapper package" - it doesn't really compile anything, it just makes shure that all sources it needs have been compiled, copies the binaries to the staging dir (this part could also be done in the individual .mk files) and creates the LRP Package.

In the beginning of this guide, we haven't added a definition for hdsupp to conf/sources.d. We'll change that now. Hdsupp is special, since one of its functions is to make sure all of the sources have been compiled already - it has dependencies. In buildtool, we use the required tag to indicate dependencies. This tag lists all sources that must be built before hdsupp can be built. Notice that hdsupp is not specified as a source, but as a package (at this point, it makes no difference to buildtool, but it most likely will in the future - only packages will be able to produce LRP files).

So, enter the following to conf/sources.d/hdsupp.cfg:

<Package hdsupp>
  Server = localrepo
  Revision = HEAD
  Directory = hdsupp
  Description = HD-Support metapackage containing the tools needed to make a harddisc or compact flash bootable
  <Requires>
    Name = syslinux
    Name = util-linux-ng
    Name = dosfstools
    Name = e2fsprogs
    Name = hdparm
  </Requires>
</Package>

Note that the hdsupp Package requires also e2fsprogs and hdparm in the Bering-uClibc 5.x version, whereas at the time the guide was originally written syslinux, util-linux and dosfstools has been all needed. So to finish the package you need also create Package definitions for e2fsprogs and hdparm.


Hdsupp package definition

As before we need to create a directory for hdsupp containing buildtool.cfg and buildtool.mk. Create a new directory repo/hdsupp, a file buildtool.cfg and enter the following information to buildtool.cfg:

<File buildtool.mk>
  Server = localrepo
  Revision = HEAD
  Directory = hdsupp
</File>

As you can see, this one is extremely simple. Since hdsupp doesn't provide any sources, we only need to specify buildtool.mk.

This is not all we need to add here - there's more to get buildpacket.pl to work. But this will be taken care of in the "Step by step" section of the buildpacket chapter buildpacket chapter .

Hdsupp makefile

You may be wondering why we need a makefile at all, since hdsupp does not provide any sources. You are right - we don't need a makefile. But to keep buildtool happy (and to make sure that buildpacket can verify that hdsupp has actually been built), we will provide one that simply does nothing.

As before, create a new file called buildtool.mk in repo/hdsupp and enter the following contents:

 #############################################################
 #
 # hdsupp
 #
 #############################################################

 include $(MASTERMAKEFILE)

 source:
   # nothing to be done

 build:
   # nothing to be done

 clean:
   # nothing to be done

That's it. You can now enter

./buildtool.pl build hdsupp 

and (hopefully) buildtool will download and compile everything needed. If something goes wrong, have a look at the buildtool-logfile (usually the error-messages are pretty clear about what's wrong).


Best Practice Notes

The following are the preferred approaches for various aspects of buildtool usage.

  1. Only specify the version of an upstream source package in one place.
    • Set a variable called package_version on the first line of buildtool.cfg
    • Use this variable to specify the name of the source .tar.gz or equivalent file within buildtool.cfg
    • Use this variable to specify the Version for each Package defined in buildtool.cfg
    • Set an environment variable called SOURCE to reference the source .tar.gz or equivalent file within buildtool.cfg
    • Use this SOURCE environment variable within buildtool.mk
    • Use $(BT_TGZ_GETDIRNAME) to extract the source directory name and version from the SOURCE file within buildtool.mk
  2. Use the upstream source package's make install target to copy the run-time files to the build/ directory.
    • Write the contents of buildtool.mk so as to compile and link the source code in the Package directory under source/ and then run make install to copy the run-time files into the Package directory under build/.
    • Add further lines to buildtool.mk to selectively copy required files under $(BT_STAGING_DIR)


tbd



Prev Up Next