ModularDevelopersGuide

The X.Org Modular Tree Developer's Guide

Contents

  1. The X.Org Modular Tree Developer's Guide
    1. Introduction
    2. Required Tools
      1. Minimum requirements
      2. Suggested packages
      3. Additional requirements for developers
    3. Getting the modular tree
      1. From the Git repository
      2. From tarballs
        1. Mixing 7.1 and 7.0 tarballs
    4. Building the modular tree
      1. Understanding dependencies
      2. Building all modules with the build scripts
      3. Building all modules with jhbuild
      4. Building individual module components by hand
      5. Building only specific modules
      6. Compiling after updates
      7. Making tarballs
      8. Standard configuration options
    5. Using the modular tree
      1. X libraries
      2. X applications
      3. Running the X server
    6. Crosscompiling
    7. New Modules

Introduction

This guide is intended to be a repository of useful information that builders and developers need to know when working on the modular X.Org source tree. As new guidelines are developed, they should be added to this doc.

A more summarized guide to building X.org is at CompileXserverManually.

Required Tools

This section lists the tools required for various people involved with the project.

Minimum requirements

The following tools are required for all builders and developers:

(click here for a per-distribution list)

[***] Please add any other tools required (along with their version numbers) that we expect to be required for the initial modular source tree release.

Suggested packages

The following external dependencies are recommended for a fully functional build:

If there is a compile error in libXFont, it is likely because libXfont-X11R7.1-1.1.0 is incompatible with recent versions of freetype2 libraries. Installing http://xorg.freedesktop.org/releases/individual/lib/libXfont-1.2.0.tar.bz2 will fix that.

If you're building Xorg with XCB support and want to use xcb-utils then you need GNU gperf 3.0.1 or newer.

We have collected various known-good versions together here.

Additional requirements for developers

In addition to the minimum requirements listed above, the following tools are required for developers:

You can check the autoconf version by running autoconf --version and similarly for the rest. As above, known-good versions are available from the extras collection.

Getting the modular tree

In this section and the next, we assume you are working in /tmp with the sources in /tmp/src and the installed tree in /tmp/modular.

From the Git repository

Note: X.Org has moved all currently maintained modules to the Git repository. See the GitPage for more information and UsingGit on how to use Git.

The git_xorg script can be used to get all the modular components from Git. The first time it is run 'git clone' is used to get the code. On subsequent runs 'git pull' is used to get only those files which have changed. 

$ mkdir /tmp/src
$ cd /tmp/src
# Now, place the git_xorg script in this directory
$ chmod a+x git_xorg.sh

The script does not download fonts, which are necessary for compilation from scratch. To download fonts, add the following line to the script. 

do_dir xorg font "${font}"        # this line added to get fonts

The git_xorg script gets the modules anonymously. Edit the 'gitbase' variable in the script to specify a developer/login name if you intend to check in code. For example: "git+ssh://johndoe@git.freedesktop.org/git/xorg". See the UsingGit page for more information.

Now run the script to get everything. 

$ ./git_xorg.sh

The script will also check out the latest xcb source. If you do not want to do that, comment out or remove the last following line from the script: 

do_dir . xcb "${xcb}"

When done, you should see the following: 

$ ls -F
app/  data/  doc/  driver/  drm/  font/  git_xorg*  lib/  pixman/ proto/  util/  xcb/  xserver/

Later, you can get the latest code for any module by running git pull in the module's directory.

From tarballs

If you are building from the release tarballs, we assume that you have downloaded the tarballs into /tmp/tarballs and will be installing everything into /tmp/modular. The tarballs for the release are arranged into a hierarchy similar to the CVS modules. For example, you can find the tarballs for each of the app module components in the app subdirectory. In addition, there is another directory called everything that contains all of the tarballs included in the other modules for convenience.

For each tarball, there are two versions: one compressed with gzip (e.g., util-macros-1.0.0.tar.gz) and one compressed with bzip2 (e.g., util-macros-1.0.0.tar.bz2). You will only need one, so to save yourself some download time, you can download either the .gz or the .bz2 files. The tarball will need to be unpacked first.

To unpack a gzipped tarball: 

$ gzip -d < xorg-macros-1.0.0.tar.gz | tar xf -

To unpack a bzip2'd tarball: 

$ bzip2 -d < xorg-macros-1.0.0.tar.bz2 | tar xf -

The modular tree should now be ready to build.

IMPORTANT NOTE: If you are building 7.1 from tarballs, make sure you have all of 7.0 available too. build-from-tarballs.sh relies on files that are not present in the 7.1 source release directory.

Mixing 7.1 and 7.0 tarballs

If you have obtained the everything directories from the 7.1 and the 7.0 distributions, and you have the build-from-tarballs.sh script, you can use http://wiki.x.org/wiki/x-merge.scm?action=raw to merge the files into a single directory. Note that you'll need to rename the x-merge.scm script if you download it with wget. (Tip: wget --user-agent="Mozilla/5.0" <url>)

To use, be in a directory holding the build-from-tarballs.sh file and two subdirectories 7.1 and 7.0 with the tarballs for each release, and then run guile -s x-merge.scm. You will finish up with a new subdirectory called build-dir which you can go into and then run the build-from-tarballs.sh script as explained below.

Alternatively you can edit the top of the x-merge.scm file for your own setup.

Building the modular tree

There are multiple ways to build the module components.

For each of these build methods, you may build either from the Git tree or the release tarballs. In the sections below we will explain how to build, install and package the modular tree components using each of these methods.

Understanding dependencies

It is important to note that the module components have dependencies either on other module components or external packages. These dependencies determine the build order that you must follow. For example, all the module components depend on the macros module component (util/macros in Git or util-macros from the pre-packaged tarballs), so it must be built before any of the other module components. The build script and jhbuild both take care of these dependencies for you.

If you insist on building the modules by hand, you must do the dependency ordering yourself. It is recommended that you inspect the build scripts to determine the proper order. This is a lot of work, so building by hand is not recommended.

Building all modules with the build scripts

The first build method uses a script to automate the build process for each module component in the order required by their various interdependencies. For details on what each of these steps are, see the Building individual module components by hand section below.

If you are building from Git and have a complete tree checked out as outlined above, then you can build using the build.sh script in the util/modular directory. There are several options for this script (see below) but the only one required is the prefix that will be passed to each module component through the --prefix= command-line option. Since many of the commands used to build module components are themselves components in the tree, your PATH needs to be set to include prefix/bin so that later components can find the programs installed by the earlier ones. For example,

Note: The build script does not currently build xcb/pthread-stubs. This package is a default requirement for xcb. You may have to edit build.sh or the xcb configuration to reflect this. 

$ cd /tmp/src/
$ PATH=/tmp/modular/bin:$PATH ./util/modular/build.sh /tmp/modular

The options to the build.sh script are: 

Usage: ./build.sh [options] prefix
  where options are:
  -d : run make distcheck in addition to others
  -D : run make dist in addition to others
  -c : run make clean in addition to others
  -m path-to-mesa-sources-for-xserver : full path to Mesa sources
  -n : do not quit after error; just print error message
  -s sudo-command : sudo command to use
  -r module/component : resume build at the specified component

Two of the more commonly used options are -D and -n. We recommend specifying the -D option to run make distcheck on each module component. By default the build.sh script will exit when it encounters a build failure, but with the -n command-line option, you can have the script print an error message and continue on with the next build step instead of immediately exiting.

If you want to include Mesa/OpenGL support in the X server, you will need unpack the Mesa source code and pass the directory of the Mesa sources to the build.sh script. For example, if you unpack MesaLib-6.4.1.tar.gz into /tmp/Mesa-6.4.1, then you can build the tree as follows: 

$ cd /tmp/src
$ PATH=/tmp/modular/bin:$PATH ./util/modular/build.sh -m /tmp/Mesa-6.4.1 -n -D /tmp/modular > build.log 2>&1

This will build the entire source tree in the appropriate build order and it will run make distcheck on each module component. If any errors occur, it will print an error message instead of exiting. You can search the log (e.g., grep '*****' build.log) to see if any errors occurred during the build.

If you are building from the released tarballs and have downloaded all of the tarballs, then you can build the release with the build-from-tarballs.sh script. You can find the most recent version of the script here.

Most of the options to this script are the same as the build.sh script described above. The options to the build-from-tarballs.sh script are: 

Usage: ./build-from-tarballs.sh [options] prefix
  where options are:
  -d : run make distcheck in addition to others
  -D : run make dist in addition to others
  -c : run make clean in addition to others
  -m path-to-mesa-sources-for-xserver : full path to Mesa sources
  -n : do not quit after error; just print error message
  -s sudo-command : sudo command to use
  -bz2 : use tarballs with bzip2 compression (default)
  -gz : use tarballs with gzip compression
  -e : build from witin the 'everything' dir instead of module dirs

By default the build-from-tarballs.sh script will build the packages from the module subdirectories (e.g., app, data, doc, etc.); however, if you downloaded the everything dir instead of the individual module subdirs, then you can use the -e option to build from the everything dir. By default it will build the packages from the module subdirectories. Note that when you are building from the everything dir, you need to run the build-from-tarballs.sh script from within the everything dir, and when you are building from the module subdirs, you need to run the script from the directory containing the module subdirs.

The other important option is to specify whether you want to use the gzippped or the bzip2'd tarballs. By default it will use the bzip2 tarballs, but you can override this with the -gz option.

For example, If you have downloaded all of the gzipped tarballs in the module subdirs into /tmp/tarballs as well as the build-from-tarballs.sh script, then you can build the release as follows: 

$ cd /tmp/tarballs
$ PATH=/tmp/modular/bin:$PATH ./build-from-tarballs.sh -m /tmp/Mesa-6.4.1 -n -gz /tmp/modular > build.log 2>&1

Building all modules with jhbuild

The second build method uses jhbuild, a program by James Henstridge that can be used to automatically download module components from Git or from tarballs and then build them in the correct dependency order.

JhBuildInstructions explains how to build the modular tree from Git/CVS or from the tarballs.

Building individual module components by hand

The third build method is to do everything manually. This is especially useful when you already have a modular Xorg installed and only want to build some newer modules from source. See below for more information.

As noted above, everything depends on the Xorg m4 macros, so we will explain how to build it first.

If you are building from Git and followed the instructions above, you have a modular tree ready to use in /tmp/src, the macros module components can be built, installed and packaged as follows: 

$ cd /tmp/src/util/macros
$ ./autogen.sh --prefix=/tmp/modular
$ make
$ make install

The first step above ./autogen.sh --prefix=/tmp/modular does two things: it creates the configure script and then it runs it using the command line options given to autogen.sh. This will create all of the Makefile's used to build, install and package the files. The --prefix=/tmp/modular option is a standard configuration option to tell the autotools where to install the files. This is very similar to defining ProjectRoot in your host.def file for monolithic tree builds. By default, the prefix is /usr/local, but since we are installing everything in /tmp/modular, we must override it with the --prefix option. There are many other configure options (some of which are unique to each individual module component). You can list these configure options by running ./configure --help after running autogen.sh to create the configure script.

Normally, the make step would build everything in that module from the source files. For modules like app or lib, this step will build the application or library from the sources. However, for the macros module component, there is nothing to build, since the m4 macro files are simple text files that need to be installed. make it will say that there is nothing to be done. This is normal for components that only need to install.

The next step is to install the files we built in the previous step. make install will create the directories in /tmp/modular as needed and then install the macros.

If you are building from the tarballs, then you must first unpack the util-macros tarball using the methods listed in the previous section, and then the package can built and installed as follows: 

$ cd util-macros-1.0.0
$ ./configure --prefix=/tmp/modular
$ make
$ make install

The first step is to change to the directory that you just unpacked the util-macros tarball to. Note that the directory you change to can easily be determined from the tarball name. For example, if the tarball is named package-1.2.3.tar.gz, then the directory to change to is package-1.2.3.

The second step is very similar to the ./autogen.sh step when building from Git. In this case, it is not necessary to run autogen.sh since the configure script was already built when the package was created. The options to the configure script are the same as the ones used if building from Git. Also, as noted above, there are many options to configure, which can be listed by running ./configure --help. The remaining two steps will build and install the package as described above when building from Git.

Before building subsequent modules, two environment variables need to be set: 

$ export PKG_CONFIG_PATH=/tmp/modular/lib/pkgconfig
$ export ACLOCAL="aclocal -I /tmp/modular/share/aclocal"

Subsequent module components can now be built by running: 

$ cd theComponentDirectory
$ ./autogen.sh --prefix=/tmp/modular    ### or if using tarballs: ./configure --prefix=/tmp/module
$ make
$ make install

If you're missing any dependencies you'll get an error message during the configuration stage.

You'll generally want to start with the 'low-level' proto and lib modules before building the xserver. See the build.sh script for a detailed build ordering for the components.

For larger components such as libX11 and the X server you may need additional autogen/configure options. After running autogen.sh you may want to run ./configure --help to see if there's other important options. You might also look at the build.sh script for other tips.

Building only specific modules

If you already have a modular Xorg installed you don't need to rebuild everything from source. If, for example, you only want to work with a specific driver module there is no need to rebuild all the libraries, fonts, documentation etc.

Assuming that a modular Xorg comes with your distribution, make sure that you have the development packages installed. They provide SDK headers and pkgconfig files.

Driver module ABIs may have changed since the Xorg version that you have installed. For example, the video driver ABI changed between Xorg 7.0 and 7.1. In that case, in order to build and test a new video driver module, you would have to build and install a new version of the Xserver first. That, in turn, probably requires updated versions of a few proto-packages. Running autogen.sh or configure will tell you which versions of which modules you are missing.

Compiling after updates

If you update your source code with git pull to get the latest changes you can often recompile the module by just running make (and make install). If the changes are more extensive (files added/removed) you'll need to re-run autogen.sh or configure first.

Making tarballs

As part of the development process, it's important to create tarballs to be sure things work properly.

Creating the packages for the various components is very easy with the autotools — simply run make dist or make distcheck. We always recommend using distcheck since it will both create the package as well as test it to make sure that it works properly. It can also run a testsuite, if one is available for it. Since the configure.ac file has been set up to build both gzip and bzip2 tarballs, the packages created are util-macros-1.0.0.tar.gz and util-macros-1.0.0.tar.bz2 in the example above.

Note that if you are only interested in using the component and not actually developing it, then you can skip this step, but for all developers, this step is required.

Standard configuration options

The standard configure script options (passed to ./configure or ./autogen.sh) are: 

Configuration:
  -h, --help              display this help and exit
      --help=short        display options specific to this package
      --help=recursive    display the short help of all the included packages
  -V, --version           display version information and exit
  -q, --quiet, --silent   do not print `checking...' messages
      --cache-file=FILE   cache test results in FILE [disabled]
  -C, --config-cache      alias for `--cache-file=config.cache'
  -n, --no-create         do not create output files
      --srcdir=DIR        find the sources in DIR [configure dir or `..']
Installation directories:
  --prefix=PREFIX         install architecture-independent files in PREFIX
                          [/usr/local]
  --exec-prefix=EPREFIX   install architecture-dependent files in EPREFIX
                          [PREFIX]
By default, `make install' will install all the files in
`/usr/local/bin', `/usr/local/lib' etc.  You can specify
an installation prefix other than `/usr/local' using `--prefix',
for instance `--prefix=$HOME'.
For better control, use the options below.
Fine tuning of the installation directories:
  --bindir=DIR           user executables [EPREFIX/bin]
  --sbindir=DIR          system admin executables [EPREFIX/sbin]
  --libexecdir=DIR       program executables [EPREFIX/libexec]
  --datadir=DIR          read-only architecture-independent data [PREFIX/share]
  --sysconfdir=DIR       read-only single-machine data [PREFIX/etc]
  --sharedstatedir=DIR   modifiable architecture-independent data [PREFIX/com]
  --localstatedir=DIR    modifiable single-machine data [PREFIX/var]
  --libdir=DIR           object code libraries [EPREFIX/lib]
  --includedir=DIR       C header files [PREFIX/include]
  --oldincludedir=DIR    C header files for non-gcc [/usr/include]
  --infodir=DIR          info documentation [PREFIX/info]
  --mandir=DIR           man documentation [PREFIX/man]
Program names:
  --program-prefix=PREFIX            prepend PREFIX to installed program names
  --program-suffix=SUFFIX            append SUFFIX to installed program names
  --program-transform-name=PROGRAM   run sed PROGRAM on installed program names

More information about the configure script can be found from the autoconf info pages.

Using the modular tree

This section describes how to use the newly compiled X libraries, programs and X server.

X libraries

The X libraries which are built will be in /tmp/modular/lib/.

To use them you have to configure the linker to find them. On GNU/Linux systems you do: 

export LD_LIBRARY_PATH=/tmp/modular/lib

Alternately, you might edit the /etc/ld.so.conf file and run ldconfig. There may be other ways to configure the linker on other systems.

Once this is done X applications should use the newly compiled libraries.

X applications

The X client programs which are built will be found in /tmp/modular/bin/. Generally, these programs can be run directly in the normal way.

Running the X server

The X server needs to be SUID root. This should happen automatically if you tell the build script to use sudo, but if make install runs as a normal user then this won't happen automatically. To do this: 

$ su
Password: <enter your root password here>
# chown root /tmp/modular/bin/Xorg
# chmod 4711 /tmp/modular/bin/Xorg
# exit

To run the new X server you can use the startx program:

XXX Verify this is correct 

startx -- /tmp/modular/bin/Xorg

Crosscompiling

There is a separate page dedicated to issues around CrossCompilingXorg.

New Modules

When adding modules to the tree, you may want to read the NewModuleGuidelines.