- 1 Installing, Configuring, and Building Software in Linux
- 2 Advanced Topics
Installing, Configuring, and Building Software in Linux
If you're too impatient (read: lazy) to read through all this, you can also watch the Scientific Computing Summer Program presentation here: Software Presentation.
Source vs. Binary
Most software available for Linux and Unix is open-source, and is provided for free. This software is often provided in two forms: a source package, and a binary package.
Source packages consist of the source code, makefiles, scripts, and other things required to compile the source code. Because you're compiling the source code, on YOUR machine, with YOUR compiler, it doesn't matter what platform you're running. When you compile the code, you will have an executable file that will run on your computer.
Binaries are pre-compiled executables. A Linux binary is different from a Mac OS X binary, which is different from a Windows binary. Often, there may be different binaries even for a single platform (for example, a Mac OS X 64-bit binary is different from a Mac 32-bit binary, and the 64-bit binary will not run on a 32-bit machine). The binary is pre-compiled, which means the developer has already compiled the source code for that particular platform.
Downloading and running a binary package for your platform is simple to do - you simply download the binary and run it from the command line. To run a Linux or Unix binary, you would type:
(Note that the binary will be located in the "bin" folder 99% of the time). To run a Mac binary, you would move the .app file into your Applications folder (or wheverever you would like to run the binary from). And in Windows, you would double-click the .exe file.
Downloading The Source Package
Source files are available on a software project's website. You'll likely see several different versions, so you'll want to get the latest version. Versions are released when problems with the software are fixed, so the latest version will probably have more features and fewer problems. However, if you see a version that is labeled "alpha" or "beta", that means it has not been completely tested yet, and so it's not stable and may crash or behave strangely.
The packages are usually in a compressed format, like .tar.gz or .tar.bz. These can be unzipped using:
$ tar xvzf program-version.tar.gz
This creates a folder called 'program-version'. This folder contains all the source code that will be compiled in order to run the program.
When you move into the folder that was created and list the files in the directory, you'll probably see several files with names like README or INSTALL. It is a good idea to read these, as the instructions given here will not work for building every program. The README or INSTALL files will give you some basic instructions on how to build and configure the program.
Configuring the Program for Installation
The first step in building your program is to configure it. This is a way of "customizing" the program to fit your needs. Often, a program's default installation settings are not enough to fit your needs. For example, if you have a 64-bit processor, the program will not install a 64-bit version by default (because most people don't have a 64-bit processor). You can specify that you have a 64-bit processor when you configure the program. You can also control whether features are enabled or disabled, where the program installs itself, and where it looks for various programs it might depend on.
To configure your program, you'll execute a file called "configure", which is essentially a script that checks to make sure you have everything you need. You run the script with various flags (meaning you add --flag to the end, where "flag" is replaced by some special keyworkd that tells the program how to behave). The configure script will check to ensure you have all the prerequisite software, all the libraries you need, that your hardware matches the configure options you gave it (that is, it makes sure that you don't try to install a 64-bit version on a 32-bit machine), and lots of other things.
Since the options available for "configure" vary greatly from program to program, you need a way of seeing what options you have available to you. To do this, you will run configure with a special flag - the help flag.
$ ./configure --help
The ./ at the beginning tells Unix to run the command named "configure" that is located in the current folder, instead of looking for a program already installed on your computer named configure.
This will print out a lot of output that will give you a long list of different flags and what each will do. For example, here is the output from running this command for the program Subversion:
Most of these options are very specialized and the average user will not need them. In fact, if you don't care about customizing the installation, you could run ./configure without options. When you don't specify a value for a flag, the configure script will use a default value.
Sometimes, however, you will need to configure your program with specific options, so the values for various flags will need to be set. Sometimes it is only one or two flags, but sometimes you may need to change as many as 10 flags, sometimes more, and set special environmental variables or compiler flags to make your compiler work like normal. It can be a hassle, if something goes wrong in the configure script, to re-type the whole line, or to keep editing the configure line on the command line. You may also lose the configure line you used, which can be frustrating if something happens to the program and you need to re-install it, or if you need to install it on another system and you forgot that one special flag you need to make everything work.
In this case, you can make a configure script. You can call it anything you'd like, but as an example I will use runconfigure.sh. The script will be a very simple script that, when run, will simply run your configure line. You can make the file using your favorite text editor, and it will look like this:
#!/bin/sh ./configure \ --flag-1=THIS \ --flag-2=THAT \ --flag-3=THE_OTHER \ VARIABLE=value
The first line tells the system that this is a script, and the script should be run using the program /bin/sh. The next lines run configure with the flags flag-1 set to the value THIS, flag-2 set to THAT, and flag-3 set to THE_OTHER. The backslash \ at the end of each line makes sure that everything is put together as one single command (if there were no backslash, it would run the first line as a command, then run the second line as a separate command, and so on).
One last step remains before you can run your custom configure script - you have to make it executable. You can do this using chmod, which changes the permissions of a file:
$ chmod u+x runconfigure.sh
Your custom configure script can be run just like the normal configure script:
This will give the same output as running ./configure except that it will run configure with the options you want.
Making Your Program
Once you've configured your program, you have told the program what you want it to install and where. The next step is to compile all of the source code into the executable files. This involves using the GNU make program. The make program uses a file called a Makefile, which is essentially a special script containing detailed instructions about how to compile each file that's part of a project. Each file needs to link to the files it refers to, and include files it uses, and the Makefile ensures each file is linked or included in the instructions given to the compiler.
Running configure sets certain variables used by GNU make. Once you have run configure, you can run GNU make by typing:
Or, in some cases,
$ make all
You can always see what is available to install by using help:
$ make help
This will display a list of different packages that can be made (also called "make targets"). Running "make" makes the default targets, and running "make all" makes all targets.
make all will compile the source code in-place - so one additional step is necessary to install and use the program.
Installing Your Program
Typing make will only compile each source code file in-place - meaning, if you specify that you want to the program to install to
/path/to/installation, and you are running configure and make in
/path/to/source/code, then when you type make, and when once make finishes, the finished product will reside in
/path/to/source/code, and not in
/path/to/installation. In these cases, you must issue a second command:
$ make install
This will create a copy of the built executable in
/path/to/installation, which you probably specified using the
--prefix=/path/to/installation option while configuring. By default, programs are (usually) installed to
Specifying Which Compiler
You can specify compilers when running configure by doing something like this:
#!/bin/sh ./configure \ --option=something \ \ CC="/usr/bin/gcc" \ CXX="/usr/bin/g++" \ F77="/usr/bin/gfortran"
This specifies that the C compiler (specified with the CC variable) should be
/usr/bin/gcc, the C++ compiler (specified with the CXX variable) should be
/usr/bin/g++, and the Fortran compiler (specified with the F77 variable) should be
Specifying Compiler Flags
If you want to specify flags for the compilers, you can use the variables
CFLAGS for C compiler flags,
CXXFLAGS for C++ compiler flags, and
FFLAGS for Fortran compiler flags.
For example, if I wanted to specify that the compiler should use verbose warnings when I am compiling, I would use the
-Wall flag for the compilers. To do this when configuring:
#!/bin/sh ./configure \ --option=something \ \ CC="/usr/bin/gcc" \ CXX="/usr/bin/g++" \ F77="/usr/bin/gfortran" \ CFLAGS="-Wall" \ CXXFLAGS="-Wall"
If I wanted to specify that the compiler should build a 64-bit binary, I would specify this using the
#!/bin/sh ./configure \ --option=something \ \ CC="/usr/bin/gcc" \ CXX="/usr/bin/g++" \ F77="/usr/bin/gfortran" \ CFLAGS="-Wall -arch x86_64" \ CXXFLAGS="-Wall -arch x86_64"
There are potential conflicts that can come about using the compiler and flags variables. For example, if I want the compiler to build binaries that work for multiple architectures (say, 32-bit and 64-bit), I can specify this with two
CFLAGS="-arch x86_64 -arch i386"
but often the configure script will strip the second
-arch flag, and this will cause errors like "File i386 does not exist".
To get around this, I can add flags to the compiler command. The configure script will not strip flags from the compiler command, it will leave it alone. So I could say:
CC="/usr/bin/gcc -arch x86_64 -arch i386"
and this would work fine.
However, this will create problems when it comes time to run the preprocessor. The preprocessor is the part that deals with all of the
#include <Header1.h> #include <Header2.h> #ifdef SOME_VARIABLE #include <Variable.h> #endif
stuff. Normally, the C and C++ preprocessors are the C and C++ commands with the
-E flag added on. But if you try and run a preprocessor command like
/usr/bin/gcc -arch x86_64 -arch i386 -E
then you're gonna have problems. You can get around this by specifying the preprocessor command, as shown in the next section.
Specifying the Preprocessor
You can specify the C or C++ preprocessor using the
CXXCPP variables, like this:
#!/bin/sh ./configure \ --option=something \ \ CC="/usr/bin/gcc -arch x86_64 -arch i386" \ CXX="/usr/bin/g++ -arch x86_64 -arch i386" \ F77="/usr/bin/gfortran" \ CFLAGS="-Wall" \ CXXFLAGS="-Wall" \ \ CPP="/usr/bin/gcc -E" \ CXX="/usr/bin/g++ -E"
For an example of all of these pieces coming together when configuring and compiling a piece of software, see the FFTW page. The FFTW libraries required compilation of both 32-bit and 64-bit versions, and configure stripped off the second
-arch flag, so I had to use all of the above techniques to get it to compile correctly.
GNU/Linux/Unixthe concrete that makes the foundations of the internet.
Command Line Utilities
Flags · Template:UnixFlag · e