[Handbook Index] [To Section 2 - OpenBSD Porting Guide]

1 - The OpenBSD ports system

1.1 - How does it work?
1.2 - Fetching the ports tree
1.3 - Configuration of the ports system
1.4 - Searching the ports tree
1.5 - A first example
1.6 - Cleaning up after a build
1.7 - Uninstalling a port's package
1.8 - Using flavors and subpackages
1.9 - Problems and Contacts
1.10 - When a Port Is Lagging Behind the Mainstream Version

1.1 - How does it work?

The ports tree, a concept originally borrowed from FreeBSD, is a set of Makefiles, one for each third party application, for controlling

where and how to fetch the source of the software,
which other software it depends upon,
how to alter the sources (if necessary),
how to configure and build it,
how to test it (optional),
how to install it.

Apart from the Makefile, each port also contains at least the following:

a PLIST or packing list, which contains instructions for package creation once the application has been built,
a DESCR or description of the application,
a distfile, containing distribution file checksums and size.

All this information is kept in a directory hierarchy under /usr/ports. This hierarchy contains three special subdirectories:

distfiles/ - where the ports system stores software distribution sets after downloading.
infrastructure/ - the main directory of the ports infrastructure, containing all necessary scripts and makefiles.
packages/ - contains all binary packages built by the ports system.

The other subdirectories all form different application categories, which contain the subdirectories of the actual ports. Complex ports may be organized to an even deeper level, for example if they have a core part and a set of extensions, or a stable and a snapshot version of the application. Every port directory must contain a pkg/ subdirectory containing packing list(s) and description file(s). There may also be patches/ and files/ subdirectories, for source patches and additional files, respectively.

When a user issues make(1) in the subdirectory of a specific port, the system will recursively walk its dependency tree, check whether the required dependencies are installed, build and install any missing dependencies, and then continue the build of the desired port. All of the building happens inside the working directory that the port creates. Normally it is under ${WRKOBJDIR}, defaulting to /usr/ports/pobj, but you may override this (see Configuration of the ports system). If WRKOBJDIR has been explicitly unset, a subdirectory of the port's main directory (package name prefixed by "w-") will be used instead.

Note: Ports are never directly installed on your system! They use a fake installation directory. Everything that gets installed there, is bundled together into a package (which is stored in the packages/ subdirectory of the ports tree as mentioned earlier). Installing a port really means: creating a package, and then installing that package!

More information about the ports system may be found in these manual pages:

ports(7) - describes the different stages (make targets) of port installation, the use of flavors and subpackages and some other options.
bsd.port.mk(5) - in depth information about all the make targets, variables, the fake (installation directory) framework, etc.
port-modules(5) - format and conventions used in port modules.
packages-specs(7), library-specs(7) - specifications for binary package names and library dependencies, respectively.
pkg_create(1) - notation used in packing lists.

1.2 - Fetching the ports tree

The ports tree, like the rest of OpenBSD, is constantly changing. The current ports tree may not be used with the previous release. This is due to changes, typically with the port make process, that require code based upon the OpenBSD-current source tree.

The ports tree works as a single entity. Updating a single directory is not guaranteed to work, as package dependencies may force you to update and recompile vast portions of the ports tree. It is strongly suggested that people don't track ports-current unless they're prepared to deal with various problems. Mailing lists such as ports-changes@openbsd.org or tech@openbsd.org will probably be invaluable.

Before continuing, you must therefore read the section about NOT mixing up your OpenBSD system and ports tree. Once you have decided which flavor of the ports tree you want, you can get the ports tree from different sources. The table below gives an overview of where you can find the different flavors, and in which form. An 'x' marks availability and '-' means it is not available through that specific source.

Source Form Flavor

-release -stable snapshots -current

CD-ROM .tar.gz x - - -

FTP mirrors .tar.gz x - x -

AnonCVS cvs checkout x x - x

On the CD-ROM and FTP mirrors, look for a file named ports.tar.gz. You want to untar this file in the /usr directory, which will create /usr/ports, and all the directories under it. For example:

$ cd /tmp
$ ftp ftp://ftp.openbsd.org/pub/OpenBSD/5.4/ports.tar.gz
$ cd /usr
$ sudo tar xzf /tmp/ports.tar.gz

The snapshots available on the FTP mirrors are generated daily from the -current ports tree. You will find the snapshots in the pub/OpenBSD/snapshots/ directory. If you are using a snapshot of the ports tree, you should have installed a matching snapshot of OpenBSD. Make sure you keep your ports tree and your OpenBSD system in sync!

The ports-current tree can be retrieved via:

Anonymous CVS (see link). The command is essentially cvs get ports. You can also use this method to retrieve the tree from a local cvsync mirror.
Anonymous ftp from ftp://ftp.openbsd.org/pub/OpenBSD/snapshots/ports.tar.gz. This archive is updated nightly.
Your web browser using the CVS web interface.

1.3 - Configuration of the ports system

NOTE: This section introduces some additional global settings for building applications from ports. You can skip this section, but then you will be required to perform many of the make(1) statements in the examples as root.

Because the OpenBSD project does not have the resources to fully review the source code of all software in the ports tree, you can configure the ports system to take a few safety precautions. The ports infrastructure is able to perform all building as a regular user, and perform only those steps that require superuser privileges as root. Examples are the fake and install make targets. However, because root privileges are always required at some point, the ports system will not save you when you decide to build a malicious application.

You can set up sudo(8) and have the ports system use it for tasks requiring superuser permissions. Just add a line to /etc/mk.conf containing
```
SUDO=/usr/bin/sudo
```
You can modify the ownerships of the ports tree so that you can write there as a regular user. In this case, the regular user has been added to the wsrc group, and the underlying directories are made group writable.
```
# chgrp -R wsrc /usr/ports
# find /usr/ports -type d -exec chmod g+w {} \;
```
You can have the ports system use systrace(1) by adding the following to /etc/mk.conf
```
USE_SYSTRACE=Yes
```
This enforces the build procedure to stay inside allowed directories, and prohibits writing in illegal places, thereby considerably reducing the risk of a damaged system. Note that the use of systrace(1) adds about 20% overhead in build time.

It is possible to use a read-only ports tree by separating directories that are written to during port building:

The working directory of ports. This is controlled by the WRKOBJDIR variable, which specifies the directory which will contain the working directories.
The directory containing distribution files. This is controlled by the DISTDIR variable.
The directory containing newly built binary packages. This is controlled by the PACKAGE_REPOSITORY variable.

For example, you could add the following lines to /etc/mk.conf

WRKOBJDIR=/usr/obj/ports
DISTDIR=/usr/distfiles
PACKAGE_REPOSITORY=/usr/packages

If desired, you can also change the ownership of these directories to your local username and group, so that the ports system can create underlying working directories as a regular user.

1.4 - Searching the ports tree

Once you have the ports tree in place on your system, it becomes very easy to search for software. Just use make search key="searchkey", as shown in the following example.

$ cd /usr/ports
$ make search key=rsnapshot
Port:	rsnapshot-1.3.1p2
Path:	net/rsnapshot
Info:	remote filesystem snapshot utility
Maint:	Antoine Jacoutot <ajacoutot@openbsd.org>
Index:	net sysutils
L-deps:	
B-deps:	STEM->=1.21:textproc/groff net/rsync
R-deps:	devel/p5-Lchown net/rsync
Archs:	any

The search result gives a nice overview of each application that is found: the port name, the path to the port, a one-line description, the port's maintainer, keywords related to the port, library/build/runtime dependencies, and architectures on which the port is known to work.

One can also use the index file to go directly to a given port:

$ cd /usr/ports
$ make key=rsnapshot describe
===>> net/rsnapshot
rsnapshot-1.3.1p2|net/rsnapshot||remote filesystem snapshot utility|net/rsnapshot/pkg/DESCR|Antoine Jacoutot <ajacoutot@openbsd.org>|net sysutils||STEM->=1.21:textproc/groff net/rsync|devel/p5-Lchown net/rsync|any|y|y|y

This mechanism, however, is a very basic one, which just runs awk(1) on the ports index file. Fine-grained searching using SQL is also available using the port called sqlports. It is a SQLite database, but basically just about any database format can be created using the ports infrastructure. The sqlports port includes the script used to generate the database, which could be used as a basis to generate databases in different formats.

A sample session could look like:

$ sqlite3 /usr/local/share/sqlports
SQLite version 3.8.0.2 OpenBSD
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> SELECT FULLPKGNAME,COMMENT FROM Ports WHERE COMMENT LIKE '%statistics%';
pg_statsinfo-2.4.1p0|monitor PostgreSQL activity & statistics
erl-bear-0.1.3p0|set of statistics functions for Erlang
p5-Devel-Cover-Report-Clover-0.35|backend for Clover reporting of coverage statistics
mailgraph-1.14p1|RRDtool frontend for Postfix statistics
R-2.15.3p0|powerful math/statistics/graphics language
p5-Math-VecStat-0.08p0|provides basic statistics on numerical vectors
py-probstat-0.912p7|probability and statistics utilities for Python
darkstat-3.0.718|network statistics gatherer with graphs
pfstat-2.3p5|packet filter statistics visualization
rtg-0.7.4p8|SNMP statistics monitoring system
slurm-0.3.3p1|network traffic monitor and statistics
tcpstat-1.5p0|report network interface statistics
wmwave-0.4p3|Window Maker dockapp to display wavelan statistics
libstatgrab-0.17|system statistics gathering library
p5-Unix-Statgrab-0.04p4|interface to libstatgrab system statistics library
diffstat-1.55|accumulates and displays statistics from a diff file
drupal7-google_analytics-1.2p0|add the Google Analytics web statistics to your site
kdf-4.11.5|KDE storage device statistics
sqlite>

The above is still a very basic search. With SQL, just about anything can be searched for, including dependencies, configure flags, shared libraries, etc.

1.5 - A first example

Let's illustrate the main concepts of the ports system with a simple example: rsnapshot. To build this application we will go into its directory and execute make install. The results should look something like the following:

$ cd /usr/ports/net/rsnapshot
$ make install
===>  Checking files for rsnapshot-1.3.1p2
`/usr/ports/distfiles/rsnapshot-1.3.1.tar.gz' is up to date.
>> (SHA256) rsnapshot-1.3.1.tar.gz: OK
===> rsnapshot-1.3.1p2 depends on: rsync-* - not found
===>  Verifying install for rsync-* in net/rsync
===>  Checking files for rsync-3.1.0
`/usr/ports/distfiles/rsync-3.1.0.tar.gz' is up to date.
>> (SHA256) rsync-3.1.0.tar.gz: OK
===>  Verifying specs: c z c z
===>  found c.73.1 z.5.0
===>  Extracting for rsync-3.1.0
===>  Patching for rsync-3.1.0
===>  Configuring for rsync-3.1.0
  [...snip...]
===>  Building for rsync-3.1.0
  [...snip...]
===>  Faking installation for rsync-3.1.0
  [...snip...]
===>  Building package for rsync-3.1.0
Create /usr/ports/packages/amd64/all/rsync-3.1.0.tgz
Link to /usr/ports/packages/amd64/ftp/rsync-3.1.0.tgz
Link to /usr/ports/packages/amd64/cdrom/rsync-3.1.0.tgz
===>  Cleaning for rsync-3.1.0
===>  Verifying specs: c z
===>  found c.73.1 z.5.0
===>  Installing rsync-3.1.0 from /usr/ports/packages/amd64/all/
rsync-3.1.0: ok
The following new rcscripts were installed: /etc/rc.d/rsyncd
See rc.d(8) for details.
===> Returning to build of rsnapshot-1.3.1p2
===> rsnapshot-1.3.1p2 depends on: rsync-* -> rsync-3.1.0
===> rsnapshot-1.3.1p2 depends on: groff->=1.21 -> groff-1.22.2p4
===>  Extracting for rsnapshot-1.3.1p2
===>  Patching for rsnapshot-1.3.1p2
===>  Configuring for rsnapshot-1.3.1p2
  [...snip...]
===>  Building for rsnapshot-1.3.1p2
  [...snip...]
===>  Faking installation for rsnapshot-1.3.1p2
  [...snip...]
===>  Building package for rsnapshot-1.3.1p2
Create /usr/ports/packages/amd64/no-arch/rsnapshot-1.3.1p2.tgz
Link to /usr/ports/packages/amd64/all/rsnapshot-1.3.1p2.tgz
Link to /usr/ports/packages/amd64/ftp/rsnapshot-1.3.1p2.tgz
Link to /usr/ports/packages/amd64/cdrom/rsnapshot-1.3.1p2.tgz
===> rsnapshot-1.3.1p2 depends on: p5-Lchown-* -> p5-Lchown-1.01p2
===>  Installing rsnapshot-1.3.1p2 from /usr/ports/packages/amd64/all/
rsnapshot-1.3.1p2: ok

Pretty easy, huh? Especially considering all the things the ports system takes care of automatically:

Fetch the sources from an ftp site (saved under /usr/ports/distfiles/),
Check the source archive integrity,
Extract the sources,
Apply OpenBSD specific patches,
Configure and build (compile) the program,
Install the files into a fake directory
Create a binary package under /usr/ports/packages/,
Install that package (usually under /usr/local/).

And this is done recursively for all dependencies of the port. Just notice the "===> Verifying install for ..." and "===> Returning to build of ..." lines in the above output, indicating the walk through the dependency tree.

If a previous version of the application you want to install, was already installed on your system, you can use make update instead of make install. This will call pkg_add(1) with the -r flag.

Note: Large applications will require a lot of system resources to build. Good examples are compilers like GCC 4.0 or the Java 2 Software Development Kit. If you get "out of memory" type of errors when building such a port, this usually has one of two causes:

Your resource limits are too restrictive. Adjust them with ksh's ulimit or csh's limit command. If that doesn't help, just become root before starting the build, or use sudo(8) with the -c flag to run the build with the resources limited by the specified login class (refer to login.conf(5) for details about login classes).
You simply don't have enough RAM in your machine.

Tips:

Users with low connectivity may refer to mirroring-ports(7) for an efficient way to grab all distfiles at once.
In OpenBSD-current, you can use the script /usr/ports/infrastructure/fetch/clean-old to track old distfiles.

1.6 - Cleaning up after a build

You probably want to clean the port's default working directory after you have built the package and installed it.

$ make clean
===>  Cleaning for rsnapshot-1.2.9

In addition, you can also clean the working directories of all dependencies of the port with this make target:

$ make clean=depends
===>  Cleaning for rsync-2.6.9
===>  Cleaning for rsnapshot-1.2.9

If you wish to remove the source distribution set(s) of the port, you would use

$ make clean=dist
===>  Cleaning for rsnapshot-1.2.9
===>  Dist cleaning for rsnapshot-1.2.9

In case you have been compiling multiple flavors of the same port, you can clear the working directories of all these flavors at once using

$ make clean=flavors

You can also clean things up as they get built, by setting a special variable. Work directories will automatically be cleaned after packages have been created:

$ make package BULK=Yes

1.7 - Uninstalling a port's package

It is very easy to uninstall a port:

$ make uninstall
===> Deinstalling for rsnapshot-1.2.9
rsnapshot-1.2.9: complete
Clean shared items: complete

This will call pkg_delete(1) to have the corresponding package removed from your system. If desired, you can also uninstall and re-install a port's package by using

$ make reinstall
===>  Cleaning for rsnapshot-1.2.9
/usr/sbin/pkg_delete rsnapshot-1.2.9
rsnapshot-1.2.9: complete
Clean shared items: complete
===>  Installing rsnapshot-1.2.9 from /usr/ports/packages/i386/all/rsnapshot-1.2.9.tgz
rsnapshot-1.2.9: complete

If you would like to get rid of the packages you just built, you can do so as follows:

$ make clean=packages
===>  Cleaning for rsnapshot-1.2.9
rm -f /usr/ports/packages/i386/all/rsnapshot-1.2.9.tgz

1.8 - Using flavors and subpackages

Please do read the ports(7) manual page, which gives a good overview of this topic. There are two mechanisms to control the packaging of software according to different needs.

The first mechanism is called flavors. A flavor usually indicates a certain set of compilation options. For instance, some applications have a "no_x11" flavor which can be used on systems without X. Some shells have a "static" flavor, which will build a statically linked version. There are ports which have different flavors for building them with different graphical toolkits. Other examples include: support for different database formats, different networking options (SSL, IPv6, ...), different paper sizes, etc.

Summary: Most likely you will use flavors when a package has not been made available for the flavor you are looking for. In this case you will specify the desired flavor and build the port yourself.

Most port flavors have their own working directory during building and every flavor will be packaged into a correspondingly named package to avoid any confusion. To see the different flavors of a certain port, you would change to its subdirectory and issue

$ make show=FLAVORS

You should also look at the port's DESCR files, as they're supposed to explain the available flavors.

The second mechanism is called subpackages. A porter may decide to create subpackages for different pieces of the same application, if they can be logically separated. You will often see subpackages for the client part and the server part of a program. Sometimes extensive documentation is bundled in a separate subpackage because it takes up quite some disk space. Extra functionality that pulls in heavy dependencies will often be packaged separately. The porter will also decide which subpackage is the main subpackage, to be installed as a default. Other examples are: extensive test suites which come with the software, separate modules with support for different things, etc.

Summary: Some ports are split into several packages. make install will only install the main subpackage.

To list the different packages built by a port, use

$ make show=PACKAGES

make install will only install the main subpackage. To install them all, use

$ make install-all

To list the different subpackages available for a port, use

$ make show=MULTI_PACKAGES

It is possible to select which subpackage(s) to install from within the ports tree. After some tests, this procedure will just call pkg_add(1) to install the desired subpackage(s).

$ env SUBPACKAGE="-server" make install

Note: The subpackages mechanism only handles packages, it does not modify any configuration options before building the port. For that purpose you must use flavors.

1.9 - Problems and Contacts

If you have trouble with an existing port, please send e-mail to the port maintainer. To see who is the maintainer of the port, type, for example:

$ cd /usr/ports/archivers/unzip
$ make show=MAINTAINER

Alternatively, if there is no maintainer, or you can't reach him/her, send mail to the OpenBSD ports mailing list, ports@openbsd.org. Please don't use the misc@openbsd.org mailing list for questions about ports. Corrections are always welcome, but in any case do please provide:

The output of uname -a,
Your OpenBSD version, including any patches you may have applied,
A complete description of the problem.

For ports that don't build correctly, a complete build transcript is almost always required. You can use the portslogger(1) script, found in /usr/ports/infrastructure/bin, for this. A sample run of portslogger(1) might be:

$ cd /usr/ports/archivers/unzip
$ su
# mkdir -p ~/portslogs
# make clean install 2>&1 | /usr/ports/infrastructure/bin/portslogger \
        ~/portslogs

After this, you should have a logfile of the build in your ~/portslogs directory that you can send to the port maintainer. Also, make sure you are not using any special options in your build, for example in /etc/mk.conf.

Alternatively, you can

Use script(1) to create a complete build transcript. Don't remove the configure information.
Attach the output of pkg_info(1) if it seems even remotely relevant.
gcc(1) internal compiler errors do ask you to report the bug to the gcc mailing-list. It does save time if you follow their direction, and provide at least the various files produced by gcc -save-temps.

1.10 - When a Port Is Lagging Behind the Mainstream Version

The ports collection is a volunteer project. Sometimes the project simply doesn't have the developer resources to keep everything up-to-date. Developers pretty much pick up what they consider interesting and can test in their environment. Your donations count for what platforms the ports can be tested on.

Some individual ports may lag behind the mainstream versions because of this. The ports collection may have a version back of a program from January while a new version of the program has been released by its developers in May three months ago. Often this is a conscious decision; the new version may have problems in it on OpenBSD that the maintainer is trying to solve, or that have simply made the application worse than the old version: OpenBSD may have different goals than the mainstream developers in other projects, which sometimes results in features and design or implementation choices that are undesirable from OpenBSD developers' point of view. The update may also be postponed because the new version is not considered a crucial update.

If you really need a new version of a port, you should ask the MAINTAINER of the port to update the port (see above on how to find out who the maintainer is); if you can send patches for this, all the better.