OOF2: The Manual
This section describes how to create an external OOF2 extension.
There's no fundamental reason that external OOF2 extensions must be built using the procedure described here. Other techniques certainly exist, and users who are proficient with swig and distutils, or can do without them, should feel free to do it their own way. The methods described here should be easy for most users and should suffice in almost all cases, though.
The top level directory, labelled
Extension in Figure 7.2 can be located anywhere.
OOF2 external extension code should be organized at the top
level into Python packages. Each
subdirectory below the top-level
Extension directory will contain the
files for one package, and this directory should have the same
name as the package. The distutils library installs each of
these packages and an
__init__.py file. The directory named
package in the figure will become a
package, with modules
(corresponding to Python files in
Extension/package) which can be loaded
into Python with
import package.module. There can
be more than one package defined in an extension directory,
and packages can contain subdirectories (subpackages). There
must be an
__init__.py file in each
build are created automatically by
setup.py when the extension is
build is used by
distutils to store intermediate
results, and its name should not be changed.
wrapper stores output from swig. Its
name is set in
All of the directories under the
Directory on the right side of Figure 7.2 are created automatically
distutils. The installation
directory itself is determined at build time, when
setup.py is run.
setup.py is a Python script that uses the
to build an OOF2 extension. The distutils library is
included with the Python distribution. (A working example
setup.py is included in the OOF2
directory.) The first lines (other than comments) of
setup.py must be
import distutils.core import oof2config import oof2extutils
distutils.core is the highest level
module from the Python distutils library. The
oofextutils modules are provided as part
of a correct OOF2 installation.
oof2config contains information on how
OOF2 itself was built, and sets
so that some OOF2 utility functions are
oof2extutils contains Python
functions and classes that aid the construction of OOF2
The rest of
setup.py is involved with
setting up the arguments for the function
distutils.core.setup, which does most of
the work of building and installing an extension. distutils
needs to be told how to construct the shared libraries and
wrappers in Figure 7.1, and which
Python files it should include.
distutils.core.setup uses a list of
objects to determine how to build the shared libraries.
setup.py must create these objects by
instance must be created for each shared library that will be
has only two required arguments, the name of the library and a
list of source files. It's invoked, for example, like this:
shlib = oof2extutils.SharedLibrary(name="mylib", sources=['mysrc/a.C', 'mysrc/b.C'])
name is the name of the
library (without a
lib prefix or
.so or other suffix).
sources is a list of C or C++ files to
compile. The file names should be specified relative to the
current directory (the directory containing
setup.py), or given as absolute path
names (beginning with “/”).
See the oof2extutils.SharedLibrary reference page for a list of additional optional arguments.
The next step is to run swig on the
swig input files and to build Python extension modules
from the results. OOF2 supplies a function,
that invokes swig with the appropriate arguments. The C++
files that swig produces need to be compiled into a wrapper
library, and distutils needs to be told that swig's Python
output files should be included in the extension package.
oof2extutils contains a utility function,
and returns both an
(for the wrapper library) and the name of the python package:
ext_obj, pkg = oof2extutils.get_swig_ext(srcdir = 'mysrc', srcfile = 'myext.swg', destdir = 'swigout', libraries = ['mylib'])
srcdir is the directory
containing the swig source files. In this example, the swig
source is in the same directory as the C++ source for the
srcfile is the name of the swig file, and
destdir is the destination directory, where
the C++ and Python output files will be written. If
srcfile contains a subdirectory name, the
same subdirectory will be created within the destination
libraries is a list of
libraries with which the extension module should be linked.
Generally, this should be the
name of the
SharedLibrary object, as in the example
above. The return values are in the indicated order, with
ext_obj being the
oof2extutils.Extension object and
pkg being the name of the Python package.
The Python extension
module created by the above call to
can be imported into any Python program as
swigout.myext, combining the names of the
destination directory (
source file (
Additional optional arguments to
are documented on its reference page, oof2extutils.get_swig_ext. If for some reason
doesn't do what you need, you'll need to read the reference page for
run_swig and for the
The last thing that
setup.py needs to
do is to call
distutils.core.setup(name = "oof2ExtensionName", version = "0.0.0", author = "John Q. Oof", author_email = "email@example.com", url = "http://www.address.net/~jqoof", ext_modules = [ext_obj], packages = [mysrc, pkg], shlibs = [shlib])
The first five arguments are self
ext_modules is a list of the
modules created by
packages is a list containing both the
Python packages returned by
and any hand-written Python packages
(in Figure 7.1, the former correspond
to the boxes labelled “Python Wrapper Code” and
the latter to the box labelled “Extension's Python
After creating the source files and writing
setup.py, building an OOF2 extension
Make sure that OOF2 itself is installed.
Build the extension by typing
% python setup.py build
in a Unix shell.
% python setup.py install
If you aren't the superuser this command will fail because you won't have permission to write to the default installation directories. There are two solutions to this problem:
Become a superuser or ask one to help you.
Install the files into a directory that you own, via the
--prefixcommand line option:
% python setup.py install --prefix=/home/oofuser
The example given above will put shared libraries in
/home/oofuser/lib and Python modules in
/home/oofuser/lib/python2.4/site-packages(with the directory name
If this method is used, it may be necessary to set environment variables to get OOF2 to run correctly. See Section 220.127.116.11.
The process described in Section 7.2.3 creates modules and libraries for OOF2, but it doesn't tell OOF2 anything about them, so the modules won't be used. Modules must be imported before they're accessible to Python. Generally, an OOF2 extension is designed so that only one of its modules needs to be imported explicitly — this main module then imports the rest of the modules.
There are three ways of importing an extension into OOF2.
Assume that the extension is called
and its main module is contained in the file
start.py. The three ways to import it
An import command can be typed explicitly in Console Window, like this:
This has the advantage of not requiring any premeditation, but it's awkward.
The import command, above, can be placed in the
.oof2rcfile in the users home directory. The extension will then be loaded automatically for all future OOF2 sessions.
The extension can be imported when OOF2 is started, by using the
--importcommand line option:
% oof2 --import ABC.start
If an OOF2 extension has been installed in a non-standard location, the Python import command may not be able to locate it, and the linker may not be able to find the shared libraries. Both of these problems will occur at run-time, and are fixed by setting shell environment variables. (The method of setting environment variables differs depending on which Unix shell you're using, and won't be discussed here.)
If you get a message like
ImportError: No module named ABC
when you try to load the extension ABC, then you need to set
PYTHONPATH variable. It should be
set to the name of the directory where the OOF2 Python
modules are installed. If you installed OOF2 with the
python setup.py install
PYTHONPATH must be set to
(but replace 2.4 by the correct Python version number). If
PYTHONPATH needs to contain more than one
directory (for example, if it's set to something else for
another extension), then just concatenate the directory
names, separated by colons.
If you get a message containing something like
ImportError: Failure linking new module: [...]
ImportError: [...]: cannot open shared object file: [...]
then you need to set
DYLD_LIBRARY_PATH on Mac OS X) to the
directory containing your extension's shared libraries. For
the example above, with
--prefix set to
LD_LIBRARY_PATH should be set to
A package is a directory containing
Python source files and an
__init__.py file. Code in a file
in the directory can be loaded into Python with
import directoryname.filename. Code within
__init__.py will run the first time
any module from the directory is imported.
__init__.py can itself import the
other files in the directory, in which case the whole
package can be loaded with
On Linux and many other Unix systems, user
accounts are located under the
/home directory, but this
practice is not universal, and on heterogeneous
clusters, home directories may be soft-linked in
complicated ways. In case of doubt, the value
variable may be used to portably refer to the
user's home directory, like this:
% python setup.py install --prefix=$HOME