3.1 The User View

N.B. Much of the discussion in this section undoubtedly belongs elsewhere, (xemacs)Packages.

From the user’s point of view, an XEmacs binary package is simply a standard tarball (usually gzipped) containing Lisp sources, compiled Lisp, documentation, and possibly data files or supporting executables. The tarball is unpacked using standard tools such as GNU tar and gzip. The package system does impose certain requirements for automatic configuration to work.

Here the main consideration is that the tarball “expects” to be unpacked from the top of a package hierarchy. A package hierarchy is basically an image of a classic Emacs “run-in-place” tree, with lisp, etc, info, man, lib-src, and pkginfo subdirectories of the top. The pkginfo subdirectory is for use by the XEmacs Packaging System administration tools, and currently contains a MANIFEST.package-name file for each package to ensure that no cruft remains when a package is removed or updated. The lisp, etc, and lib-src subdirectories are further subdivided, with a subdirectory for each package. The info directory obeys the usual conventions. I.e., the info directory is flat with a(n) (optional) dir file and one (set of) info file(s) per package. The man subdirectory typically contains documentation sources, separated by package. (It does not contain man(1) pages, as Emacs provides very few of them.)

There are several standard package hierarchies, and administrators can configure others at build time, while users can configure others at run time. The standard system hierarchies are all subdirectories of ${PREFIX}/share/sxemacs/. These are the site-packages, xemacs-packages, mule-packages, infodock-packages, and sxemacs-packages hierarchies. Each has the structure described above, but the purposes differ. The xemacs-packages is the normal place for installing “official” packages and many third-party libraries. Unfortunately, it is not yet quite possible to read libraries containing international characters with a non-Mule SXEmacs, so such libraries are sequestered in the mule-packages hierarchy. Some packages are compatible only with the Infodock development environment, and they will be installed in the infodock-packages hierarchy. The site-packages hierarchy is for packages not distributed by XEmacs.org, typically locally developed.

Users who do not have sufficient privilege to install packages in the system hierarchies may install package hierarchies under $XDG_DATA_HOME/sxemacs. At present only the site-packages, xemacs-packages, mule-packages, and sxemacs-packages hierarchies are supported, but it might make sense to extend this to support infodock-packages hierarchies in the future.

The package hierarchies are not searched directly for libraries to be loaded; this would be very costly. Instead, the hierarchies are ordered according to certain rules, and searched for package lisp directories at invocation. These directories are added to the general load-path. As usual, it is load-path that is searched at run-time. This approach is somewhat costly at initialization, but results in a very “clean” load-path.

The order of search can be changed at build time by specifying the ‘--with-package-path’ option to configure, or at run-time by specifying the EMACSPACKAGEPATH environment variable See (sxemacs)Packages. Also, the location for early-packages may be set via the -user-pkgs-directory command line argument (see (sxemacs)Command Switches).

The default order of search is hierarchically determined. There are 3 “categories” of lisp hierarchies in SXEmacs, well, actually there are 4, but we’ll get to that in a moment. In order, they are: early-packages, late-packages, and core.

The early-packages are the user’s local packages, stored under their $HOME. Typically, these packages will be in $XDG_DATA_HOME/sxemacs. The late-packages are the “system-wide” packages, typically installed in $PREFIX/share/sxemacs, and core is the lisp that is shipped with SXEmacs and installed to $PREFIX/share/sxemacs-$version.

The early-packages and late-packages are further split into package types. These types are (in search order): site-packages, sxemacs-packages, infodock-packages, mule-packages, and xemacs-packages.

The core lisp is also sub-divided into mule, ffi, and “normal”, and searched in that order.

Oh, that 4th category I spoke of earlier, those are the last-packages. By default, SXEmacs does not have a pre-configured last package hierarchy. Last hierarchies are primarily for using package hierarchies of outdated versions of XEmacs as a fallback option. For example, it is possible to run SXEmacs with the XEmacs 20.4 package hierarchy as a last hierarchy.

In each hierarchy there should be a lisp subdirectory, containing directories named for the packages. Each package’s Lisp libraries thus are contained in a directory of the form category/type/lisp/package/. The 3 directories that make up the core lisp are, in search order…

$PREFIX/share/sxemacs-version/lisp/mule
$PREFIX/share/sxemacs-version/lisp/ffi
$PREFIX/share/sxemacs-version/lisp

With such a complex search algorithm, the possibility of libraries being shadowed by another library with the same name is quite real. There are two considerations here. First, every SXEmacs package contains certain libraries with constant names. These are

_pkg.el

Lisp code to inform the package administration system about the package

auto-autoloads.el

Lisp code to set up autoloaded functions and variables that may be needed at load time

custom-load.el

definitions of configuration variables for use with the Customize system.

They are special-cased, because the way they are used prevents shadowing from being an issue.

Second, it is possible that multiple copies of some library, or different libraries with the same name, are installed in various places in the hierarchies. To detect such shadows, use list-load-path-shadows.

Finally, note that most basic Emacs functionality, including most of the Lisp API, is implemented in Lisp libraries. Because they use internal reserved APIs that are subject to change according the needs of the developers, these libraries are distributed with the XEmacs binary, and are called core Lisp libraries. Most core Lisp libraries are “preloaded” into the Emacs binary and in normal usage are never explicitly loaded. However, they can be explicitly loaded, and if so they are searched on load-path. Furthermore, functions such as locate-library will also search on the load-path. The searching takes place under somewhat different rules from those used for packaged Lisp. It is probably easiest to think of the package hierarchy searching algorithm as receiving a load-path initialized to the core Lisp directories.