Maintaining Tds Compliance For Custom Texmf Trees

Why TDS Compliance Matters

The TeX Directory Structure (TDS) is a specification for organizing TeX-related files on a system. Following the TDS makes it easier for both humans and programs to locate files. The kpathsea library used by TeX distributions relies on certain conventions to find files. Failing to follow TDS can lead to issues with file discovery and portability across systems.

Some key reasons to maintain TDS compliance for custom TeXMF trees include:

  • Ensures files are discovered properly by kpathsea and TeX programs. Non-compliant structures can fail silently.
  • Avoids conflicts between system and local files when names/locations don't follow conventions.
  • Improves portability of custom trees across different LaTeX installations.
  • Other TDS-compliant utilities can better interact with the tree.
  • Following established practices reduces confusion for other users working with your files.

Even for personal customizations, using the standard TDS hierarchy and appropriately registering the tree can prevent head-scratching errors when sharing documents or moving files across systems.

Setting Up a Custom Tree Structure

When adding a new TeXMF tree such as TEXMFHOME for user-specific files or TEXMFLOCAL for administrator-level customizations, we must first consider the appropriate location and permissions for such trees, then establish appropriate relative subdirectory paths.

Location and Privileges

TEXMFLOCAL should be created at /usr/local/texlive/texmf-local on Unix-like systems or a parallel "texmf-local" folder within the distribution tree on Windows. It should be owned and writable by the administrator account which installed TeX. This ensures system-level changes can be made, while regular users' home texmf trees remain unmodified.

TEXMFHOME targets the individual user level, so it should be located in the user's home folder - ~/texmf on Linux/macOS, or under %APPDATA% on Windows. The regular user should have full control over all files and folders within.

Directory Structure

Within each main tree, the directory hierarchy should mirror the TDS specification's structure:

texmf/
    |- doc/
    |- fonts/
    |- metafont/ 
    |- metapost/
    |- scripts/
    |- source/ 
    |- tex/

Here the subfolds match those of the default system tree, containing relevant LaTeX packages, fonts, scripts, and documentation. Custom files should be added into the appropriate subdirectory: fonts under fonts/, packages (.sty) under tex/, executables under scripts/, etc.

Registering Trees with kpsewhich

After setting up a structure for a custom TeXMF tree, we must register its location with kpathsea. The kpsewhich utility manages TeX file path configuration and builds the ls-R filename databases used for fast lookup. The main methods of registering trees are using the configuration files or environment variables.

Configuration Files Method

In the directory <sysconfig>/texmf, create (or edit) the "web2c" file, adding lines like:

TEXMFHOME = ~/texmf
TEXMFLOCAL = /usr/local/texlive/texmf-local

This explicitly tells programs like TeXworks where to look for the custom trees. But may need to rebuild ls-R databases after changes.

Environment Variables Method

Alternatively on Unix we can add environment variable declarations to shell startup scripts like ~/.profile:

export TEXMFHOME=~/texmf
export TEXMFLOCAL=/usr/local/texlive/texmf-local  

And similarly add them on Windows under System Properties > Environment Variables. This method makes the locations visible to TeX programs without altering configuration.

Updating the File Name Database

Along with the main tree folders, kpathsea relies on special databases to quickly map TeX requests to actual file locations. After initially creating and registering a custom TeXMF tree, we must initialize and update these filename databases so the tree is searchable.

Running mktexlsr

The mktexlsr tool rebuilds the ls-R file map of filenames to physical paths within a particular tree. Run it after establishing a new tree, adding/removing files, or altering contents. Invoked as:

mktexlsr ~/texmf  # For TEXMFHOME 
sudo mktexlsr /usr/local/texmf-local # For TEXMFLOCAL

This creates/updates the ls-R file describing contents of a tree for kpsewhich to utilize when searching for files.

Updating the Global Map

Additionally on some TeX distributions, we may need to refresh the global filename database containing pointers to all managed trees with:

sudo texhash  

This ensures the new custom trees are visible to external utilities and scripts, similar to refreshing system DNS caches.

Setting TEXMFHOME and TEXMFLOCAL

In addition to listing custom tree locations explicitly in configuration files or environment variables, most LaTeX distributions also support two special variables for user and system trees: TEXMFHOME and TEXMFLOCAL. These can be set instead of or along with the full path names.

TEXMFHOME

TEXMFHOME specifies the user-level home texmf tree for a particular operating system account. Rather than an absolute path, it dynamically resolves to a system-appropriate location like ~/.texmf or %APPDATA%\texmf. To configure, use:

export TEXMFHOME=~/texmf # Linux/macOS 
setx TEXMFHOME "%AppData%\texmf" # Windows

This will direct TeX programs to search the user home tree without specifying exact folder names. The folder will then be auto-created if missing.

TEXMFLOCAL

Similarly, TEXMFLOCAL abstractly refers to the local system-level texmf tree, separate from the user home tree. This tree typically resides at /usr/local/texlive/texmf-local or in the distribution texmf tree. Configure with:

  
export TEXMFLOCAL=/usr/local/texlive/texmf-local # Linux/macOS
setx TEXMFLOCAL %ProgramFiles%\texmf-local # Windows 

Again this avoids hardcoding paths while allowing TeX distributions to interpret the system tree location in a platform-appropriate way.

Runtime Path Configuration

When using TeX programs, the discovery of custom TeXMF trees depends on first configuring the search path to include them. This runtime path resolution affects TeX compilers, previewers, IDEs, and utilities.

Environment Variables Method

On Linux/UNIX we can prepend custom trees to the search path for the current shell session with:

export TEXINPUTS=.:~/texmf//: 
export BSTINPUTS=.:~/texmf//:
export BIBINPUTS=.:~/texmf//:

The double trailing slashes indicate "recursively traverse subdirectories".

tlaunch Method

Alternatively on Windows, the tlaunch utility started with TeX distributions can configure search paths across Windows sessions. Add entries under:

 
tlaunch -> Options -> General options -> File paths

List both TEXMFHOME and TEXMFLOCAL trees to resolve custom package and class locations.

Example Custom Tree Setup

As an integrated example, say we want to add a user-level texmf tree called mytexmf to easily access custom packages, classes, and fonts. And also add an administrator texmf tree for distribution-wide extensions called localtexmf.

On a Linux system, we could configure the user and local trees as follows:

  1. Create folder: mkdir ~/mytexmf
  2. Add template structure: mkdir {,~/mytexmf}/{tex,fonts,doc}
  3. Register: export TEXMFHOME=~/mytexmf
  4. Initialize: mktexlsr ~/mytexmf

And for the local admin tree:

  1. sudo mkdir /usr/local/texlive/localtexmf
  2. sudo mkdir/{,/usr/local/texlive/localtexmf}/{tex,fonts}
  3. Add: export TEXMFLOCAL=/usr/local/texlive/localtexmf
  4. Update: sudo mktexlsr /usr/local/texlive/localtexmf

We could then add packages, documents, etc. into mytexmf and localtexmf. And confirm they are discovered properly by kpsewhich and other utilities before use in documents.

Troubleshooting Issues

Some common issues when working with custom TeXMF trees and how to troubleshoot them:

Files Not Found

If TeX compilers fail to locate custom packages/fonts, check that:

  • Tree is registered with kpsewhich (configuration files or env variables)
  • ls-R databases rebuilt with mktexlsr after additions
  • Search path order includes custom trees (TLConfig or environment)

Old Versions Found First

If older instances of a file keep being discovered before updated ones, try:

  • Clean stale ls-R residue with mktexlsr {tree}
  • Prepend new trees earlier in the search path
  • Isolate rogue old versions and remove/hide them

Permission Errors

Custom trees may fail when invoked by programs due to restrictive permissions. Ensure:

  • TeX user has read + execute access for all constituent files/folders
  • Search path consistency across controller users, including elevating scripts with sudo if required

Maintaining Your Tree Over Time

As we continue to add new content into custom TeXMF trees, whether user-specific packages or site-wide fonts, keeping both the file system structure and kpathsea integration up to date is vital for ongoing access to inserted files.

Wrapping Additions/Removals

Get in the habit of bookending file copy and delete operations to custom trees with calls to mktexlsr and texhash. This refreshes the filename index and avoids puzzling missing file situations:

mktexlsr ~/mytexmf # Reset tree index
cp mycustom.sty ~/mytexmf/tex # Add package 
mktexlsr ~/mytexmf # Reindex additions

texhash # Refresh global ls-R database pointers 

Periodic Audits

Occasionally audit entire custom trees to catch issues early:

  • Update all ls-R databases with mktexlsr
  • Check key packages compile on isolated test documents
  • Prune unused legacy files
  • Verify permissions allow reading files as different users

This exercise may discover inconsistencies before they affect critical production workflows.

Wrapping Up

Maintaining compliance with the TeX directory structure specification when adding custom trees streamlines usage and avoids frustrating file discovery problems. A few simple steps like proper registration, indexing, search path configuration, and occasional hygiene will keep custom additions accessible.

Leave a Reply

Your email address will not be published. Required fields are marked *