How Debiman Renders Debian Man Pages

Debiman is the custom, Go-based documentation engine responsible for powering Debian's official online manual repository at manpages.debian.org. It processes hundreds of thousands of manual pages packaged across various Debian suites, formats them from raw roff sources into modern HTML, and delivers them rapidly through an optimized combination of static generation and dynamic request routing. This article examines the ingestion pipeline, the rendering mechanisms, cross-referencing logic, and the serving architecture that enables debiman to present Debian's software documentation seamlessly.

Archive Ingestion and Package Extraction

Debiman operates directly on the Debian package archive mirrors. Instead of installing packages into a virtual environment, it inspects the Debian Contents indices and downloads raw binary package archives (.deb files) across multiple suites, including stable, testing, unstable, and experimental, as well as backports.

Once acquired, debiman inspects the packages internally:

  • It scans standard manual directories, primarily /usr/share/man/.
  • It detects and decompresses manual source files, which are commonly compressed with gzip.
  • It extracts metadata from the Debian control files, such as package name, version, architecture, and suite, attaching this context to every extracted manual page.

The Conversion Pipeline: From roff to HTML

Manual pages in Unix-like systems are authored using roff (specifically troff or groff) macro packages like man and mdoc. Debiman converts these raw markup files into semantic HTML using mandoc, a specialized compiler designed to parse manual page formats quickly and safely.

Using mandoc instead of traditional groff pipelines provides significant performance improvements and emits structured HTML5 instead of terminal control sequences. Debiman interfaces directly with mandoc to:

  1. Parse the structural tags of the manual page (such as SH for section headers, TP for tagged paragraphs, and B/I for bold/italic styling).
  2. Clean up non-standard or malformed markup that exists across older software packages.
  3. Emit uniform HTML elements that adhere to modern accessibility and responsive design standards.

A critical feature of debiman is its ability to build an interconnected web of manual pages out of isolated source files. When a manual page references another utility—such as tar(1) referencing gzip(1)—debiman dynamically resolves these references into functional hyperlinks.

During the conversion phase, debiman builds an internal index containing:

  • The command or function name.
  • The section number (e.g., 1 for user commands, 2 for system calls, 3 for library functions, 5 for file formats).
  • The language or locale code.
  • The originating Debian suite and package version.

If a reference to example(2) is parsed, debiman queries this index to determine the canonical path for that page within the same suite and language, wrapping the text in an <a> anchor tag that targets the correct relative or absolute URL.

Content Deduplication and Storage

Debian maintains vast numbers of packages, many of which share identical manual pages across architectures and even across releases. To prevent redundant computation and conserve storage, debiman implements content-addressable deduplication:

  • Debiman calculates cryptographic hashes (such as SHA-256) of the raw and rendered manual page files.
  • If a package in Debian testing contains the exact same man page binary or roff source as stable, debiman reuses the previously rendered HTML asset.
  • Symlinks or alias references are mapped in an internal database, ensuring that updates to packages only trigger re-rendering for assets that have actually changed.

Hybrid Dynamic Serving: Static HTML with Dynamic Routing

While debiman generates HTML files ahead of time to ensure low latency and high scalability, the end-user experience behaves dynamically through an intelligent HTTP redirect and lookup layer known as debiman-auxserver.

When a user requests a URL like /grep:

  1. The auxiliary server receives the request and determines the client's preferred language through HTTP Accept-Language headers.
  2. It evaluates the suite hierarchy, defaulting to the current Debian stable release unless an explicit path (such as /testing/ or /unstable/) is requested.
  3. It performs an in-memory route lookup to locate the best matching static file (e.g., /jessie/grep.1.en.html or /bookworm/grep.1.fr.html).
  4. The server redirects the user or directly serves the resolved static page with appropriate caching headers.

This hybrid model allows debiman to handle millions of page variations, historical versions, and localization preferences with minimal CPU overhead while maintaining the responsiveness of a dynamic web application.