How to Parse HTML and XML with Ruby Nokogiri

Ruby’s Nokogiri gem is a powerful, high-performance library used for parsing, searching, and manipulating HTML and XML documents. Built as a Ruby wrapper around the native C libraries libxml2 and libxslt, Nokogiri provides developers with a fast and intuitive interface. This article explains how Nokogiri loads raw markup, queries elements using CSS selectors and XPath, and updates or modifies document structures programmatically.

1. Parsing Documents

Nokogiri converts raw HTML or XML strings, files, or open network streams into an in-memory Document Object Model (DOM).

Parsing HTML

To parse an HTML document, pass a string or file stream to Nokogiri::HTML5 (for modern HTML5 standards) or Nokogiri::HTML4:

require 'nokogiri'

html_data = '<div><p class="intro">Hello World</p></div>'
doc = Nokogiri::HTML5(html_data)

Parsing XML

Parsing XML works similarly using the Nokogiri::XML method:

xml_data = '<catalog><book id="1"><title>Ruby Basics</title></book></catalog>'
doc = Nokogiri::XML(xml_data)

Nokogiri automatically handles malformed documents by attempting to correct syntax errors during parsing.

2. Searching Documents

Once a document is parsed, Nokogiri allows you to traverse and query nodes using either standard CSS selectors or XPath expressions.

Using CSS Selectors

The .css method returns a NodeSet (an array-like collection of elements) matching the provided selector:

# Retrieve all matching elements
paragraphs = doc.css('p.intro')

# Retrieve the first matching element
first_para = doc.at_css('p.intro')
puts first_para.text # Output: Hello World

Using XPath

XPath allows for complex queries, such as selecting nodes based on their exact hierarchy or inner text content:

# Retrieve all titles inside book tags
titles = doc.xpath('//catalog/book/title')

# Retrieve a single element using XPath
first_title = doc.at_xpath('//catalog/book/title')

3. Manipulating Nodes and Content

Nokogiri provides direct methods to edit content, change attributes, and add or remove elements in the document tree.

Updating Text and Attributes

You can read and modify the inner text or attributes of any selected node:

node = doc.at_css('p.intro')

# Modify text content
node.content = 'Welcome to Nokogiri'

# Modify or add attributes
node['class'] = 'highlighted'
node['id'] = 'main-text'

Adding New Elements

You can create new elements or insert markup relative to an existing node using methods like add_child, before, or after:

# Append a child node
container = doc.at_css('div')
container.add_child('<span>Additional Info</span>')

# Insert a sibling node
node.after('<p>Second paragraph</p>')

Removing Elements

To delete an element completely from the document tree, call .remove (or its alias .unlink):

doc.css('.remove-me').remove

4. Outputting Serialized Markup

After modifying the DOM, convert the document or any individual node back into a formatted string:

# Output full HTML
clean_html = doc.to_html

# Output full XML
clean_xml = doc.to_xml(indent: 2)

Nokogiri ensures that the exported markup is valid, properly nested, and formatted according to the document’s specification.