How Does HDMI Transmit Audio and Video Together?
HDMI transmits uncompressed video alongside uncompressed or compressed audio across a single cable by interleaving audio packets directly into the inactive blanking intervals of the video stream. Using high-speed differential signaling, HDMI alternates between sending raw pixel values during active display moments and multi-channel audio data during the tiny pauses between lines and frames, guaranteeing seamless audio-video synchronization without requiring separate data paths.
The Physical Foundation: TMDS Architecture
At the hardware layer of standard HDMI versions up to 2.0, transmission relies on Transition Minimized Differential Signaling (TMDS). A standard HDMI interface utilizes four primary physical lanes:
- Three Data Lanes: Channels 0, 1, and 2 carry serialized color, audio, and metadata payloads.
- One Clock Lane: A dedicated physical channel that maintains clock synchronization between the transmitter (source) and the receiver (sink).
TMDS encodes 8-bit input bytes into 10-bit symbols. This 8b/10b encoding minimizes voltage transitions across the copper pairs, reducing electromagnetic interference and maintaining direct current balance across the differential wires.
The Three Operational Periods
Rather than dedicating separate physical pins to audio and video, HDMI divides its transmission time into three distinct operational periods across the three TMDS data lanes.
1. Video Data Period
During the Video Data Period, the display controller transmits active, uncompressed pixel color components. Channel 0 typically carries the blue component along with synchronization markers, Channel 1 carries green, and Channel 2 carries red (or luminance and chrominance values depending on the color space, such as RGB or YCbCr). Each active display pixel maps directly to the physical lines without lossy compression.
2. Control Period
The Control Period occurs between active lines and frames. In this phase, the interface transmits horizontal and vertical synchronization signals (HSYNC and VSYNC) as 2-bit values encoded into 10-bit control characters. These characters establish boundary lines for each frame and reset the raster scan timing on the display.
3. Data Island Period
The mechanism that enables audio to travel over the exact same wires as video is the Data Island Period. Cathode-ray tube displays originally required brief pauses between raster lines (horizontal blanking) and between full frames (vertical blanking) to reset the electron gun. Modern digital displays do not require physical beam resetting, but HDMI retains these blanking intervals.
During horizontal and vertical blanking intervals, HDMI switches from sending video pixels to transmitting 32-byte data packets known as Data Islands:
- Packet Structure: Each packet consists of a 4-byte header describing the packet type and 28 bytes of payload protected by error correction codes (BCH).
- Audio Packing: Audio data—whether pulse-code modulated (PCM) uncompressed sound or bitstream compressed audio like Dolby Digital or DTS—is packetized into these 32-byte units.
- Packet Types: Data islands also transport audio clock regeneration information, general control packets, and Consumer Electronics Control (CEC) metadata.
Because data islands fit entirely within periods where the display is not rendering pixels, audio transmission consumes zero active video bandwidth.
Clock Regeneration and Audio Synchronization
Video pixel clocks run at frequencies tied directly to display resolution and refresh rate (such as 148.5 MHz for 1080p at 60 Hz), whereas audio clocks operate at standard acoustic sampling rates (such as 48 kHz, 96 kHz, or 192 kHz).
To eliminate drift between the audio and video clocks without requiring an extra physical clock wire, HDMI uses an Audio Clock Regeneration architecture. The transmitter sends two integer values inside periodic data island packets:
- \(N\) (Numerator): A fixed fractional multiplier.
- \(CTS\) (Cycle Time Stamp): A measured counter value representing the exact number of video clock cycles elapsed over a set audio interval.
The receiver’s internal Phase-Locked Loop (PLL) recreates the original audio master clock frequency (\(f_s\)) by computing:
\[f_s = f_{\text{TMDS\_clock}} \times \frac{N}{CTS}\]
This continuous mathematical adjustment ensures that audio playback remains locked in phase with the incoming video frames.
Modern Evolution: Fixed Rate Link (HDMI 2.1)
HDMI 2.1 introduced Fixed Rate Link (FRL), replacing TMDS for ultra-high-bandwidth transmissions up to 48 Gbps. FRL reclaims the physical clock lane as a fourth data lane and switches to an efficient 16b/18b encoding scheme with embedded clock recovery.
Despite the architectural shift in physical signaling, the underlying logical concept remains identical: active video, packetized audio, and supplementary display metadata are interleaved into high-speed serial packet bursts, allowing rich surround sound and pristine high-resolution frames to travel harmoniously through a single connection.