XML Parser Character Encoding Mismatch Behavior

According to the official W3C XML specification, an XML parser must treat a character encoding mismatch or invalid byte sequence as a fatal error. When such a conflict occurs, a conforming parser must immediately halt normal processing, refuse to pass the corrupted or ambiguous data to the consuming application, and report the error. XML was intentionally designed without lenient error recovery to prevent silent data corruption, meaning parsers are prohibited from guessing or attempting to auto-correct mismatched encodings.

The Fatal Error Rule

The W3C XML standard enforces a strict “no error recovery” policy for well-formedness and encoding violations:

• Immediate Halt: As soon as an encoding mismatch or illegal byte sequence is detected, the parser must stop normal data processing.
• Error Reporting: The parser must generate a fatal error message indicating the encoding failure.
• No Speculative Correction: The parser is strictly prohibited from guessing the encoding, replacing invalid characters silently, or attempting to repair the document.

Determining Encoding Precedence

Encoding mismatches typically occur when different sources declare conflicting encodings. Standard parsers follow a strict hierarchy of precedence to determine the true encoding:

1. External Transport Headers: If the XML document is served over a protocol like HTTP, the MIME type declaration in the Content-Type header (e.g., Content-Type: application/xml; charset=UTF-8) generally takes precedence.
2. Byte Order Mark (BOM): If present at the beginning of the file, the BOM explicitly dictates whether the file is UTF-8, UTF-16 (Big/Little Endian), or UTF-32.
3. Internal XML Declaration: The encoding attribute in the prolog (e.g., <?xml version="1.0" encoding="ISO-8859-1"?>) is used if there are no overriding external headers or conflicting BOMs.
4. Default Fallback: If no encoding is declared, conforming parsers default to UTF-8 (or UTF-16 if detected via a BOM).

Common Mismatch Scenarios

• BOM vs. Prolog Declaration: If a document contains a UTF-8 BOM but explicitly declares encoding="UTF-16", the parser must raise a fatal error and stop.
• HTTP Header vs. Prolog Declaration: If the HTTP header specifies charset=UTF-8 but the document contains encoding="windows-1252", the parser must prioritize the transport layer or raise a mismatch error if the byte streams do not conform.
• Invalid Byte Sequences: If a file is declared as UTF-8 but contains non-UTF-8 byte sequences (such as isolated high bytes typical of Latin-1), the parser must immediately fail.