Character Encoding Converter (UTF-8/16/32, Latin-1) mechanism

Model scope

The spec must explicitly define UTF-16/UTF-32 endianness, BOM policy, malformed-input behavior, Latin-1 versus Windows-1252, replacement policy, and the algorithm or limitations for mojibake-risk detection. The empty honest_label is a concern: the instrument should state these limitations rather than imply that encoding conversion can reliably detect mojibake.

• The formula description is incomplete: byte-length rules alone do not define encoding conversion, decoding, malformed-sequence handling, or replacement behavior.
• UTF-16 and UTF-32 require an explicit endianness and BOM policy; the spec only names UTF-16 and UTF-32.
• UTF-8 validity must reject overlong encodings, surrogate code points, code points above U+10FFFF, invalid continuation bytes, and truncated sequences.
• UTF-16 validity must reject unpaired surrogates and define whether a BOM is accepted, required, emitted, or counted.
• Latin-1 is ISO-8859-1 only if explicitly distinguished from Windows-1252; characters above U+00FF are unmappable and require a defined error policy.
• Mojibake risk has no stated computable formula; arbitrary text cannot reliably be classified as mojibake from encoding metadata alone.

Units and conventions

• All byte lengths are counts of octets (8-bit bytes), not character counts or UTF-16 code-unit counts.
• UTF-16LE and UTF-32LE are used in the test vectors; no BOM is emitted or counted.
• UTF-8 uses the standard scalar-value ranges: U+0000-U+007F uses 1 byte, U+0080-U+07FF uses 2, U+0800-U+D7FF and U+E000-U+FFFF use 3, and U+10000-U+10FFFF uses 4.
• UTF-16LE uses 2 bytes for each non-surrogate BMP scalar and 4 bytes for each supplementary scalar represented by a surrogate pair.
• UTF-32LE uses 4 bytes for each valid Unicode scalar value.
• Latin-1 means ISO-8859-1, mapping U+0000-U+00FF to one byte; it does not mean Windows-1252.

Inspectable vector

One of 4 source-bound review vectors:

{
  "inputs": {
    "text": "Cat",
    "source_encoding": "Unicode text",
    "target_encoding": "UTF-16LE"
  },
  "expected_outputs": {
    "utf8_bytes_hex": "43 61 74",
    "utf8_byte_length": 3,
    "utf16le_bytes_hex": "43 00 61 00 74 00",
    "utf16le_byte_length": 6,
    "utf32le_bytes_hex": "43 00 00 00 61 00 00 00 74 00 00 00",
    "utf32le_byte_length": 12,
    "latin1_bytes_hex": "43 61 74",
    "latin1_byte_length": 3,
    "target_bytes_hex": "43 00 61 00 74 00",
    "invalid_sequence_warnings": []
  },
  "working": "U+0043, U+0061, and U+0074 are each in U+0000-U+007F, so UTF-8 length = 1+1+1 = 3 bytes. Each is a BMP scalar, so UTF-16LE length = 2+2+2 = 6 bytes. UTF-32LE length = 4+4+4 = 12 bytes. All three code points are <= U+00FF, so Latin-1 length = 1+1+1 = 3 bytes. UTF-16LE encoding is U+0043 -> 43 00, U+0061 -> 61 00, U+0074 -> 74 00."
}

Failure boundary

BREAK_CLAIMED · SELF_FLAGGED

The specification does not define UTF-16 endianness or how BOM-less UTF-16 input is interpreted. The bytes 00 41 validly decode as U+4100 under UTF-16LE or U+0041 under UTF-16BE, producing different UTF-8 outputs.

Other records

Sources and evidence

These pointers bind the page to evidence. They do not grant authority or collapse verification into deployment.

• VERIFIED_LEDGER.json
• evidence-tranche5/tranche5-results.json
• BOUNDARY_HAZARD_CATALOG.json