Source: https://guthlabs.ai/docs/instruments/unicode-ascii-code-point-lookup/mechanism/

instrument mechanism / verified-clean

# Unicode / ASCII Code Point Lookup mechanism

The inspectable method record for Unicode / ASCII Code Point Lookup, kept separate from implementation and qualification.

Status: **VERIFIED_CLEAN**Lifecycle: **verified-clean**

## Model scope

The UTF-8 and UTF-16 algorithms are standard when restricted to valid Unicode scalar values. The instrument should expose its Unicode/UCD version and clearly display limited character-name coverage, as stated by the honest_label. Final acceptance requires decisions on pasted-string semantics, invalid-input behavior, Unicode versioning, and whether HTML output is named or numeric and decimal or hexadecimal.

- The stated UTF-8 rule is standard and correct for Unicode scalar values, but it is underspecified: implementation must apply the exact bit-packing rules for 1-, 2-, 3-, and 4-byte sequences.

- The specification does not state the UTF-16 conversion rule: code points U+0000 through U+FFFF excluding surrogates use one code unit, while supplementary code points use a high/low surrogate pair.

- The specification does not define handling for invalid code points above U+10FFFF, surrogate code points U+D800 through U+DFFF, malformed input, or an empty string.

- The input 'pasted string' is ambiguous: the instrument must define whether it reports one Unicode scalar, one extended grapheme cluster, or all scalar values in the string.

- The Unicode version for character names and encoding metadata is unspecified. Results should identify the bundled Unicode Character Database version.

- The HTML entity output is ambiguous because it could mean a named entity, a decimal numeric character reference, or a hexadecimal numeric character reference. Test outputs below use hexadecimal numeric character references such as A.

## Units and conventions

- Code points are Unicode scalar values represented in hexadecimal as U+XXXX and in decimal as nonnegative integers.

- UTF-8 outputs are byte values shown as two-digit hexadecimal octets in transmission order.

- UTF-16 outputs are 16-bit code units shown in hexadecimal; no byte endianness is implied.

- HTML outputs use hexadecimal numeric character references in the form &#xHEX;.

- Character names and the listed code-point records use the Unicode Character Database, version 15.1.0.

## Inspectable vector

One of 4 source-bound review vectors:

````
{
  "inputs": {
    "character": "A"
  },
  "expected_outputs": {
    "code_point_decimal": 65,
    "code_point_hex": "U+0041",
    "character_name": "LATIN CAPITAL LETTER A",
    "utf8_bytes": [
      "41"
    ],
    "utf16_code_units": [
      "0041"
    ],
    "html_entity": "A"
  },
  "reference_data": {
    "unicode_character_record": {
      "code_point": "U+0041",
      "name": "LATIN CAPITAL LETTER A",
      "source_convention": "Unicode Character Database 15.1.0 / Unicode Standard"
    },
    "encoding_rule_fragment": {
      "applicable_range": "U+0000..U+007F",
      "rule": "UTF-8 byte 1 = code point",
      "source_convention": "Unicode Standard, UTF-8 encoding form"
    },
    "utf16_rule_fragment": {
      "applicable_range": "U+0000..U+FFFF excluding surrogates",
      "rule": "one UTF-16 code unit equal to the code point",
      "source_convention": "Unicode Standard, UTF-16 encoding form"
    },
    "html_reference_convention": {
      "rule": "hexadecimal numeric character reference &#xHEX;",
      "source_convention": "WHATWG HTML Living Standard"
    }
  },
  "working": "U+0041 = decimal 65. UTF-8: 0x41 is in U+0000..U+007F, so emit one byte 0x41. UTF-16: BMP non-surrogate value, so emit one code unit 0x0041. HTML numeric reference: code point hexadecimal value 41 gives A."
}
````

## Failure boundary

- Reject code points greater than U+10FFFF.

- Reject surrogate code points U+D800..U+DFFF as standalone Unicode scalar values; they must not be encoded as ordinary UTF-8.

- Prevent overlong UTF-8 encodings and reject invalid continuation-byte patterns if decoding is also supported.

- A pasted string can contain multiple scalars, combining marks, variation selectors, or emoji sequences; character-name and encoding output must define whether all scalars or only one grapheme cluster is processed.

- Character-name lookup can fail for unassigned characters, private-use characters, control characters, and characters outside the bundled subset.

- UTF-16 code units are not necessarily equivalent to user-perceived characters; supplementary characters produce two code units.

- If HTML output is inserted into a page, escaping and context handling are required; a numeric character reference is not a general-purpose HTML sanitization mechanism.

## Other records

[01**Overview**](<https://guthlabs.ai/docs/instruments/unicode-ascii-code-point-lookup/>)[03**Implementation**](<https://guthlabs.ai/docs/instruments/unicode-ascii-code-point-lookup/implementation/>)[04**Qualification**](<https://guthlabs.ai/docs/instruments/unicode-ascii-code-point-lookup/qualification/>)

## Sources and evidence

These pointers bind the page to evidence. They do not grant authority or collapse verification into deployment.

- `VERIFIED_LEDGER.json`

- `evidence-tranche7/tranche7-results.json`