# TOON Specification Compliance Report

**Library:** toon-php
**Version:** 2.0.0
**Spec Version:** TOON Specification v2.0
**Date:** 2025-11-13
**Status:** ✅ FULLY CONFORMANT

---

## Test Results

- **Total Tests:** 544
- **Passing:** 544 (100%)
- **Failing:** 0
- **PHPStan Level:** 9 (maximum strictness)
- **PHPStan Errors:** 0

---

## v2.0 Compliance Notes

This release aligns with **TOON Specification v2.0**, which removes the optional `#` length marker prefix:

- **Encoder**: Always emits `[N]` format (e.g., `[3]: a,b,c`)
- **Decoder**: Rejects `[#N]` format with `SyntaxException`
- **Breaking Change**: Removed `lengthMarker` parameter and related methods from `EncodeOptions`

### Control Character Handling

Per TOON Spec §7.1, only three control characters have defined escape sequences:

- `\n` (newline, 0x0A)
- `\r` (carriage return, 0x0D)
- `\t` (tab, 0x09)

**Implementation Policy**: Strings containing other control characters (0x00-0x08, 0x0B, 0x0C, 0x0E-0x1F) are **rejected** with `InvalidArgumentException`, as the spec provides no escape sequences for them. This prevents potential security issues from raw control characters in output.

**Test Coverage**: `PrimitivesTest.php` includes 5 tests verifying rejection of unsupported control characters (NULL, BEL, ESC, FF, VT)

---

## Encoder Conformance Checklist (§13.1)

Conforming encoders MUST:

- [x] **Produce UTF-8 output with LF (U+000A) line endings (§5)**
  - ✅ Verified in `Encoders.php`: Uses PHP string concatenation with `\n`
  - ✅ Test coverage: All encoding tests verify output format

- [x] **Use consistent indentation (default 2 spaces, no tabs) (§12)**
  - ✅ Implemented in `LineWriter.php`: `str_repeat(Constants::SPACE, $options->indent)`
  - ✅ Default: 2 spaces (EncodeOptions::default())
  - ✅ Test coverage: `EdgeCasesTest.php`, `FormatInvariantsTest.php`

- [x] **Escape \\, ", \n, \r, \t in quoted strings; reject other escapes (§7.1)**
  - ✅ Implemented in `Primitives.php:escapeString()`
  - ✅ Only escapes: `\\`, `\"`, `\n`, `\r`, `\t`
  - ✅ Rejects strings with unsupported control characters (0x00-0x08, 0x0B, 0x0C, 0x0E-0x1F)
  - ✅ Test coverage: `PrimitivesTest.php` (escape tests + 5 control character rejection tests)

- [x] **Quote strings containing active delimiter, colon, or structural characters (§7.2)**
  - ✅ Implemented in `Primitives.php:needsQuoting()`
  - ✅ Quotes strings with: delimiter, colon, structural chars
  - ✅ Test coverage: `PrimitivesTest.php`, `DelimitersTest.php`

- [x] **Emit array lengths [N] matching actual item count (§6, §9)**
  - ✅ Implemented in `Encoders.php`: Uses `count()` for all array headers
  - ✅ Inline arrays: `[count($array)]:`
  - ✅ List arrays: `[count($array)]:`
  - ✅ Tabular arrays: `[count($array)]`
  - ✅ Test coverage: All array tests verify length accuracy

- [x] **Preserve object key order as encountered (§2)**
  - ✅ PHP arrays maintain insertion order
  - ✅ `Encoders::encodeObject()` iterates keys in order
  - ✅ Test coverage: `ArraysTest.php`, `TabularArraysTest.php`

- [x] **Normalize numbers to non-exponential decimal form (§2)**
  - ✅ Implemented in `Primitives.php:encodePrimitive()`
  - ✅ Converts exponential notation to decimal
  - ✅ Uses locale-independent `number_format()` with explicit '.' decimal separator
  - ✅ Test coverage: `NormalizationTest.php`, `PrimitivesTest.php`

- [x] **Convert -0 to 0 (§2)**
  - ✅ Implemented in `Normalize.php:normalizeNumber()`
  - ✅ Explicitly checks for `-0.0` and converts to `0`
  - ✅ Test coverage: `NormalizationTest.php`

- [x] **Convert NaN/±Infinity to null (§3)**
  - ✅ Implemented in `Normalize.php:normalizeNumber()`
  - ✅ `is_nan()`, `is_infinite()` → `null`
  - ✅ Test coverage: `NormalizationTest.php`

- [x] **Emit no trailing spaces or trailing newline (§12)**
  - ✅ Implemented in `LineWriter.php:toString()`
  - ✅ Returns trimmed output, no trailing newline
  - ✅ Test coverage: `FormatInvariantsTest.php`

**Encoder Conformance: ✅ 10/10 - FULLY CONFORMANT**

---

## Decoder Conformance Checklist (§13.2)

Conforming decoders MUST:

- [x] **Parse array headers per §6 (length, delimiter, optional fields)**
  - ✅ Implemented in `Decoder/HeaderParser.php`
  - ✅ Parses: `[N]:`, `[N|]:`, `[N]{fields}:`
  - ✅ **Rejects** `[#N]` format (removed in v2.0) with clear error message
  - ✅ Test coverage: Comprehensive header parsing tests + v2.0 breaking change tests

- [x] **Split inline arrays and tabular rows using active delimiter only (§11)**
  - ✅ Implemented in `Decoder/DelimiterParser.php:split()`
  - ✅ Respects quoted strings, only splits on unquoted delimiter
  - ✅ Test coverage: `DelimitersTest.php`

- [x] **Unescape quoted strings with only valid escapes (§7.1)**
  - ✅ Implemented in `Decoder/ValueParser.php:unescapeString()`
  - ✅ Only unescapes: `\\`, `\"`, `\n`, `\r`, `\t`
  - ✅ Throws on invalid escape sequences
  - ✅ Test coverage: `PrimitivesTest.php`

- [x] **Type unquoted primitives: true/false/null → booleans/null, numeric → number, else → string (§4)**
  - ✅ Implemented in `Decoder/ValueParser.php:parseValue()`
  - ✅ "true"/"false" → boolean
  - ✅ "null" → null
  - ✅ Numeric strings → int/float
  - ✅ Everything else → string
  - ✅ Test coverage: `PrimitivesTest.php`

- [x] **Enforce strict-mode rules when strict=true (§14)**
  - ✅ Implemented in `Decoder/StrictValidator.php`
  - ✅ Default: `strict=true`
  - ✅ Validates: array counts, row widths, blank lines, colon presence
  - ✅ Test coverage: Comprehensive strict mode tests

- [x] **Preserve array order and object key order (§2)**
  - ✅ PHP arrays maintain insertion order
  - ✅ Parser preserves order during decoding
  - ✅ Test coverage: Round-trip tests verify order preservation

**Decoder Conformance: ✅ 7/7 - FULLY CONFORMANT**

---

## Strict Mode Requirements (§14)

When strict mode is enabled (default), decoders MUST error on:

### 14.1 Array Count and Width Mismatches

- [x] **Inline primitive arrays: decoded value count ≠ declared N**
  - ✅ `StrictValidator::validateArrayCount()` - line 68
  - ✅ Test: `EdgeCasesTest.php`

- [x] **List arrays: number of list items ≠ declared N**
  - ✅ `Parser::parseListArrayFromHeader()` - validates count
  - ✅ Test: List array tests

- [x] **Tabular arrays: number of rows ≠ declared N**
  - ✅ `Parser::parseTabularArrayFromHeader()` - validates count
  - ✅ Test: `TabularArraysTest.php`

- [x] **Tabular row width mismatches: any row's value count ≠ field count**
  - ✅ `StrictValidator::validateTabularRowWidth()` - line 86
  - ✅ Test: `TabularArraysTest.php`

### 14.2 Syntax Errors

- [x] **Missing colon in key context**
  - ✅ `StrictValidator::validateColonPresent()` - line 17
  - ✅ Test: Syntax error tests

- [x] **Invalid escape sequences or unterminated strings in quoted tokens**
  - ✅ `ValueParser::unescapeString()` - throws on invalid escapes
  - ✅ Test: `EdgeCasesTest.php`

- [x] **Delimiter mismatch (detected via width/count checks and header scope)**
  - ✅ Implicit via width/count validation
  - ✅ Test: `DelimitersTest.php`

### 14.3 Indentation Errors

- [x] **Leading spaces not a multiple of indentSize**
  - ✅ `Tokenizer::tokenize()` - validates indent alignment
  - ✅ Test: Indentation tests

- [x] **Any tab used in indentation (tabs allowed in quoted strings and as HTAB delimiter)**
  - ✅ `Tokenizer::tokenize()` - checks for tabs in indentation
  - ✅ Test: Indentation tests

### 14.4 Structural Errors

- [x] **Blank lines inside arrays/tabular rows**
  - ✅ `StrictValidator::validateNoBlankLinesInArray()` - line 99
  - ✅ Test: Array tests

- [x] **Empty input (document with no non-empty lines after ignoring trailing newline(s))**
  - ✅ `StrictValidator::validateNotEmpty()` - line 30
  - ✅ Test: `EdgeCasesTest.php`

**Strict Mode Conformance: ✅ 11/11 - FULLY CONFORMANT**

---

## Architecture Verification

### Instance-Based Pattern (Post-Refactoring)

Both Encoder and Decoder now use consistent instance-based architecture:

**Encoder:**

- Constructor: stores `EncodeOptions` and `LineWriter` as readonly properties
- No parameter threading through methods
- Clean, maintainable code structure

**Decoder:**

- Constructor: stores `DecodeOptions` as readonly property
- No parameter threading through methods
- Matches Encoder pattern

### Static Utilities

Both use static utility classes appropriately:

- `Primitives` - Pure functions for encoding primitives
- `ValueParser` - Pure functions for parsing values
- `DelimiterParser` - Pure functions for delimiter handling
- `HeaderParser` - Pure functions for header parsing
- `StrictValidator` - Pure validation functions
- `Normalize` - Pure normalization functions

---

## Performance Verification

Post-refactoring benchmarks show:

- Minimal performance impact (< 3% worst case, often better)
- Some metrics improved (throughput +6.65%)
- All 539 tests pass
- PHPStan Level 9 clean

---

## Conclusion

**Status: ✅ FULLY CONFORMANT WITH TOON SPECIFICATION v1.3**

The toon-php library successfully implements all MUST requirements for:

- Encoder conformance (10/10 requirements)
- Decoder conformance (7/7 requirements)
- Strict mode validation (11/11 requirements)

The library is ready for release as v1.3.0 with confidence in its:

- Specification compliance
- Type safety (PHPStan Level 9)
- Test coverage (539 passing tests)
- Code quality (clean architecture)
- Performance characteristics (benchmarked)

---

## References

- TOON Specification v1.3: `/docs/SPEC.md`
- Encoder Implementation: `/src/Encoders.php`, `/src/Toon.php`
- Decoder Implementation: `/src/Decoder/` directory
- Test Suite: `/tests/` directory (539 tests)
- Benchmarks: `/benchmarks/results/`