WRWEB ENGINEERING
REFORGING09 · 1/7
ALL LESSONS→WEEK 09→MONDAY

UNDERSTAND · HTML TO THE FRONTIER

Tokenization and parsing

From bytes to a document30–60 MINUTESCORE + PRACTICAL
01

GROUND

Problem

What becomes confusing, fragile, or impossible without understanding tokenization and parsing? This lesson answers that through explanation, a worked example, two runnable exercises, and a reference solution. No teacher-supplied worksheet is required.

HTML evolved from linked documents into a semantic application platform with forms, native validation, disclosure, dialogs, media, and accessibility mappings.
02

LEARN

Concept explanation

Tokenization and parsing belongs to “From bytes to a document”. HTML parsing constructs a fault-tolerant DOM whose semantics feed accessibility APIs.

For tokenization and parsing, trace concrete input, state transition, output, and failure through a parser-to-DOM-to-accessibility-tree contract with native behavior before JavaScript.

Markup enters a parser, becomes a DOM, maps into accessibility semantics, and exposes native controls with built-in state and behavior. Apply that model to supplied normal, boundary, and failure cases; each case below names its input and expected evidence.

Observable

Evidence produced by the tokenization and parsing experiment: output, state, trace, bytes, timing, or diagnostics.

Invariant

Condition that must remain true while inputs or implementation of tokenization and parsing change.

Boundary

Point where tokenization and parsing crosses ownership, representation, time, process, network, or trust.

Example bank

Compare normal, boundary, failure, and cross-layer cases. Predict each observation before revealing the explanation.

Baseline · one variable

SETUPCreate document with title, main, headings, list, link, and correctly labeled form control.

OBSERVEDOM differs predictably from malformed source; landmark and names appear in accessibility tree; missing label is detectable.

WHY IT MATTERSThis isolates the normal contract of tokenization and parsing; preserve its raw evidence as the control for every later comparison.

Boundary · same contract, harder input

SETUPIntroduce recoverable malformed nesting and compare source with DOM.

OBSERVERecord what remains invariant and the first representation, owner, size, or timing value that changes in Chromium Elements · Accessibility · Network.

WHY IT MATTERSA boundary example is useful only when one named dimension changes and everything else stays comparable.

Failure · evidence before repair

SETUPRemove label and identify lost accessible name.

OBSERVECapture the first divergence from the baseline, including exact input, diagnostic, state, and recovery result. Expected recovery: DOM differs predictably from malformed source; landmark and names appear in accessibility tree; missing label is detectable.

WHY IT MATTERSThe diagnostic is part of the interface. Repair the proven cause, not the most visible symptom.

Cross-layer · follow ownership

SETUPTrace tokenization and parsing one layer below its usual abstraction through a parser-to-DOM-to-accessibility-tree contract with native behavior before JavaScript.

OBSERVEUse Elements and Accessibility panels; compare source markup, DOM properties, focus order, accessible names, and submitted form data.

WHY IT MATTERSThe lower layer is earned when it explains evidence the current layer cannot. Otherwise keep tokenization and parsing at the simpler boundary.

03

SEE

Worked example

Start from supplied index.html. Focus: Create document with title, main, headings, list, link, and correctly labeled form control.

  1. Run: Open index.html directly, then inspect DOM and Accessibility trees.
  2. Save baseline evidence. Use Elements and Accessibility panels; compare source markup, DOM properties, focus order, accessible names, and submitted form data.
  3. Boundary case: Introduce recoverable malformed nesting and compare source with DOM.
  4. Failure case: Remove label and identify lost accessible name.
RESULT
DOM differs predictably from malformed source; landmark and names appear in accessibility tree; missing label is detectable. Starter-level baseline: Page has one main landmark, one level-one heading, a correctly named input, native required validation, and a submit action.
04

START HERE

Starter material

PREREQUISITESA modern browser and plain-text editor. No package install or server required for baseline lessons.

ONE-TIME SETUPCreate an empty folder, save starter as index.html, then use browser File → Open File to open it.

Create index.html, paste this exact content, then run the command below.

<!doctype html>
<html lang="en">
<meta charset="utf-8">
<title>tokenization and parsing</title>
<main>
  <h1>tokenization and parsing</h1>
  <form><label>Observation <input name="observation" required></label>
    <button>Submit</button></form>
  <output aria-live="polite">Ready</output>
</main>
RUNOpen index.html directly, then inspect DOM and Accessibility trees.

STOP / CLEANUPClose the browser tab when finished; the file remains portable.

05

DO WITH GUIDANCE

Guided exercise

Observe one rule: tokenization and parsing

  1. Normal case: Create document with title, main, headings, list, link, and correctly labeled form control.
  2. Write predicted evidence from this named case before running starter.
  3. Change one input while holding environment constant.
  4. Run exact normal case. Save commands, inputs, outputs, and diagnostics in notebook.
  5. Explain changed evidence using lesson mental model in no more than five sentences.
Concrete guided solution
  1. Copy the supplied index.html unchanged and run: Open index.html directly, then inspect DOM and Accessibility trees.
  2. Write this prediction before inspecting output: DOM differs predictably from malformed source; landmark and names appear in accessibility tree; missing label is detectable.
  3. Perform only the named normal case: Create document with title, main, headings, list, link, and correctly labeled form control.
  4. Save the raw output, then annotate input → transition → evidence. Use Chromium Elements · Accessibility · Network to confirm the transition rather than inferring it.
  5. Compare prediction with evidence; if they differ, keep both and write the rule that explains the difference. Reference baseline: Page has one main landmark, one level-one heading, a correctly named input, native required validation, and a submit action.
06

DO ALONE

Independent exercise

Find the boundary: tokenization and parsing

  1. Create second case from blank file: Introduce recoverable malformed nesting and compare source with DOM.
  2. Then create controlled failure: Remove label and identify lost accessible name.
  3. Use Chromium Elements · Accessibility · Network to prove behavior, then repair controlled failure.
  4. Compare result against supplied acceptance checks and reference approach before marking complete.
Concrete independent solution
  1. Duplicate the starter into a clean comparison case; change only this boundary: Introduce recoverable malformed nesting and compare source with DOM.
  2. Save its evidence beside the baseline and identify the first changed value. Use Elements and Accessibility panels; compare source markup, DOM properties, focus order, accessible names, and submitted form data.
  3. Create the exact controlled failure: Remove label and identify lost accessible name.
  4. Reproduce baseline exactly: Create document with title, main, headings, list, link, and correctly labeled form control.
  5. Write observation table with columns input, state transition, output, and failure for tokenization and parsing.
  6. Run boundary case unchanged: Introduce recoverable malformed nesting and compare source with DOM.
  7. Trigger controlled failure: Remove label and identify lost accessible name.
  8. Compare evidence with reference outcome: DOM differs predictably from malformed source; landmark and names appear in accessibility tree; missing label is detectable.
  9. Rerun baseline, boundary, and repaired failure together. Accept only if all reproduce: DOM differs predictably from malformed source; landmark and names appear in accessibility tree; missing label is detectable.
07

COMPARE

Expected result

  • DOM differs predictably from malformed source; landmark and names appear in accessibility tree; missing label is detectable.
  • Page has one main landmark, one level-one heading, a correctly named input, native required validation, and a submit action.
  • Controlled tokenization and parsing failure produces captured evidence; repair restores stated invariant without hiding error.
08

PROVE

Acceptance checks

Lesson is complete only when every check is true. Each check is stored locally and travels with your JSON backup.

0/5 complete · saved on this device

09

UNSTICK

Hints

Reveal hints
  1. Start with supplied normal case exactly as written: Create document with title, main, headings, list, link, and correctly labeled form control.
  2. For boundary case, change only named dimension: Introduce recoverable malformed nesting and compare source with DOM.
  3. If result is confusing, diff raw inputs and evidence before editing implementation.
  4. If tool shows nothing useful, move observation one boundary lower: representation, runtime, OS, or network.
10

VERIFY

Solution

Attempt both exercises before opening reference approach.

Reveal reference solution
  1. Run unmodified starter and preserve baseline evidence: Page has one main landmark, one level-one heading, a correctly named input, native required validation, and a submit action.
  2. Reproduce baseline exactly: Create document with title, main, headings, list, link, and correctly labeled form control.
  3. Write observation table with columns input, state transition, output, and failure for tokenization and parsing.
  4. Run boundary case unchanged: Introduce recoverable malformed nesting and compare source with DOM.
  5. Trigger controlled failure: Remove label and identify lost accessible name.
  6. Compare evidence with reference outcome: DOM differs predictably from malformed source; landmark and names appear in accessibility tree; missing label is detectable.
11

PREDICT · INSPECT · BREAK · DEBUG · MEASURE

Interrogate reality

Prediction: write expected output, state transition, ordering, and failure evidence before running either exercise.

Inspection: Use Elements and Accessibility trees; compare attributes, properties, focus order, form data, and no-JavaScript behavior.

Measurement: Measure transferred bytes, parse cost, interaction readiness, keyboard steps, and failure behavior without scripts.

INSPECT

Capture raw evidence before explaining.

BREAK

Change one assumption and force controlled failure.

DEBUG

Find cause with Chromium Elements · Accessibility · Network before editing fix.

TOOL DRILL · keyboard only · record one retrievable command or shortcut
12

MASTERY + FRONTIER + BOUNDARY

Own the knowledge

TEACH

Explain tokenization and parsing at beginner, intermediate, and senior depth.

REBUILD

Recreate smallest useful example from blank file without notes or AI.

RETRIEVE

Schedule recall for day 1, 7, 30, and 90.

Creative frontier lab

Try first without opening the solutions. The constraints invite invention; the reference gives one concrete direction, never the only valid answer.

Constraint inversion

Re-solve tokenization and parsing by removing the most convenient abstraction. ship the interaction with native elements and zero custom JavaScript.

CONSTRAINTKeep the same inputs, observable result, and failure evidence; change the means, not the contract.

ORIGINAL IDEATurn subtraction into a design tool: the missing abstraction should reveal which responsibility it used to hide.

Reveal frontier solution
  1. Freeze the contract as three fixtures: Create document with title, main, headings, list, link, and correctly labeled form control. / Introduce recoverable malformed nesting and compare source with DOM. / Remove label and identify lost accessible name.
  2. List every convenience used by the starter; remove the highest-level one while preserving Open index.html directly, then inspect DOM and Accessibility trees..
  3. Implement the smallest replacement using ship the interaction with native elements and zero custom JavaScript.
  4. Run all fixtures and compare raw evidence. Keep the simpler version unless the removed abstraction has a demonstrated benefit.
Representation x-ray

Build an explanation artifact for tokenization and parsing: compare source, parsed DOM, focus order, form payload, and accessibility tree.

CONSTRAINTA peer must be able to locate the first divergence without reading implementation code.

ORIGINAL IDEATreat the explanation itself as a product: make invisible transitions visible, replayable, and diffable.

Reveal frontier solution
  1. Create one row or timestamped event for each transition in: Create document with title, main, headings, list, link, and correctly labeled form control.
  2. For every row record input, representation, owner, operation, output, and tool evidence from Chromium Elements · Accessibility · Network.
  3. Replay Introduce recoverable malformed nesting and compare source with DOM.; highlight only changed rows.
  4. Replay Remove label and identify lost accessible name.; stop at the first divergent row and attach its recovery action.
Adversarial remix

Combine the boundary and failure into a new user-visible scenario for tokenization and parsing. make one document usable by keyboard, screen reader, print, and direct URL fragments.

CONSTRAINTDo not merely add more input. Invent a recovery interaction, alternate representation, or self-checking behavior.

ORIGINAL IDEAMake the system teach its own limits: the artifact should expose the invariant and offer a safe next action when it breaks.

Reveal frontier solution
  1. Combine these two pressures without changing them: Introduce recoverable malformed nesting and compare source with DOM. AND Remove label and identify lost accessible name.
  2. Name the invariant that must survive and the user-visible evidence when it cannot: DOM differs predictably from malformed source; landmark and names appear in accessibility tree; missing label is detectable.
  3. Implement this original direction: make one document usable by keyboard, screen reader, print, and direct URL fragments.
  4. Demonstrate baseline, combined failure, recovery, then baseline again; save the sequence as a regression fixture.

Capability frontier

Push tokenization and parsing until another layer becomes justified. Record one robust technique, one contextual trade-off, and one labeled hack or historical curiosity.

CORE · PRACTICAL · CONTEXTUAL · HACK · FRAGILE · HISTORICAL · GOLF

Boundary

HTML coordinates navigation and submission well, but complex local computation and synchronization quickly become awkward.

If this vanished tomorrow…

Return to meaningful text, links, forms, HTTP requests, and browser defaults.

Why next layer is earned

CSS is earned when correct semantics need deliberate layout, hierarchy, responsive behavior, and visual state.