TOOTRA.TKS

TKS v7.4 Compiler/VM Pipeline (Rust)

This document defines the implementation pipeline and crate layout for the v7.4 compiler/VM.

Pipeline Stages

  1. Lexing

    • Input: .tks source
    • Output: token stream with spans (file/line/col)
    • Unicode + ASCII supported (Unicode normalized to token kinds; ASCII default output).
    • Ordinal literal patterns (omega + n, omega * n, omega ^ n) and ket/bra tokens.

  2. Parsing

    • Input: tokens
    • Output: AST (v7.3 AST shape) with source locations
    • Parser supports:
      • v7.2 core grammar
      • v7.3 extensions (effects, transfinite, quantum)
      • v6.1 canonical aliases (expr^k, <<...>>)
    • Precedence notes: handler nesting right-assoc, ordinal exponent right-assoc.

  3. Desugaring

    • Normalize aliases to canonical AST forms:
      • expr^k -> Noetic(k, expr)
      • <k1:k2:... [blocked]>(expr) -> Fractal([k1,k2,...], expr)
      • Unicode fractal tokens -> ASCII fractal node
      • Ordinal literal tokens -> OrdAdd/OrdMul/OrdExp forms
    • Normalize effect rows (remove duplicates, order-insensitive canonicalization).

  4. Name Resolution + Module Loading

    • Build module graph (imports, exports, signatures)
    • Load .tksi interfaces for dependencies
    • Resolve identifiers, handler names, effect names, extern symbols

  5. Type + Effect Inference

    • Hindley-Milner inference with row-polymorphic effects
    • Validate effect declarations and handler clauses
    • Assign Type + EffectRow to each AST node
    • Enforce effect row subtyping and effect boundary safety (imports/exports)
    • Apply ordinal constraints and bounded quantification rules

  6. Lowering to ANF IR

    • Convert to effect-aware ANF IR
    • Make perform/handle/resume explicit and ready for CPS lowering
    • Lower transfinite loops and quantum ops to IR terms

  7. Bytecode Generation

    • Map IR to v7.3 bytecode instruction set
    • Emit .tkso with metadata (module name, imports, exports, constants)
    • Encode handler metadata, ordinal tables, quantum constants, FFI linkage info

  8. Linking / Execution

    • tksvm loads .tkso and resolves imports
    • Optional module linking step for single-file executables
    • Run on the v7.3 VM state model*