Skip to content

EPIC: T27F — Native generative UI and visual application engine #1648

Open
Open
EPIC: T27F — Native generative UI and visual application engine#1648

Description

@dharmarooms
dharmarooms
opened on Oct 16, 2026

Vision update: from generated UI to a declarative system-to-experience pipeline

T27F should investigate a deeper architectural change: describe a system and its state/behavior once, then derive its computation, interactive UI, diagrams and visualizations from that description. Do not reduce the goal to generating React code or styling existing screens.

Proposed conceptual pipeline (hypothesis, not yet implemented)

  1. Intent / Queen orchestration — interpret the request, identify missing constraints, and select capabilities.
  2. Versioned system/world description — typed entities, properties, relationships, units, constraints, initial/boundary conditions, time, provenance and uncertainty.
  3. Domain model + computation — explicit rules/equations or a selected solver; optional learned models are evaluated separately and never assumed correct.
  4. Renderer-neutral UI IR — trusted widget/component IDs, data bindings, layout, actions, theme, accessibility, provenance and capability declarations.
  5. T27F renderer — start with the current React/TypeScript frontend; render panels, graphs, AST-supported diagrams, trajectories or 3D views as justified.
  6. Verification loop — compare computed results with independent references/observations; expose error, uncertainty, solver settings and unsupported assumptions.

The system description, simulation state and UI layout must remain separate but connected through explicit typed contracts. One state source should drive both the solver and the visualization.

Research notes and primary resources

These projects solve different layers; do not treat them as direct interchangeable competitors or claim an absolute winner without a task-specific benchmark.

Required research matrix

For every candidate record the date checked, category, current version/status, source links, license (verify from LICENSE, otherwise mark unverified), schema/validation, streaming/partial updates, state and actions, React integration, renderer portability, accessibility, security/trust boundaries, persistence/theme/fallback, maintenance signals, integration effort, and known gaps. Distinguish facts from recommendations. Include at least one runnable same-scenario spike where feasible and publish scripts/raw results. A2UI schema, AG-UI interaction protocol, MCP Apps tool UI, and full app generators are different categories.

Revised MVP sequence

  1. Audit actual Trinity frontend, dashboard/widgets, .t27 grammar/compiler, AST/IR, tests/oracles, data APIs and relevant physics/numeric specs. Cite paths/symbols and mark implemented/partial/proposed/unknown.
  2. Compare at least two real .t27 candidates; do not preselect GF16.
  3. Choose one research workspace pilot based on parser/IR coverage, evidence, tests, usefulness of visual representation and implementation effort.
  4. Define the smallest versioned UI IR and trusted widget manifest, or document evidence for reusing an existing format.
  5. Generate one workspace from natural language using actual project data: source/spec, AST-derived diagram only if supported, test/benchmark status, provenance/uncertainty, preview/accept, regenerate, theme, save/load and reset.
  6. Test malformed output, unknown widget/action, missing data, denied permissions, widget/tool failure, unsupported syntax, interrupted generation and stale saved state.
  7. Benchmark against the best-fit existing approach and current React baseline on identical tasks; publish environment, versions, scripts, raw output and limitations.
  8. Only after the declarative pipeline is validated, add the first physics experiment: two-/few-body Newtonian mechanics with documented initial conditions, unit checks, an independent reference, conservation/error diagnostics, adjustable parameters and trajectory/graphs. Expand to harder domains only after validation.

Scientific and engineering guardrails

  • Physical constants alone do not encode a complete simulator. Laws/equations, initial and boundary conditions, units/material parameters, numerical methods and validation are also required.
  • A plausible generated image/animation is not evidence that its dynamics are correct.
  • Do not claim a single model can emulate the exact universe, or that no training is needed. Exact atom-level universe simulation is not an MVP goal.
  • Prefer a hybrid architecture: agent interprets and routes; a typed representation describes the system; a specialized solver computes; T27F renders; independent references validate.
  • No arbitrary model-generated JavaScript/eval in the trusted UI path. Validate every plan against an allowlist and explicit permissions.
  • No broad frontend rewrite and no claim of performance superiority without measurements.

Agent skills required

Prepare three focused, reusable skills in the repository's established skill convention, following the Agent Skills format:

  1. t27f-architecture — contracts between intent, world/system model, computation, UI IR and renderer; repository audit; safety and test gates.
  2. generative-ui-research — source-backed competitor research, version/status/license verification, taxonomy and reproducible matrix/benchmark.
  3. world-model-simulation — system-state schema, solver selection, first mechanics experiment, units, numerical validation, uncertainty and limits.

Each skill must include YAML frontmatter (name, description), procedural steps, links to this epic/MVP and primary sources, failure modes, and acceptance criteria. Keep each SKILL.md focused and move long references into adjacent reference files. Validate the skill format and check the repository's own-language-first policy and existing skills loader/catalog before adding a second canonical copy. Do not claim these skills exist until their files are committed and discoverable.

Deliverables / acceptance

  • Dated competitor matrix with verified primary sources.
  • Repository audit and at least two candidate-spec comparison.
  • Architecture decision record explaining reuse/adapt/build.
  • Versioned UI IR/widget manifest proposal and trust-boundary design.
  • Runnable end-to-end workspace with tests and reproducible baseline.
  • A small validated physics demonstration only after the UI/system-description chain works.
  • Three validated skills, with exact paths and discoverability verified.
  • Final report clearly separating evidence, implementation, assumptions and unresolved questions.

This issue is a product/research direction, not a claim that the architecture, simulator, or skills are already implemented.

Extension proposal: shared world model for generative UI + visual/game runtime

Treat a game/interactive world as a first-class T27F use case, not as a separate unrelated product. The key hypothesis is that a stable object/world model can be presented through multiple replaceable views: research dashboard, 2D map, 3D scene, game-like world, schematic or abstract visualization. Changing a view/theme must not silently mutate the underlying world state.

Separate contracts

  • World / domain state: stable entity IDs, typed components/properties, relationships, transforms, units, ownership and lifecycle.
  • Behavior / simulation: explicit systems, rules, event handlers, physics/collision and time-step configuration. Visual styling must not define physical behavior.
  • Presentation profile: geometry/asset mapping, materials/colors, lighting, camera, typography, layout and animation style. Profiles are versioned and replaceable.
  • Interaction contract: allowed user actions, controls, selection, manipulation, accessibility and permission checks.
  • Renderer adapter: initially the existing web/React path for UI and a proven 2D/3D renderer where needed; do not assume a new engine must be written.
  • Persistence / snapshots: save world state and presentation separately; support undo/version history and restore-default for the presentation without resetting simulation state.
  • Validation / determinism: tests for state transitions, replayable events where feasible, and separation of authoritative state from visual interpolation.

User-facing capability

A user can ask “show this system as a minimal 2D diagram”, “switch to a low-poly game style”, or “return to default”. The system generates or selects a validated presentation profile and applies it to the same stable world model. Provide preview/accept, undo, reset-to-default, and versioned saved variants. Do not regenerate the domain model just to change colors or shape.

Engine research — do not decide by analogy alone

Smallest proof

  1. Create one stable test world with a few typed entities and known relationships.
  2. Render it in two presentation profiles (for example, schematic and low-poly 3D) while preserving entity IDs and state.
  3. Change theme/materials/layout and return to the default profile; verify that simulation state is unchanged.
  4. Add one explicit behavior/interaction (move/select/trigger) and test that both views operate on the same state.
  5. Save/load world state and presentation independently; test invalid profile fields and unknown component/action rejection.
  6. Measure renderer performance and interaction latency before expanding to full game-engine scope.

This is an architectural hypothesis and research scope, not a claim that T27F is already a game engine. Full game functionality (physics, collision, animation, assets, networking, audio, editor and platform builds) must be scoped separately.

Child workstream: Evolution Runtime

Track bounded self-adaptation as a separate vertical slice: #1655 — T27F Evolution Runtime: prototype bounded self-adaptation, hot-swap, rollback and live visualization. The generative-UI MVP validates declarative model → trusted renderer; #1655 separately validates safe runtime changes and live event visualization. #1654 remains a separate Queen/Bee task-intake reliability workstream. No prototype or runtime dispatch is claimed yet.

Activity

  1. dharmarooms commented on Oct 16, 2026

    @dharmarooms
    dharmarooms
    on Oct 16, 2026 – with ChatGPT Codex Connector
    Author

    Product & token economy workstream — subscription + usage + courses

    Status: proposed design to validate; not implemented, not a final pricing decision.

    Product hypothesis

    Use a hybrid model rather than forcing every participant into one payment model:

    • Subscription: platform/Queen/lab access, feature limits, and an explicit monthly included usage allowance.
    • Usage credits (“tokens”): metered GPU/compute and other paid services; show price/estimated consumption before execution and a usage ledger after.
    • Courses: separately priced one-time purchases or modules. A subscription includes course access only when the selected plan explicitly says so. Purchased courses remain accessible without duplicate charges.
    • Providers: contribute verified, accepted compute and receive credits/rewards based on completed work and an agreed rate card. Compute value can offset a subscription only if the provider plan explicitly permits it; if earned value is below the plan price, the provider pays the difference or chooses a lower tier.
    • Researchers/course authors: define separate, transparent reward/revenue-share rules; do not conflate contribution rewards with user spending credits.

    Required economic model

    Build a spreadsheet/calculator with explicit, editable assumptions:

    1. Provider GPU type, available hours, utilization, energy/hosting cost, failure rate, verification overhead, and payout/credit rate.
    2. Workload class (inference/training/scientific compute), measured resource consumption, retries, queue time, and independent result verification.
    3. Subscription price, included allowance, variable cost, support/payment fees, gross margin, and provider credit cap.
    4. Course price, creator share, platform share, refund/revocation policy, and repeat-access behavior.
    5. Credit issuance/redemption, expiry/refunds, fraud/abuse controls, and a stress test for provider supply/demand.

    Proof of Compute / trust boundaries

    Do not reward merely having a GPU online. Define per-workload evidence: signed job receipt, input/job identity, resource telemetry, accepted output or verifier result, duplicate/replay protection, and dispute/failure handling. Not every workload has a cheap cryptographic proof of correct computation; document the assurance level per workload. Start with an internal ledger/accounting unit; do not promise a tradable token or 1:1 cash value before legal, security, and unit-economics review.

    MVP experiment and acceptance

    • Compare subscription-only, usage-only, and hybrid offers with a clear target segment and hypotheses; validate willingness to pay via interviews/landing-page or small paid pilot, not opinions alone.
    • Publish sample customer/provider journeys and a unit-economics table with conservative/base/optimistic scenarios.
    • Show one concrete worked example: provider contribution → verified credits → subscription offset; separately show user subscription → included allowance → overage and course purchase.
    • Define limits, top-ups, price disclosure, refunds, course entitlement persistence, and anti-abuse rules.
    • Label all prices and token conversion rates as assumptions until measured and approved.
    • Do not ship pricing/token contracts until product, finance, and legal review are complete.

    Decision to test

    Default hypothesis: subscription for platform access + included allowance; usage credits for metered compute; separate course purchases unless explicitly bundled. Providers can offset a capped subscription amount with verified compute credits. Validate against simpler alternatives before locking this in.

    Related work: #1652 (T27F MVP/benchmark) and #1654/#1645 (Queen/Bee reliability and runtime performance).

  2. dharmarooms commented on Oct 16, 2026

    @dharmarooms
    dharmarooms
    on Oct 16, 2026 – with ChatGPT Codex Connector
    Author

    Product integration: T27-native specifications in the existing map

    This belongs in this T27F epic. Preserve the existing visual design and map/Arena experience; change the underlying logic and representation so the project can progressively move from TypeScript to the native T27 language instead of building another parallel system.

    Product behavior

    • The existing map remains the main navigation surface.
    • Each cell/node maps to a real project component and its specification.
    • When a user zooms into a cell, reveal the corresponding specification as a visual, navigable representation of the program: components, relationships, dependencies, interfaces, and relevant task/status links.
    • Let users move from the map to the specification, and from specification elements to their implementation/tasks, without losing context.
    • Show the lifecycle of related work (proposed, specified, in progress, validation/review, accepted, blocked) where those states are available from the source of truth.

    T27 as the native representation

    Treat T27F as a path toward representing and working with the system in T27, not merely rendering TypeScript code in a diagram.

    • Audit the current TypeScript logic and identify the boundaries where it can be represented in T27.
    • Define a gradual migration path: existing TypeScript remains interoperable during transition; do not require a risky all-at-once rewrite.
    • Specifications/IR should become the stable bridge between the visual map, T27-native logic, implementation, and validation.
    • Keep traceability from each visual element to its source specification, code, tests, and GitHub task.
    • Prefer reuse of existing T27F architecture and existing map components; no redesign and no duplicate subsystem.

    Suggested delivery sequence

    1. Audit the current T27F epic and existing map/specification flows; identify what already exists.
    2. Define the spec/IR schema and mapping from map cell → specification → program components/dependencies → source/tests/issues.
    3. Prototype the zoom-in experience for one representative specification, with real traceable data.
    4. Add status and validation links so contributors can follow their chosen direction and see progress.
    5. Demonstrate one small T27-native path end-to-end while preserving compatibility with existing TypeScript.

    Acceptance criteria

    • Zooming into a map cell opens its linked specification view in the existing UI.
    • The view presents a navigable program structure and its relationships, not only a static image.
    • Each displayed element links back to the relevant specification/source/tests/task when available.
    • Task status is visible from the authoritative task source; no invented progress.
    • At least one pilot demonstrates the TypeScript-to-T27 transition through a defined spec/IR boundary.
    • Existing design and map navigation remain intact; implementation reuses the current T27F architecture.
    • Document current capabilities versus planned capabilities clearly; do not imply full TypeScript replacement before a working migration path is proven.

  3. dharmarooms commented on Oct 16, 2026

    @dharmarooms
    dharmarooms
    on Oct 16, 2026 – with ChatGPT Codex Connector
    Author

    Product/MVP extension: shareable specification-honeycomb widget + T27-native onboarding

    This extends the existing T27F epic. Do not redesign the current honeycomb/map UI or create a parallel product. Reuse the current map, cells/specifications, engineering courses, and existing issue/status infrastructure.

    Core interaction

    • Zooming into a honeycomb cell reveals its specification as the focused, enlarged cell.
    • Keep the surrounding neighboring cells visible around the edges, so the user understands that the specification belongs to a larger connected system (the focused cell is enlarged, not detached from the map).
    • The focused view should expose the specification's structure and links/dependencies between components, plus relevant implementation status, tests, source/spec and related GitHub issues where available.
    • Keep world/specification data and UI presentation separate; zoom/theme/presentation must not mutate the underlying specification or system state.

    Viral distribution

    • Make the specification view a self-contained, shareable widget/view with a stable URL and, if compatible with the current architecture, an embeddable preview/card for blogs, documentation, GitHub issues and social posts.
    • A shared link should open the same specification focused at the correct zoom/selection, while retaining enough surrounding cells to communicate the larger map.
    • Show clear provenance/version and current status; distinguish live data from snapshots. Do not imply a task is complete unless linked evidence supports that status.
    • Provide a clear path from the shared view to explore the full map, contribute to the spec, or follow its issue/course.

    Onboarding as a course

    • Use the existing engineering-course mechanism to create an onboarding course, rather than inventing a separate onboarding subsystem.
    • Teach the user by doing: explore a cell/specification, understand dependencies, follow a task/status, propose a change or support an existing direction, and verify the result.
    • Connect course progression to real actions and existing contribution/reward rules; do not promise token payouts or GPU marketplace capabilities that are not implemented.

    T27-native full-stack direction

    • The MVP should be a vertical slice from specification/data model through rendering, interaction, status and sharing, with T27 as the intended native language/runtime path and no new JavaScript/TypeScript logic added by default.
    • First audit what T27 compiler/runtime, renderer bindings, persistence and browser integration can actually support. Explicitly document any unavoidable compatibility boundary; do not claim a complete JS-free browser stack until a runnable build proves it.
    • Migrate incrementally behind stable contracts; preserve the current map design and existing working functionality. Avoid a big-bang rewrite.

    MVP acceptance criteria

    1. Select one real existing specification/cell and render it enlarged within the surrounding honeycomb.
    2. Demonstrate that neighboring cells remain visible and linked relationships are navigable.
    3. Display real spec/source/test/issue status with explicit unknown/unavailable states rather than invented progress.
    4. Open the same focused view from a stable shareable URL; demonstrate an embeddable preview if supported by the existing stack.
    5. Create the onboarding course using the existing course mechanism and link its steps to real product actions.
    6. Build and test the vertical slice using the existing T27-first policy; record precisely which layers run in T27 and any remaining JS/TypeScript dependency.
    7. Verify state/specification remains unchanged when only zoom, presentation or theme changes.

    This is a scoped extension of T27F, not a claim that the widget, embed flow, onboarding course or fully JS-free frontend already exists.

  4. dharmarooms commented on Oct 16, 2026

    @dharmarooms
    dharmarooms
    on Oct 16, 2026 – with ChatGPT Codex Connector
    Author

    MVP extension: the specification honeycomb becomes a shareable learning unit / open university

    This extends T27F and must reuse the existing honeycomb map, specification model, engineering-course system and task/status infrastructure. Preserve the current visual design; do not create a separate course platform or parallel map.

    Product concept

    Each honeycomb cell is both a system specification and a learning entry point. Zooming into a cell opens its enlarged, explorable specification while neighboring cells remain visible at the edges, preserving context and showing how this unit connects to the larger system. The same view should be shareable via a stable deep link and, where the current architecture permits, an embeddable widget/card suitable for X posts, blogs, documentation and GitHub issues.

    A course attached to each specification

    • Every specification can reference an onboarding/learning course explaining what the spec represents, why it exists, its inputs/outputs, dependencies, examples, tests and contribution opportunities.
    • Reuse the existing engineering course structure and its 27-episode/lesson pattern where applicable; do not hard-code 27 lessons for every topic if the existing course model allows shorter or longer tracks.
    • Support shared curriculum modules for common theory and specification-specific lessons for unique concepts. Model this as reusable lesson/module references plus an ordered course path, not duplicated lesson content.
    • Let a learner move from concept → interactive spec/map → related task or test → contribution → verified result. Show course progress and task status only from actual persisted data.
    • Course completion, contribution credit and any TRI/reward mechanism are separate concepts; award credit only under existing explicit rules and do not imply token value or GPU payouts that are not implemented.

    Viral sharing / growth loop

    1. A public link opens one specification-focused honeycomb view with neighboring cells and version/status context.
    2. The page explains the spec and offers “Learn this cell”, “Explore dependencies”, “Support this direction” and “Contribute” actions, using existing product flows where possible.
    3. The linked course teaches the shared theory and spec-specific concepts, then points to a real task/test.
    4. A learner can share the cell/course link with attribution and the current spec/course version; embeds must be sandboxed/allowlisted and must not execute arbitrary generated code.

    T27-native full-stack goal

    Treat this as the smallest end-to-end vertical slice for T27F: specification/course references → typed data contracts → focused honeycomb UI → progress/status → deep link/share/optional embed. T27 is the intended native implementation path. First audit actual compiler/runtime/browser integration and document what can run in T27 today; do not claim a fully JavaScript/TypeScript-free browser stack until a runnable, tested build proves it. Avoid adding new JS/TS business logic and migrate incrementally behind stable contracts.

    MVP acceptance criteria

    1. Pick one real existing specification and render its focused cell with neighboring honeycomb cells visible.
    2. Attach an existing-format course to it, containing both reusable shared-theory modules and at least one spec-specific lesson.
    3. Link course steps to real spec elements, dependencies, tests or issues; show real progress and honest unavailable states.
    4. Share the focused view through a stable URL; demonstrate embed only if supported and security-reviewed.
    5. Verify that switching focus/theme/presentation does not mutate specification or course content.
    6. Run and document the complete vertical slice, including exact T27 versus JS/TS boundaries, tests and limitations.

    This is an MVP/product requirement in the existing epic, not a claim that the university, share widget, course links or full T27-native frontend are already implemented.

  5. dharmarooms commented on Oct 16, 2026

    @dharmarooms
    dharmarooms
    on Oct 16, 2026 – with ChatGPT Codex Connector
    Author

    Follow-up directions split into two concrete workstreams:

    1. Visualization research and frontend foundation: research graph exploration, scientific visualization, scrollytelling, focus+context, shareable deep links and safe embeds; then prototype a reusable visualization host inside the existing frontend, preserving square honeycomb geometry and reusing current map/course/status infrastructure.
    2. T27U — Trinity University: define a specialist University Scientist agent within the existing 27-team/Queen architecture. It owns spec-to-course mapping, reusable shared theory modules, spec-specific lessons and a provenance-backed research/teaching loop. First audit the actual agent registry and “golden circle” definition; don't claim a live agent until registered and tested.

    Research starting points: Microsoft Research — Data Stories using Animated Transitions, ScrollyVis, Cytoscape.js, Sigma.js, Flourish scrollytelling examples. These are references to evaluate, not a pre-decided dependency choice.

  6. gHashTag
    added 5 commits that reference this issue on Oct 17, 2026

    spec(t27f): topic-to-value engine and /t27f skill, phase 1 slice (#9952)

    7181e6b

    spec(t27f): renderer-neutral UI plan IR and trusted widget catalog (C…

    cdfdc11

    docs(t27f): T27F is the visual spec of .t27, not the topic-to-value e…

    efe3c94

    spec(t27f): add the spec -> graph IR exporter in t27 (Closes #10162) (#…

    d486d5f

    feat(website): /t27f/graph/ draws the T27F graph view of a .t27 spec (#…

    aedbc7a
  7. github-actions
    added a commit that references this issue on Oct 18, 2026

    Deploy aedbc7a

    5c6b28c
  8. gHashTag
    added a commit that references this issue on Oct 18, 2026

    feat(website): /t27f/graph/ in the site's own style (dark, Outfit and…

    eb77dc2
  9. github-actions
    added a commit that references this issue on Oct 18, 2026

    Deploy eb77dc2

    fc090c3
  10. gHashTag
    added 3 commits that reference this issue on Oct 18, 2026

    fix(website): Graph layer sets no state in its effect body

    1d1690e

    test(website): queen-specs-chat pins the frame route with the view

    82246c2

    feat(website): a Graph layer in the Spec Explorer, drawn by the T27F …

    ed677e2
  11. github-actions
    added a commit that references this issue on Oct 18, 2026

    Deploy ed677e2

    4d86ab0

  12. gHashTag commented on Oct 19, 2026

    @gHashTag
    gHashTag
    on Oct 19, 2026
    Owner

    Part of mega epic gHashTag/t27#11038, the one map of every T27F and gen-UI track.

    🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions