feat(ai-powerups): admin extensions written by the assistant [EXPERIMENTAL / WIP] by adrians5j · Pull Request #5742 · webiny/webiny-js

adrians5j · 2026-09-19T18:20:07Z

Experimental prototype. Not a merge candidate.
Opened for review of the direction and the findings, not the code. It still contains probe files
that ship inside a published package, it has no tests, and several gaps listed at the bottom are
unresolved. Do not merge as-is.

What this explores

Whether a user can change their admin UI by asking for it, instead of writing an extension, pushing
it, and waiting for a deploy. Everything here is an extension Webiny already supports in code; the
question is whether the assistant can write one and the admin can run it.

Two kinds are working end to end: custom field renderers and sidebar menu items.

What actually works, verified in a browser

Typing "build me a menuBuilder renderer" into the command palette, approving the tool call, and
reloading gives a working drag-to-reorder field renderer bound to a real form. The renderer appears
in the CMS field editor's Appearance tab, can be picked per field, and renders in a real entry form.
A red text input and a live character counter were both built this way and confirmed rendering.

Asking for a sidebar item under Settings produces one that appears and navigates.

No file was touched, nothing committed, no rebuild.

How it fits together

Storage. A private CMS model, wbyAdminComponent, with a kind discriminator. Field renderers
store TSX; menus store JSON.

Writing. createFieldRenderer and createMenu are AiSdkToolDefinitions, both behind the
approval gate.

Running. app-admin gains features/generatedComponents/: bundle the stored source with
esbuild-wasm in the browser, evaluate it with new Function against injected dependencies, and
register it. ai-powerups fetches on boot and registers both kinds.

Guidance. Each extension kind is authored once in skills/shared/extensions/ and rendered two
ways: the MCP skill a developer gets, and the tool description the assistant gets. They differ only
by execution environment, since a developer imports freely and has a compiler while the assistant
has neither.

The finding worth your time

Generation quality was the only thing that ever failed. The mechanism worked first time, every time.

Six bugs across four attempts, and every one was a plausible guess at an API that nothing rejected
until it was on screen. <Text text={...} /> when Text takes children, while Button and
FormComponentLabel do take text. Tree given tree/render, the prop names of the library it
wraps but does not re-expose. sort omitted, so a drag-to-reorder list silently alphabetized itself
the moment someone typed a label. field.validation iterated as if it were a list of rules.

Three rounds of adding the missing fact to a prose contract fixed each bug and none of the class. So
the contract is generated from the real .d.ts files now, with anything declared inside
@types/react filtered out, which takes InputProps from 309 properties to the 17 a caller
actually chooses. The next renderer generated after that change got every earlier bug right on the
first attempt with no correction round.

One sample is not a measurement. But the failures it removes were the entire correction loop.

Known gaps

Probe files ship in app-admin. src/probe/ is exploration and is mounted at
/_/form-model-demo. It has to come out, or move to the project's extensions/, before any real
merge. The findings comment in MenuBuilderFieldRenderer.tsx is the design rationale and should
move somewhere it stays useful.
No tests. None, for code that evaluates stored source in a signed-in admin session.
Nothing can be turned off. The model has an enabled field and nothing reads it. A bad
renderer can only be removed by editing the CMS entry directly.
CmsFormModelBuilder is a singleton that reads the renderer list once. A renderer registered
after the first entry form resolves is dropped silently, because applyFieldProps skips
builder.renderer() on a miss rather than erroring.
No promotion between environments. Something built in staging has no route to production.
This is the question I would expect a customer to ask first.
The approval gate assumes the approver reads the source, which does not fit the audience the
idea is aimed at.
~/admin/... imports in ai-powerups' admin/Extension.tsx were not rewritten to relative
paths on build for a newly added directory, while neighbouring identical imports were. Worked
around with a relative import; the cause is unexplained.

Trying it

Ask the assistant in the command palette for a field renderer or a menu item, approve the call, and
reload. packages/app-admin/src/probe/GeneratedRendererProbePage.tsx at /_/form-model-demo has a
field asking for a menuBuilder renderer that nobody has written yet.

Regenerate the guidance and type reference with yarn generate-skills.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SnN5e1pnoEBwkKshTrETs6

…the admin already ships? Hand-written menu-builder renderer for an object list field, using nothing the generated code would have to bring itself. Not shipping code; it exists to answer one question before any generator is designed. It answers yes, and by more than expected: `admin-ui` exports `Tree`, which already does drag, drop, nesting and collapse, so even the drag library named in the tier-1 plan is unnecessary. The reorder itself is `field.moveItem`, which the form model already exposes, so the renderer supplies presentation and calls the model. Findings are recorded at the bottom of the file, including the one that matters most for the generator: naming the libraries is not enough. Four lines of the first draft failed to compile, all plausible guesses at an API rather than nonsense. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The second half of the probe: can a string that never went through the repo's compiler become a working field renderer in the admin? - rendererRuntime.ts — the injection set (React, admin-ui, observer, the two createFieldRenderer helpers) and loadRenderer(), which evaluates a bundled IIFE and calls createRenderer(runtime). Same mechanism ComponentEditorPresenter already uses for remote components. - bundleRenderer.ts — strips imports, wraps the source in that factory, and transpiles JSX with esbuild-wasm in the browser. Nothing resolves at load time, so a generated renderer cannot reach for a dependency it was not given. - GENERATED_SOURCE.ts — the cold-generated renderer as a string, which is how it will arrive: out of a model, into the CMS, never through a build. - GeneratedRendererProbePage.tsx — builds a real FormModel, bundles, loads, and renders through FormView. Mounted in the existing FormModelDemo page. - generated/MenuBuilderFieldRenderer.tsx — what the cold agent produced, kept for the diff against the hand-written one. Everything compiles. It is NOT yet confirmed on screen: this checkout's webiny.config.tsx currently fails to render in Node on an SVG import, which reproduces with the whole tree stashed and so is not caused by any of this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…hat only runtime finds The cold-generated menu builder now renders in the admin, bound to a real FormModel, with no build step for the generated code itself. Adding items, typing into them and reading back form.getData() all work. Getting there turned up two bugs that compile cleanly and only show up on screen with data in them: - Custom node data was passed to Tree flat and read back flat. Only the read side is flat. NodeDto.data goes in nested and WithDefaultNodeData<T> comes back merged. - No sort prop was passed, so Tree fell through to react-dnd-treeview's default and alphabetized the rows by label. A menu builder whose whole feature is drag-to-reorder silently reordered itself as soon as a label was typed. Nothing throws, and the form data stays correct. That second one is the interesting result: a compile-and-retry loop would never have caught it. Findings are written up at the bottom of MenuBuilderFieldRenderer.tsx. Also fixes the bundler hanging forever. The wasmURL pinned 0.28.1 while the installed package is 0.28.2, and a version mismatch throws inside the worker where nothing rejects the promise. It now takes the version from the package, and memoizes the init so a second mount can't call initialize twice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

… then runs Closes the loop end to end. You type "build me a menuBuilder renderer" into the command palette, approve the tool call, reload, and the field renders through code nobody committed. API: - A private CMS model, wbyAdminComponent, holding the source. Its own model rather than a corner of the website builder's, since the two share nothing but the word "component". - createFieldRenderer, an AiSdkToolDefinition. Needs approval, and for a stronger reason than the other write tools: it stores code that later runs in an admin user's browser, so the approval screen is the code review. - listAdminComponents(kind) for the admin to read back. Newest wins, one per name, because asking for a fix writes a new entry rather than editing the old one and the stale version would otherwise take the name back. Admin: - app-admin gains a real home for the loader that was sitting in probe/, plus GeneratedFieldRenderer, which bundles one stored source and registers it under the name fields ask for. Registration waits for the bundle so a half-loaded renderer never claims a name. - ai-powerups fetches on boot and registers whatever is stored. The interesting part is the tool description. Told in prose to pass sort={false} and to nest node data, the model wrote tree={...} and render={...} instead of nodes/renderer: the prop names of @minoru/react-dnd-treeview, which admin-ui's Tree wraps but does not re-expose. JSX takes unknown props without complaint, so it compiled, loaded, registered, and drew an empty box. Giving it the literal signature instead of a description fixed it on the next attempt. Which is the finding worth keeping: the contract has to name props rather than describe them, and a wrapper over a well-known library is where a model will confidently reach for the wrong API. Notes are at the bottom of MenuBuilderFieldRenderer.tsx. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

A generated renderer needs two registrations to be usable and only one was there. AdminConfig.Form.FieldRenderer resolves a name to a component when a form renders; CmsFieldRenderer is the catalogue the field editor's Appearance tab offers. With only the first, a renderer worked but nothing could ever select it, because that dropdown is the only way to set a field's renderer name. createFieldRenderer now also takes label, fieldType and appliesTo, so canUse can keep a menu builder off a number field, and the renderer list has a short title with a real description under it rather than the same sentence twice. Also fixes the bundler: it used esbuild's jsx loader while every label on the input said TSX, so the first type annotation the model wrote was a syntax error. Types are erased, never checked. Verified in a real entry form: a red text input and a live character counter that turns orange past 20 characters, both picked from the Appearance tab. Three more wrong-API bugs turned up on the way, and by now the pattern matters more than the bugs. <Text text={...} /> when Text takes children and only Button and FormComponentLabel take text. Iterating field.validation looking for rules when it is a { isValid, message } result. Each was a plausible guess that React or the form model accepted silently. Adding each one to the prose contract fixed that bug and not the class, and the contract is now most of a hand-written type definition. Generating it from the real types is the way out. Notes are at the bottom of MenuBuilderFieldRenderer.tsx. Known gap worth a look in review: CmsFormModelBuilder is a singleton that reads the renderer list once, so a renderer registered after the first entry form resolves it is left out of the map, and applyFieldProps then skips builder.renderer() silently. The boot fetch wins in practice because reaching an entry form takes a navigation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The prose contract was wrong three times in a row, and every time the right answer was already sitting in a .d.ts. So it is generated now. A script reads the built types of admin-ui and app-admin with ts-morph and emits the component props, the size unions, the Tree node shapes and the field view model. The hand-written half keeps only what no type can state: that sort={false} is effectively mandatory, that a renderer must not hold field data in local state, that a guessed prop name fails silently rather than throwing. What makes it readable is dropping anything declared inside @types/react. InputProps resolves to 309 properties, 292 of them inherited DOM attributes. The 17 that remain are the ones a caller actually chooses. children is the one deliberate exception, since whether a component takes children is exactly what went wrong before. The next renderer generated after the change, a drag-to-reorder checklist asked for cold, got every earlier bug right first time and needed no correction round: nodes and renderer rather than the wrapped library's tree and render, sort={false}, string ids, custom data nested going in and flat coming back, aria-label on IconButton, and both <Text>children</Text> and <FormComponentLabel text={...} /> correct in the same file. That last pair is the distinction it had generalised away twice. One sample is not a measurement, and the generator cannot help with anything that is a judgement rather than a shape. But the failures it removes were the entire correction loop up to now. Regenerate with: yarn workspace @webiny/ai-powerups generate-renderer-contract Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnN5e1pnoEBwkKshTrETs6

@dev

…and for the assistant A developer writing a field renderer by hand and the assistant writing one from a prompt need the same knowledge. Until now they had two copies: a 772-line form-model skill served over MCP, and a contract I hand-wrote for the tool description, which was wrong three times. Each extension kind is now authored once in skills/shared/extensions/ with @shared, @dev and @admin sections, and scripts/generateExtensionContracts.mjs renders both targets: - skills/user-skills/admin/extensions/<kind>/SKILL.md, shipped in @webiny/mcp by its prepublishOnly copy. Gets @shared + @dev: imports, file paths, AdminConfig.Form.FieldRenderer and CmsFieldRenderer registration. - rendererContract.generated.ts in ai-powerups. Gets @admin first, since the sandbox rules override anything the shared half implies, then @shared. The sections differ only by execution environment, which is the whole reason they cannot be one undifferentiated file: a developer imports freely and has a compiler, the assistant has neither. Sharing the file verbatim would teach the assistant to write import statements. Both bake a copy at build time from the one repo source, so there is no runtime coupling between two published packages, and a project gets the guidance that shipped with its own runtime. Also closes the drift I introduced last commit. The injected component list lived in bundleRenderer.ts and again in the generator's hardcoded array. bundleRenderer now exports INJECTED_COMPONENTS as the single definition, the wrapper builds its destructure from it, and the generator reads it from source. Verified by adding a component to the list and watching it appear in the contract. A component can no longer be described as available without being in scope, or vice versa. One command regenerates everything: yarn generate-skills now runs the existing skills generator and this one. Regenerating also produces a 43-file whitespace-and-frontmatter diff across skills/user-skills/generated/, which is pre-existing drift between the committed tree and what the generator emits today. Left out of this commit so it stays reviewable. Worth a look on its own, since it tags admin/configs with context: webiny-api. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnN5e1pnoEBwkKshTrETs6

…xtension kind Menus prove the shared-source structure generalises, and they are the cheap case on purpose: a menu is configuration, not code. Nothing is bundled, nothing is evaluated, and GeneratedMenus maps stored rows onto the same AdminConfig.Menu an extension would use, so the sidebar cannot tell one of these from one written by hand. That is the argument for sorting extension kinds by whether they genuinely need arbitrary code. This one needed none of the machinery the field renderer does: no esbuild, no new Function, no contract about which components exist, and no way to fail at render time beyond pointing at a path that does not resolve. The stored JSON is still parsed defensively, because a hand-edited CMS entry is the one route that bypasses the tool's validation and a malformed row must not take the sidebar down. Authored in skills/shared/extensions/menu.md like the field renderer, so the MCP skill and the tool description come from one source. Its @admin section is short, which is the shape you want: the sandbox has little to say when there is no sandbox. Verified end to end: asked for a Reports item under Settings, approved, and it appears in the sidebar and navigates. One thing to know if you touch this file: `~/admin/...` in ai-powerups' admin/Extension.tsx does not get rewritten to a relative path on build for a newly added directory, and the admin app then fails to resolve it. The neighbouring imports use the same alias and work, so this looks like stale build state rather than a rule. Switched to a relative import; worth a look if it bites someone else. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnN5e1pnoEBwkKshTrETs6

coderabbitai · 2026-09-19T18:20:13Z

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-09-19T18:20:36Z

🚓 Slop Cop

⚠️ 5 thing(s) worth a look before merging.

Coherent, well-documented experimental PR that matches its stated (self-disclosed WIP) scope, but ships probe/demo code and an unsandboxed new Function() eval path into a published package, both already acknowledged by the author as gaps to fix before merging.

🚨 Should this be in the PR?

🔴 High — PR explicitly self-flagged as experimental/WIP, not a merge candidate

The PR title and description clearly state this is an experimental prototype ('Do not merge as-is') containing probe files, no tests, and multiple known unresolved gaps (no disable mechanism, singleton race condition, no environment promotion, approval-gate trust assumption). This is intentional and disclosed, but flagging it here per the review's mandate to catch content the author would want to know about before merging — the author already knows and says so.

🟠 Medium — Probe/demo code shipped inside the published app-admin package

packages/app-admin/src/probe/GeneratedRendererProbePage.tsx and packages/app-admin/src/probe/MenuBuilderFieldRenderer.tsx (278 lines combined) are exploration/demo code wired into packages/app-admin/src/features/formModel/demo/FormModelDemo.tsx and mounted at a real route (/_/form-model-demo). The PR description itself acknowledges these must come out before any real merge, but they are present in this diff and increase the published package's footprint with non-production code, including a long stream-of-consciousness comment block in MenuBuilderFieldRenderer.tsx documenting the author's experimentation process (not doc comments).

🟡 Low — Unexplained workaround left in code with unresolved root cause

The PR description notes '~/admin/... imports in ai-powerups' admin/Extension.tsx were not rewritten to relative paths on build ... Worked around with a relative import; the cause is unexplained' (see the relative ./presentation/GeneratedMenus/GeneratedMenus.js import in packages/ai-powerups/src/admin/Extension.tsx next to ~/admin/... imports). This is a known, disclosed gap rather than an accident, but it's an unexplained build inconsistency worth resolving before merge.

📏 Code-style rule checks

🟠 Medium — new Function() usage for evaluating generated code

packages/app-admin/src/features/generatedComponents/rendererRuntime.ts uses new Function(...) to evaluate bundled, stored source in the browser with no sandboxing, explicitly flagged by the author as a security question still to be settled. While not a formal listed style rule, this is a significant architectural risk worth calling out (also self-acknowledged in the PR gaps list).

🟡 Low — Multi-line block comment not using '/ ... /' consistently ending with period

In packages/ai-powerups/src/api/features/AdminComponents/menuContract.generated.ts and rendererContract.generated.ts, the generated file header comments use /* ... */ correctly, but several inline comments across new files (e.g. one-line /* Anything unrecognised widens to "both" ... */ in AdminComponentsRepository.ts) are multi-line using /* */ which is fine, but some single-line // comments added in new files (e.g. in bundleRenderer.ts, GeneratedFieldRenderers.tsx) generally follow the period convention correctly — no clear violation found beyond minor inconsistency, low confidence.

_{Automated, non-blocking heads-up from an LLM. It can be wrong — use your judgment. Regenerates on every push.}

Before this, "create a char count renderer and put it on Article's title" ended with the renderer written and the user still doing the last two steps by hand: reload, then pick it under Appearance and save the model. setFieldRenderer closes that gap, so the whole sentence completes. Kept separate from createFieldRenderer rather than folded into it. Writing code and changing a content model are two different decisions and the approval screen should show them as two, so someone can take the renderer and still refuse to have it applied to that particular field. It also lets an existing renderer be reused elsewhere without regenerating. It checks what it can. A renderer's stored fieldType and appliesTo are matched against the target field, because a renderer reading field.items on a single-value field renders nothing rather than failing. A name that is not a generated renderer is allowed through, since that is how you put a field back to a built-in, but the response says it was not checked. A code-defined model gets a specific message rather than a raw error, because "Cannot update model defined via code" is not something the user can fix from the admin and the next question is always why. Verified with the one-shot ask: it ran createFieldRenderer, then proposed setFieldRenderer as a second approval, and the field renders "0 characters" in a real entry form. Also fences the generated type reference. yarn format was stripping the leading indentation out of the markdown, which flattened the nesting and made every regenerate-then-format cycle produce a 200-line diff. Fenced blocks are left alone, and the cycle is now idempotent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

adrians5j and others added 8 commits September 18, 2026 15:35

Navigation Menu

feat(ai-powerups): admin extensions written by the assistant [EXPERIMENTAL / WIP] - #5742

feat(ai-powerups): admin extensions written by the assistant [EXPERIMENTAL / WIP]#5742
adrians5j wants to merge 9 commits into
nextfrom
claude/ai-admin-component-probe

adrians5j commented Sep 19, 2026

Uh oh!

coderabbitai Bot commented Sep 19, 2026

Draft PR not reviewed

Uh oh!

github-actions Bot commented Sep 19, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

adrians5j commented Sep 19, 2026

What this explores

What actually works, verified in a browser

How it fits together

The finding worth your time

Known gaps

Trying it

Uh oh!

coderabbitai Bot commented Sep 19, 2026

Draft PR not reviewed

Uh oh!

github-actions Bot commented Sep 19, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant