fix(toml): escape control characters so generated command files parse#3341
Merged
mnriem merged 2 commits intoJul 8, 2026
Merged
Conversation
both toml renderers (TomlIntegration._render_toml_string for gemini/tabnine and CommandRegistrar.render_toml_command for extension/preset commands) wrote control characters raw into a multiline or basic string. toml forbids literal control chars (U+0000-U+001F except tab/newline, and U+007F) in every string form, and a bare CR that is not part of a CRLF pair, so a description or body containing one produced a .toml file that fails to parse. route any value with such a character to a fully-escaped basic string that emits the leftover control chars as \uXXXX. added regression tests that round-trip through tomllib.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes TOML command/config rendering so generated .toml files remain parseable when descriptions or bodies contain illegal control characters (including a bare \r), addressing issue #3340.
Changes:
- Add detection of TOML-illegal control characters and route affected values through a fully-escaped basic-string renderer.
- Update both TOML rendering paths (
TomlIntegrationandCommandRegistrar) to avoid emitting forbidden literal control characters. - Add regression tests that round-trip rendered TOML through
tomllibto ensure parseability and value preservation.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/integrations/base.py |
Adds control-character detection + a fully-escaping basic-string fallback in TomlIntegration to guarantee valid TOML output. |
src/specify_cli/agents.py |
Ensures CommandRegistrar.render_toml_command() escapes illegal control characters in bodies (and updates basic-string escaping to emit \\uXXXX). |
tests/integrations/test_integration_base_toml.py |
Adds a regression test that verifies _render_toml_string() outputs TOML that parses and round-trips control characters. |
tests/test_extensions.py |
Adds a regression test ensuring extension/preset TOML command rendering escapes control characters so tomllib can parse the output. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 4/4 changed files
- Comments generated: 1
- Review effort level: Low
mnriem
requested changes
Jul 6, 2026
mnriem
left a comment
mnriem
left a comment
Collaborator
There was a problem hiding this comment.
Please address Copilot feedback
the control-char detection and basic-string escaping added for both toml renderers were copy-pasted into agents.py and integrations/base.py. move the two functions into specify_cli/_toml_string.py and have both renderers delegate to it, so the escaping rules can't drift apart later. no behavior change; both renderers now reference the same implementation.
Contributor
Author
|
pushed 0bf4bdc which moves both helpers into a single specify_cli/_toml_string.py and has both renderers delegate to it, so there's only one implementation to keep in sync now. no behavior change, all toml suites still pass. |
This was referenced Jul 7, 2026
mnriem
approved these changes
Jul 8, 2026
Collaborator
|
Thank you! |
kanfil
added a commit
to tikalk/agentic-sdlc-spec-kit
that referenced
this pull request
Jul 8, 2026
Upstream commits merged (7): - 94c7ec2 escape TOML control chars so generated command files parse (github#3341) - 5a901a6 fall through to grep/sed when python3 is a broken stub (github#3312) - 295eb22 port update-agent-context to Python (github#3387) - ba1ce36 remove Cursor from CLI check list in README (github#3184) - 13d2cca document missing CLI flags and integrations (github#3182) - 0da969d add LLM Wiki extension to community catalog (github#3361) - a7b4391 release 0.12.8, begin 0.12.9.dev0 (github#3410) New file: src/specify_cli/_toml_string.py Conflicts: pyproject.toml (trivial version); base.py (import block — combined fork accent() with upstream _toml_string imports). Assisted-by: OpenCode (model: glm-5.2, autonomous)
5 tasks
marcelsafin
added a commit
to marcelsafin/spec-kit
that referenced
this pull request
Jul 10, 2026
…derer YAML forbids C0 control characters (except tab and newline) and DEL in every scalar form, and a bare CR acts as a line break inside a block scalar. _render_yaml wrote the body verbatim into a |2 literal block scalar, so such bodies produced recipes the YAML parser rejects. Detect block-scalar-unsafe characters and fall back to an escaped double-quoted scalar via yaml.safe_dump, mirroring the TOML renderer's fallback strategy from github#3341. Fixes github#3382 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #3340
both toml renderers wrote control characters into their output raw, so a description or command body containing one produced a
.tomlfile that fails to parse. toml forbids literal control characters (U+0000-U+001F except tab and newline, plus U+007F) in every string form, and a bare CR that is not part of a CRLF pair.two code paths were affected:
TomlIntegration._render_toml_stringinsrc/specify_cli/integrations/base.py(gemini, tabnine)CommandRegistrar.render_toml_command/_render_basic_toml_stringinsrc/specify_cli/agents.py(extension/preset commands)the fix adds a small guard: any value that contains an illegal control char is routed to a fully-escaped basic string that emits the leftover control chars as \uXXXX, so the output always parses. the normal triple-quote multiline paths are unchanged for values without control chars.
both changes have a regression test that round-trips the rendered output through tomllib (test_toml_string_escapes_control_characters in the base toml suite, test_render_toml_command_escapes_control_characters in test_extensions). i confirmed each test fails on the pre-fix code by stashing the source and re-running.