Repository navigation
Conversation
Module Scorecard Check
|
| Presentation & Onboarding | Agent Integration | Credential Hygiene | Restricted-Environment Readiness | Engineering Quality | Overall |
|---|---|---|---|---|---|
| 6 / 17 | 15 / 25 | 20 / 20 | 7 / 20 | 7 / 10 | 60 / 100 |
Drilldown
Presentation & Onboarding — 6 / 17
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Configuration-mode examples | 12 | half | README shows launcher/coder_app, MCP+rules, pinned version, and downstream-sync examples with sensible defaults, but the key "bring-your-own binary" mode (install_auggie=false) is never shown as a README example — only described in the variable description/main.tf. |
| Visual preview | 5 | 0 | No image, GIF, or video anywhere in the README. |
Credential Hygiene — 20 / 20
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Secrets marked sensitive | 16 | full | augment_session_token is sensitive = true; README examples reference var.augment_session_token with no inline literal/placeholder secret values. |
| Non-hardcoded auth path | 4 | full | README's "Authentication" section documents auggie login as the non-hardcoded path and auggie token print to populate the sensitive variable, avoiding pasted keys. |
Restricted-Environment Readiness — 7 / 20
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Mirrorable artifact source | 5 | 0 | Install uses npm install -g @augmentcode/auggie[@version]; no variable overrides the npm registry/download URL — auggie_version only pins a version string, not a source URL. |
| Bring-your-own binary | 10 | half | install_auggie = false skips install and validates a pre-existing auggie on PATH (enforced in scripts/install.sh.tftpl), but this mode is only documented in the main.tf variable description, not demonstrated or called out in the README itself. |
| Egress transparency | 3 | 0 | No dedicated README section enumerating install/runtime endpoints (e.g., npm registry, Augment API); endpoints are not documented anywhere. |
| Runs without sudo | 2 | full | scripts/install.sh.tftpl never invokes sudo; npm install uses a user-writable $HOME/.npm-global prefix, confirmed by code inspection. |
Engineering Quality — 7 / 10
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Input quality | 6 | half | All variables have clear descriptions and sensible defaults (e.g., install_auggie=true, auggie_version=""), but no validation blocks are present anywhere in main.tf. |
| Test coverage | 4 | full | main.tftest.hcl covers defaults, workdir trimming, secret non-leakage, encoding of inputs, and scripts output ordering; main.test.ts provides extensive end-to-end container tests (install paths, MCP merging, rules, model settings, secret handling). |
Agent Integration — 15 / 25
| Criterion | Max | Score | Notes |
|---|---|---|---|
| AI governance | 10 | 0 | No mention of Coder AI Gateway or Agent Firewall anywhere in the README; v1 changelog explicitly removes prior AgentAPI integration without replacement governance story. |
| Dashboard entry point | 5 | full | README's "Standalone mode with a launcher app" example shows a full coder_app resource with open_in, icon, and launch command. |
| Session continuity | 5 | full | The launcher example explicitly runs auggie --continue to resume the previous conversation, documenting session continuity across relaunches. |
| Managed configuration | 5 | full | README documents managed MCP servers (mcp, mcp_files), rules (rules → ~/.augment/rules/coder-module.md), and model config (auggie_model → ~/.augment/settings.json). |
Overall — 60 / 100
Raw 55 / 92 → round(55 / 92 × 100) = 60
Tip
You can run this locally by telling your agent: "review this module against .github/scorecard/SCORECARD.md".
Scored against SCORECARD.md with solstice-1. Language-model scores are advisory.
|
Automated review requested by @matifali. Please sanity-check before acting on it.
SummaryThe agentapi module is replaced with Correctness
Leftovers
Findingsshould-fix S1 (blocking for me): changes to module-managed MCP servers never apply after the first start. should-fix S2: rules can't be cleared. nit N1: v0 leftovers on persistent homes. nit N2: jq is now required but not documented. nit N3: variable attribute order. nit N4: constraints are inconsistent with the series. nit N5: tests still import from the agentapi module. nit N6: MCP names with newlines break pairing. nit N7: the PR body says the npm install happens "now on every start", but v0 also reinstalled every start. Not a regression; just a wording fix. Tests
CI and reviews
Series consistencyIt follows the same layout as kiro-cli and claude-code (coder-utils, |
|
Re S1 (module-managed MCP servers never update after the first start): not changing this here. "Existing name wins" is intentional and matches the rest of the series and
Auggie does the same via a non-interactive Generated with Coder Agents. |
|
|
||
| write_rules() { | ||
| if [ -z "$${ARG_RULES}" ]; then | ||
| return |
There was a problem hiding this comment.
Could we remove ~/.augment/rules/coder-module.md when rules is empty? Auggie automatically loads this file, so clearing the template input leaves the previous rules active after a restart. Previously, clearing the input stopped passing --rules to the CLI.
There was a problem hiding this comment.
Fixed in e95795a. When rules is empty, the module now removes ~/.augment/rules/coder-module.md (a file the module owns), so clearing the input stops applying the previous rules on the next start. Added stale-rules-are-removed-when-rules-is-empty, which fails without the fix.
Generated with Coder Agents.
|
|
||
| mkdir -p "$(dirname "$${SETTINGS_FILE}")" | ||
| echo "$${existing}" | jq --arg model "$${ARG_AUGGIE_MODEL}" '.model = $model' > "$${SETTINGS_FILE}.tmp" | ||
| mv "$${SETTINGS_FILE}.tmp" "$${SETTINGS_FILE}" |
There was a problem hiding this comment.
@35C4n0r I think we could preserve the existing file permissions when updating the model here. Replacing settings.json with the temporary file makes a previously owner-only file readable by other users who can access its directory. I reproduced this with another user in the same group, who could then read the dummy MCP credentials used in the test.
There was a problem hiding this comment.
Fixed in 795ddb4. setup_model now rewrites settings.json in place instead of replacing it with a temp file, so the existing mode and owner are kept (and a symlinked file is written through). Added model-keeps-settings-permissions, which seeds a 0600 file and checks it is still 0600; it fails without the fix.
Generated with Coder Agents.
…n rules is empty Auggie auto-loads ~/.augment/rules/*.md, so a cleared rules input kept applying the previous rules after restart.
…ns when setting the model Replacing the file with a temp file reset an owner-only settings.json to the default mode, exposing MCP credentials to other users.
|
/review |
DevelopmentCats
left a comment
DevelopmentCats
left a comment
There was a problem hiding this comment.
Full review of 795ddb4 with the repo's AGENTS.md conventions and the sibling coder-utils based modules as context.
Verified locally
terraform validateandterraform fmt -checkpass on the moduleprettier --checkpasses on all changed filesshellcheckon the rendered install script (through style severity): only the expected SC2016 infos from the${ARG_*}template placeholders- The new
main.tftest.hclexpectations match thecoder-utilscontract:scriptsoutput order is pre_install, install, post_install, and thecoder-labs-auggie-*naming falls out ofmodule_directory - Module data moved to
~/.coder-modules/coder-labs/auggieper the AGENTS.md layout,#!/usr/bin/env bashshebangs everywhere (including the mock), andset -uordering is compliant (.bashrcis sourced beforeset -euo pipefail) augment_session_tokenis sensitive, gated on non-empty viacount, and never rendered into scripts; the test suite asserts the no-leak property including the base64 encoding- Version bumped to 1.0.0 in all README examples; breaking changes documented in the WARNING block;
coder-utilspinned to 0.0.1 likecoder/claude-code
Not verified here
terraform testfails in this environment withtemplate ID is missing: CODER_WORKSPACE_TEMPLATE_ID is requiredfrom the coder provider data source, butcoder/claude-codeonmainfails identically, so this is environmental and not caused by this PRbun test main.test.tsneeds Docker, which this workspace lacks, so the container tests the PR reports are unverified by me
Findings
Nothing blocking. Three suggestions and one nit inline. Two smaller observations:
require_jqexits 1 whenjqis missing andmcporauggie_modelis set. A loud failure is defensible since the alternative is silently dropping config, but it is a new hard dependency compared to v0, which wrote JSON with plain bash. Fine if intended.- The PR description reports 12 passing bun tests but the suite now contains 14; consider refreshing the testing section.
- Upgraded workspaces keep the stale v0 files
~/.augment/coder_mcp.jsonand~/.augment/user_mcp.json. Nothing reads them anymore, so this is harmless, but a cleanuprm -fnext to thewrite_ruleslogic would keep things tidy. Not worth blocking on.
The rest of the migration is clean: MCP handling matches the real CLI behavior (existing names win, the exit 3 path is handled), model writing preserves file permissions and leaves JSONC settings alone, and the README examples and launcher app are correct.
Generated by the Coder agent in response to /review.
| variable "mcp" { | ||
| type = string | ||
| description = "MCP configuration as a JSON string for the auggie cli, check https://docs.augmentcode.com/cli/integrations#mcp-integrations" | ||
| description = "MCP servers as a JSON string ({\"mcpServers\": {...}}), added to ~/.augment/settings.json with `auggie mcp add-json`. Servers already configured win on duplicate names. See https://docs.augmentcode.com/cli/integrations#mcp-integrations" | ||
| default = "" | ||
| } |
There was a problem hiding this comment.
An invalid mcp string is only caught at runtime today: it is skipped with a warning after the workspace has already started. A plan-time validation surfaces the typo when the template is applied instead:
| variable "mcp" { | |
| type = string | |
| description = "MCP configuration as a JSON string for the auggie cli, check https://docs.augmentcode.com/cli/integrations#mcp-integrations" | |
| description = "MCP servers as a JSON string ({\"mcpServers\": {...}}), added to ~/.augment/settings.json with `auggie mcp add-json`. Servers already configured win on duplicate names. See https://docs.augmentcode.com/cli/integrations#mcp-integrations" | |
| default = "" | |
| } | |
| variable "mcp" { | |
| type = string | |
| description = "MCP servers as a JSON string ({\"mcpServers\": {...}}), added to ~/.augment/settings.json with `auggie mcp add-json`. Servers already configured win on duplicate names. See https://docs.augmentcode.com/cli/integrations#mcp-integrations" | |
| default = "" | |
| validation { | |
| condition = var.mcp == "" || can(jsondecode(var.mcp)) | |
| error_message = "mcp must be a valid JSON string, e.g. {\"mcpServers\": {...}}." | |
| } | |
| } |
Optional, since the runtime warning already prevents a broken install; this just moves the failure earlier.
| for profile in "$HOME/.profile" "$HOME/.bash_profile" "$HOME/.bashrc" "$HOME/.zprofile" "$HOME/.zshrc"; do | ||
| if [ -f "$${profile}" ]; then | ||
| if ! grep -q "$${path_dir}" "$${profile}" 2> /dev/null; then | ||
| echo "export PATH=\"\$PATH:$${path_dir}\"" >> "$${profile}" |
There was a problem hiding this comment.
This appends the npm-global bin dir after $PATH, but v0's install.sh prepended it (export PATH="$HOME/.npm-global/bin:$PATH"). With appending, an image that ships an older auggie earlier on PATH keeps shadowing the npm-installed one in login shells, which also undercuts the writability rationale for the user prefix in the decision log. fish_add_path below already prepends by default, so this would also make the two consistent:
| echo "export PATH=\"\$PATH:$${path_dir}\"" >> "$${profile}" | |
| echo "export PATH=\"$${path_dir}:\$PATH\"" >> "$${profile}" |
| printf "Auggie CLI version: %s\n" "$${AUGGIE_VERSION}" | ||
|
|
||
| if [ -n "$${CODER_SCRIPT_BIN_DIR:-}" ] && [ ! -e "$${CODER_SCRIPT_BIN_DIR}/auggie" ]; then | ||
| ln -s "$${AUGGIE_BIN}" "$${CODER_SCRIPT_BIN_DIR}/auggie" |
There was a problem hiding this comment.
Tiny edge case: if a dangling symlink already exists at this path, [ ! -e ] is true but ln -s fails with File exists and no symlink gets created (the failure does not abort because the function runs inside an if). ln -sfn covers both the fresh and dangling cases:
| ln -s "$${AUGGIE_BIN}" "$${CODER_SCRIPT_BIN_DIR}/auggie" | |
| ln -sfn "$${AUGGIE_BIN}" "$${CODER_SCRIPT_BIN_DIR}/auggie" |
| else | ||
| echo "Warning: Failed to add MCP server '$${name}' ($${source_desc}): $${output}" | ||
| fi | ||
| done < <(echo "$${mcp_json}" | jq -r '.mcpServers // {} | to_entries[] | .key, (.value | @json)') |
There was a problem hiding this comment.
Small footgun: if someone passes a bare server map without the mcpServers wrapper (a shape some other CLIs accept), this silently adds zero servers. jq -r '(.mcpServers // .) | to_entries[] | ...' would also accept that shape; the trade-off is that a non-server object would then produce add-json warnings instead of a silent no-op. Fine to keep as is since the README documents the mcpServers format, but worth a thought.
Closes REG-82
Refactor
coder-labs/auggieto install and configure Auggie CLI only, matching the claude-code, codex, copilot, cursor-cli, opencode, and kiro-cli migrations. Changes beyond the AgentAPI removal are limited to what the current Auggie docs require.order,group,install_agentapi,agentapi_version,ai_prompt,report_tasks,cli_app,web_app_display_name,cli_app_display_name,continue_previous_conversation,interaction_mode), thetask_app_idoutput, the start script, and the task-reportingcoderMCP server.folderis renamed toworkdir, optional, and created if missing.npm install -g @augmentcode/auggie(into~/.npm-global, as before), now on every start wheninstall_auggie = true; withfalse,auggiemust be onPATH. Removed theauggie_versionvalidation; it is base64-encoded into the script.mcpandmcp_filesare added to~/.augment/settings.jsonwithauggie mcp add-jsonat install time instead of--mcp-configflags at launch. Existing names are kept with a per-name warning, matching the CLI without--replace.rulesare written to the user rules folder (~/.augment/rules/coder-module.md) instead of a--rulesflag.auggie_modelsets themodelkey in~/.augment/settings.jsoninstead of--model.augment_session_tokenis now sensitive andAUGMENT_SESSION_AUTHis only created when set; it was previously also rendered into the start script.scriptsoutput; README reduced to examples and doc links; rewrite tests. Bump to1.0.0.Important
Model calls and MCP connections were not verified: they need an Augment account, which was not available.
Decision log
npm install -g @augmentcode/auggie, Node 20+~/.npm-globalprefix (keepsauggie upgradewritable); keep thenode/npmcheck and.bashrcsourcingAUGMENT_SESSION_AUTHenv var with theauggie token printJSONcoder_envonly, gated on non-empty, never in scripts or logs~/.augment/settings.jsonmcpServers;auggie mcp add-jsonworks without login, preserves JSONC, and non-interactively exits 3 with "already exists" unless--replaceauggie mcp add-json; existing servers win with a warning; no jq edit of MCP--rulesflag, or auto-loaded~/.augment/rules/*.md(always applied)rulesto~/.augment/rules/coder-module.md;auggie rules listshows it--model//model; settings schema hasmodel, and with it setauggielogsIgnoring --model=<value>when no models are availableauggie_modelby writingmodelwith jq; a JSONC settings file is left unchanged with a warning--allow-indexing)auggie_versionandinteraction_modehad themTesting
terraform test(6 passed),bun test main.test.ts(12 passed, container-based),terraform fmt,terraform validate, prettier, shellcheck on the rendered script (all severities)codercom/enterprise-node:latestvia the module's rendered pre_install and install scripts:CODER_SCRIPT_BIN_DIRsymlink and profile PATH entries createdtheme, usersharedserver wins with a warning);everythingandmcp_filesserver added;auggie mcp listshows all threeauggie rules listshows~/.augment/rules/coder-module.md; themodelkey is picked up (Ignoring --model=test-model because no model options are availablewithout login)AUGMENT_SESSION_AUTH(dummy token gets a 401 from Augment), absent from scripts, logs, and~/.augment; rotated value appliedCODER_SCRIPT_BIN_DIR, downstreamcoder exp syncscript ran after pre_install, install, post_install; seeded settings kept, usersharedserver kept with a warning,everythingandmcp_filesservers added,modelset, rules written,AUGMENT_SESSION_AUTHsetauggie --help,auggie mcp --help, and duplicateauggie mcp add-jsonbehavior (exit 3, existing entry and JSONC kept) on 0.36.0Generated with Coder Agents.