Skip to content

Fill in Phase 1 rollout-note placeholders after 48h soak #448

Closed

Description

@jung-thomas

Phase 1 rollout-note placeholder fill-in (post-soak)

Self-reminder for Tom. Phase 1 of the knowledge graph (#381) shipped via PR #442. The rollout note docs/superpowers/done/2026-06-19-knowledge-graph-phase1-shipped.md was committed with 9 <!-- FILL IN POST-SOAK --> placeholders. They get filled after a 48h DEV soak with KNOWLEDGE_GRAPH_ENABLED=true.

Soak window

  • Flag flipped: 2026-06-19 17:27:10 -04:00 (= 21:27:10 UTC)
    • Method: cf set-env tutorials-srv KNOWLEDGE_GRAPH_ENABLED true && cf restart tutorials-srv
    • Caveat: This is cf set-env, not mtaext. Next MTA redeploy will silently drop the flag back to false unless deploy/dev.mtaext is also patched. See feedback_cf_set_env_drops_on_redeploy.
  • Soak ends: 2026-06-21 21:27 UTC (~48h later)
    • Sunday consolidate-concepts-job (47 3 * * 0) runs Sun 03:47 UTC = ~28h in
    • Two nightly extract-concepts-job runs (13 2 * * *) at Sat 02:13 UTC + Sun 02:13 UTC

Pre-soak baseline (capture BEFORE filling)

Before this issue closes, snapshot the BEFORE-state for the post-soak delta:

-- Concept tables row counts
SELECT TABLE_NAME, RECORD_COUNT, DISK_SIZE
  FROM "PUBLIC"."M_TABLES"
 WHERE TABLE_NAME LIKE 'COM_SAP_DEVELOPERS_IMS_CONCEPT%'
    OR TABLE_NAME = 'COM_SAP_DEVELOPERS_IMS_GRAPHMETADATA'
    OR TABLE_NAME = 'COM_SAP_DEVELOPERS_IMS_TUTORIALCONCEPTLINKS'
    OR TABLE_NAME = 'COM_SAP_DEVELOPERS_IMS_CONCEPTEDGES';

-- HDI total disk
SELECT SUM(DISK_SIZE) AS bytes_used
  FROM "PUBLIC"."M_TABLES"
 WHERE SCHEMA_NAME = (SELECT CURRENT_SCHEMA FROM DUMMY);

What needs filling

The 9 sections in docs/superpowers/done/2026-06-19-knowledge-graph-phase1-shipped.md:

  1. Concept-registry counts (status breakdown)

    SELECT status, COUNT(*) FROM "COM_SAP_DEVELOPERS_IMS_CONCEPTS" GROUP BY status;
  2. Graph projection metrics — read GraphMetadata singleton (UUID 00000000-0000-0000-0000-000000000001)

    SELECT lastBuiltAt, lastBuildDurationMs,
           conceptCount, edgeCount,
           teachesCount, requiresCount, relatedToCount,
           extendsCount, partOfCount, taggedWithCount,
           aboutProductCount, inCategoryCount, coCompletedWithCount
      FROM "COM_SAP_DEVELOPERS_IMS_GRAPHMETADATA"
     WHERE ID = '00000000-0000-0000-0000-000000000001';
  3. Manual UX review of 10 tutorial OPs — qualitative notes on sidebar quality (no SQL — eyeballed). Pick 10 representative tutorials from different missions; check that the 4 sidebar sections (key concepts, related, prerequisites, what to learn next) render sensibly.

  4. Telemetry baseline — events since flip

    SELECT eventName, COUNT(*) AS n
      FROM "COM_SAP_DEVELOPERS_IMS_UIEVENT"
     WHERE eventName LIKE 'kg.%'
       AND createdAt >= '2026-06-19T21:27:10Z'
     GROUP BY eventName
     ORDER BY n DESC;
  5. Cron health — confirm extract-concepts-job and consolidate-concepts-job ran without errors

    SELECT jobName, status, startedAt, durationMs, errorMessage
      FROM "COM_SAP_DEVELOPERS_IMS_JOBEXECUTIONLOG"
     WHERE jobName IN ('extract-concepts-job', 'consolidate-concepts-job')
       AND startedAt >= '2026-06-19T21:27:10Z'
     ORDER BY startedAt DESC;
  6. HTTP health — smoke-test 200/304 on a few /graph/neighborhood?slug=... paths via the approuter (authenticated). Manually browse 3-5 tutorials and DevTools-network-inspect for 200 + non-empty response.

  7. HANA usage delta — re-run the pre-soak baseline queries above and compute the delta.

  8. Follow-up issues opened — list any new issues filed during soak (this one #TBD will be cited; Research: move runtime-tunable env vars to a DB-backed admin UI #444 already exists).

  9. Lessons captured — anything that surprised you during the 48h that's worth a memory or CLAUDE.md note. E.g. cron-timing variance, unexpected predicate counts, sidebar latency observations.

After filling

Open a small PR that updates docs/superpowers/done/2026-06-19-knowledge-graph-phase1-shipped.md and merges to main. Close this issue when the rollout note has no <!-- FILL IN POST-SOAK --> markers remaining.

Refs #381, #442

Activity

  1. jung-thomas commented on Jun 19, 2026

    @jung-thomas
    ContributorAuthor

    BEFORE-baseline captured (2026-06-19 21:54 UTC, ~27min after flip)

    COM_SAP_DEVELOPERS_IMS_CONCEPTS               0 rows  1440 bytes
    COM_SAP_DEVELOPERS_IMS_CONCEPTEDGES           0 rows  1440 bytes
    COM_SAP_DEVELOPERS_IMS_GRAPHMETADATA          0 rows  1440 bytes
    COM_SAP_DEVELOPERS_IMS_TUTORIALCONCEPTLINKS   0 rows  1440 bytes
    
    HDI total: 98 tables, 12,958,051 rows, 1,126,220,754 bytes (~1.05 GB)
    

    GraphMetadata singleton hasn't been created yet — it's lazily inserted by the consolidator's first run. That's also expected.

    No KG cron firings yet — verified via:

    SELECT COUNT(*) FROM COM_SAP_DEVELOPERS_IMS_PIPELINELOG
     WHERE PIPELINETYPE = 'SCHEDULED_JOB'
       AND (METADATA LIKE '%extractConcepts%' OR METADATA LIKE '%consolidateConcepts%');
    -- 0

    Section 5 SQL correction

    The original section-5 query referenced COM_SAP_DEVELOPERS_IMS_JOBEXECUTIONLOG — that table doesn't exist. The actual schema is PipelineLog with pipelineType='SCHEDULED_JOB' and the job name in METADATA JSON ({jobName: 'extractConcepts'}). Replace section-5 SQL with:

    SELECT
        JSON_VALUE(METADATA, '$.jobName') AS job_name,
        STATUS, STARTEDAT, FINISHEDAT, DURATIONMS,
        SUMMARY, ERRORDETAILS
      FROM "COM_SAP_DEVELOPERS_IMS_PIPELINELOG"
     WHERE PIPELINETYPE = 'SCHEDULED_JOB'
       AND (METADATA LIKE '%extractConcepts%' OR METADATA LIKE '%consolidateConcepts%')
       AND STARTEDAT >= '2026-06-19T21:27:10Z'
     ORDER BY STARTEDAT DESC;

    Cron lock-names confirmed in srv/jobs/scheduler.js:188-199:

    • extractConcepts — daily at 02:13 UTC
    • consolidateConcepts — weekly Sun at 03:47 UTC
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions