Phase 4 — Cross-corpus content beyond tutorials (blogs, learning journeys, trials, videos, samples)

[jung-thomas]

Phase 4 — Cross-corpus content beyond tutorials

Sub-issue of #381. From a comment by Tom on the parent issue (Nora's suggestion):

"Would be awesome to also add blog posts, learning journeys, basic trials and other relevant upskilling information."

Phase 4 is the bigger ambition: extend the knowledge graph beyond just developers.sap.com/tutorials/* to capture the broader SAP upskilling ecosystem.

What this delivers

Today the graph contains:

• ~1500 tutorials (from sap-tutorials/* GitHub org)
• ~80-150 AI-extracted concepts
• 8 ontology predicates linking them

Phase 4 expands the corpus to include other SAP upskilling content types as first-class graph nodes, not just metadata on tutorial nodes:

• Blog posts — community.sap.com authored content (potentially via existing fetch tooling like sap-devs MCP get_recent_news)
• Learning Journeys — learning.sap.com structured paths (the sap-devs MCP already exposes search_learning_journeys)
• Free trials / discovery missions — discovery-center.cloud.sap (existing search_discovery integration)
• Videos — SAP Developers YouTube + Tech Bytes (existing search_videos integration)
• API docs — SAP Business Accelerator Hub (api.sap.com) — when they're a meaningful prerequisite or follow-up to a tutorial
• Code samples — SAP-samples GitHub org (existing get_samples integration)

Each becomes a node type with appropriate IRI prefix:

kg:tutorial/<slug>            (existing)
kg:concept/<slug>             (existing)
kg:mission/<slug>             (existing)
kg:product/<slug>             (existing)

kg:blog-post/<slug>            (NEW)
kg:learning-journey/<slug>    (NEW)
kg:trial/<slug>                (NEW)
kg:discovery-mission/<slug>   (NEW)
kg:video/<id>                  (NEW)
kg:api-doc/<slug>              (NEW)
kg:sample/<repo>/<path>       (NEW)

And new edge types where they make sense:

• kg:hasReferenceMaterial — tutorial → blog/video/api-doc
• kg:partOfJourney — tutorial → learning-journey (already a structural relationship)
• kg:trialFor — concept → trial (e.g. "concept cap-deployment has trial cap-trial-environment")
• kg:officialReference — concept → api-doc

Why this matters

Phase 1's whatToLearnNext recommendation is bounded by the tutorial corpus. A user finishing cap-getting-started might genuinely benefit more from:

• Reading the "What's New in CAP 10" blog post (event-driven, time-sensitive)
• Following the "BTP Solution Architect" learning journey (broader curriculum)
• Spinning up a free trial to try the concept hands-on

…than from another tutorial. Phase 4 makes those paths first-class.

Showcase value: also strong. "The KG knows everything SAP-developer-ecosystem and recommends the right next thing regardless of content type" is a more ambitious story than "the KG indexes our 1500 tutorials."

What's already in place

• sap-devs MCP server has tools for fetching all of these content types: get_recent_news, search_learning_journeys, search_discovery, search_videos, search_resources, get_samples. Each returns structured JSON.
• Phase 1's extractor pattern (per-content-item, content-hash-keyed cache, LLM-based concept extraction) generalizes — it doesn't actually care that the content is a tutorial.
• Phase 1's projection layer (kg-projection.js) and SPARQL client are content-type-agnostic.
• HANA KGE has no opinion on what node types exist; adding new IRI prefixes is purely a projection-side concern.

Scope

This phase is substantially bigger than 1, 2, or 3 because each new content source has its own:

• Fetch + cache pipeline
• Frontmatter / metadata schema
• Update cadence (some are publish-once, others are time-sensitive)
• Authority / freshness expectation (a learning-journey landing page is canonical; a blog post can be deleted)
• Linking semantics (some link to tutorials structurally; others are orthogonal references)

So Phase 4 is best treated as a portfolio of sub-phases:

4.1 — Learning Journeys (highest value, lowest scope)

Already structured curriculum data; mostly a content-fetch + 4 new edge types (kg:partOfJourney, kg:journeyPrerequisite, etc.). One sub-PR:

• New CDS entity LearningJourneys
• Nightly fetch via sap-devs MCP
• Edges from tutorial → journey and journey → journey
• Surfaced in the sidebar's whatToLearnNext re-ranker (and Phase 3's explore page if both ship)

4.2 — Blog posts (moderate value, time-sensitive)

Blog posts go stale; the graph needs a freshness/relevance signal. New edge type kg:hasReferenceMaterial from concept → blog post; only emit edges where the blog post's publishedAt is within 18 months OR the post has been explicitly marked canonical. Time-decay re-ranker logic in JS layer.

4.3 — Discovery missions + free trials (high marketing value)

Lifecycle-bound: a free trial is short-lived. New edge type kg:trialFor; surfaced specifically in the "what to learn next" rail when a concept has an associated free trial available now.

4.4 — Videos (high engagement value)

YouTube content has decent metadata via the existing search_videos MCP integration. New edge type kg:videoReferenceFor; expose in concept pages (Phase 3) and optionally inline in the sidebar.

4.5 — API docs (high authority, low frequency)

Authoritative reference material. Concepts like cap-cqn should link out to https://cap.cloud.sap/docs/cds/cql. Mostly a hand-curated initial seed (50-100 mappings) + an LLM-pass to suggest more.

4.6 — Code samples (high engagement, tricky linking)

SAP-samples repo content. Less obvious extraction target — code samples don't teach concepts the way tutorials do; they embody them. Probably a different edge type (kg:embodiesConcept) with semantics distinct from kg:teaches.

Out of scope (probably ever)

• Cross-organization content (e.g. third-party SAP-related blogs on Medium / dev.to). Maintenance burden too high.
• Real-time content (live SAP TV). Cache TTL doesn't match.
• Paywalled SAP Press / SAP Learning Hub content. Linking to gated material is poor UX.

Design questions to resolve before starting

• Order of sub-phases: 4.1 (learning journeys) is probably the highest leverage. 4.2 (blogs) is the most time-sensitive — needs decay logic that the simpler items don't. 4.3-4.5 are roughly equal in value. 4.6 is the trickiest.
• MCP-tool fetch caching: the sap-devs MCP is already wired into the build pipeline. Reuse vs new dedicated fetcher? Probably reuse — it's the source-of-truth for these endpoints.
• Stale-content garbage collection: blog posts get deleted; learning journeys get reorganized. Need a regular GC pass that prunes graph nodes whose source content is gone.
• Concept linking: does the same extractConcepts LLM call work on a blog post? Probably yes for the "teaches" relation, but the prompt may need tuning ("this is a blog post about X concept, not a tutorial that teaches X").
• UI surface: Phase 1's sidebar shows tutorials. Phase 4 wants to show non-tutorial recommendations alongside. Probably add a 5th section "Other resources" that aggregates blog/video/journey/trial — OR keep the 4 tutorial-focused sections and add a separate "Reference materials" panel. Spec decision.

Acceptance criteria

This is a research issue today, not an implementation issue. Acceptance:

• Sub-phase order decided + each sub-phase has its own implementation issue
• Decision on UI surface integration (5th section vs separate panel)
• Decision on MCP-tool reuse vs new fetcher
• Decision on stale-content GC mechanism
• First sub-phase (probably 4.1) has a brainstorm + spec + plan documents in docs/superpowers/

Why this is genuinely Phase 4 (not Phase 2 or 3)

• Phase 2 (Joule path-gen) only needs the existing tutorial graph
• Phase 3 (explore page + concept pages) only needs the existing tutorial graph
• Phase 4 fundamentally changes what the graph IS — it's no longer a "tutorial graph", it's an "SAP-developer-content graph". That's a bigger conceptual shift than expanding any single surface.

Land Phases 2 + 3 first — they validate the existing infrastructure under real load. Phase 4 is the next-generation move.

Refs #381