RSS4.0 — Personalized Social Media Digests

Stop doomscrolling. Get everything you follow, in one digest, in minutes a day.

RSS4.0 is a news-feed aggregator for social media. You give it a plain list of the people and publications you follow — on X (Twitter), Substack, YouTube, or anything with an RSS feed — and it delivers a daily or hourly digest to your Telegram, on your schedule or on demand.

No accounts. No login. No algorithm deciding what you see. Just your sources, condensed.

Why

Checking every feed takes hours a day, and the platforms are designed to keep you there. RSS4.0 inverts that: instead of you visiting the feeds, the feeds come to you — batched, deduplicated, and readable in a few minutes. You choose the sources and the frequency; there is no engagement loop.

The deliberate trade-off: no similar-content suggestions. You only see what you chose to follow. (The echo-chamber question is a roadmap topic, not an MVP problem.)

Features

• No login — what you follow is plain markdown list files (sources/*.md), one handle or URL per line; settings are one small sources.yaml.
• X (Twitter) support — first-class, via a swappable adapter (see caveats).
• Interest groups — ## group headers in any source list split your follows into topical sections (finance, science, memes…); each group gets its own digest section and its own LLM recap, so war news never blends into a life-science article. A focus:/aim: line per group or platform states why you follow those accounts — the LLM reads it.
• Anything with an RSS URL — Substack, YouTube channels, blogs, podcasts, Mastodon, newsletters.
• Telegram delivery — a bot sends your digest hourly or at the daily times you pick (e.g. times: ["07:30", "12:00", "18:00"]). Scheduled digests never repeat an item; /digest on demand offers a choice: since last digest / today / last 24h (or skip the menu: /digest today). Change times and digest size from the chat with /settings times 7h30 12h 18h or /settings max 5 — applied live, persisted across restarts.
• Readable format — sections per platform and per X group (X / finance, YouTube), sources grouped inside, newest first, with hour timestamps.
• Markdown archive — every non-empty digest is saved as a dated .md file in digests/, with a full-content twin in raw/: complete tweet text (threads read top-down), video descriptions, article summaries, and significant images (charts, memes — as stable remote URLs). The raw file is what the LLM recap and /chat read.
• Quality filter — X retweets and replies are excluded by default (your own thread continuations are kept), plus an optional keyword blocklist for any platform (filters: in sources.yaml).
• Local LLM recap (digest.mode: llm) — per-section synthesized briefs from a GGUF model running on your machine (llama-cpp-python, no API key). The prompt weights trending topics (repeated across posts = today's real news), protects rare posters (a sage who tweets once is never drowned out by prolific posters), and follows your per-section focus: line. The plain link digest always ships too — no model, no problem.
• Chat with your digest — /chat x/finance yesterday in Telegram loads that slice of the raw data and answers questions about it, citing post URLs. Strictly grounded: the model only discusses the loaded posts and refuses everything else. /clear wipes the conversation, /endchat leaves.

How it works

sources/*.{txt,md} ──► adapters ──► normalized items ──► SQLite (dedupe)
                                                            │
                              ┌─────────────────────────────┤
                              ▼                             ▼
                    digest (links) + raw (full)   LLM recap + /chat (optional)
                              │                             │
                              └──► Telegram + digests/*.md, raw/*.md ◄──┘

Every platform reduces to a feed:

Platform	How	Reliability
Substack	native RSS (<pub>.substack.com/feed)	rock solid
YouTube	native RSS (youtube.com/feeds/videos.xml?channel_id=…)	rock solid
Any blog/podcast	its RSS/Atom URL	rock solid
X (Twitter)	public Nitter instance RSS (default — no account needed), or twscrape / self-hosted RSSHub	best-effort

Quick start

pip install -e .
cp sources.example.yaml sources.yaml   # settings: schedule, telegram, X backend
cp -r sources.example sources          # what you follow — edit these .md files
rss40 fetch                            # pull new items into the local store
rss40 digest                           # build a digest → digests/YYYY-MM-DD-HHMM.md
rss40 run --loop                       # keep running: fetch + deliver on your schedule

Setup

Telegram delivery (5 minutes, once)

1. Create a bot: message @BotFather on Telegram → /newbot → follow the prompts → copy the token it gives you.
2. Set the token as the RSS40_TELEGRAM_TOKEN environment variable (or put it under telegram.token in sources.yaml).
3. Run rss40 run --loop.
4. Send your bot /start — it replies with a confirmation and your chat id. That reply is your proof the bot works; /digest already works in that chat.
5. Put the chat id in sources.yaml under telegram.chat_id and restart rss40 run --loop. That both routes scheduled digests to you and claims the bot: from then on it answers you alone — strangers who find it get a refusal, and /digest never works for anyone until a chat id is configured. (Alternatively skip the always-on bot and schedule rss40 digest --send with Task Scheduler / cron / GitHub Actions — see docs/DEPLOYMENT.md.)

X sources (nothing, usually)

The default backend fetches X handles as RSS from a public Nitter instance — no account, no cookie, no setup. Just list handles in sources/x.md, one per line, optionally under ## group headers that give each interest its own digest section and recap:

karpathy

## finance
focus: oil trades, energy markets
DeItaone | Walter Bloomberg
unusual_whales

## science
paulg | Paul Graham

Public instances 503 or rate-limit now and then, so RSS4.0 ships with several and fails over automatically — a fetch tries each in order until one answers. To override or extend the list, set x.nitter_urls in sources.yaml with healthy instances from the health tracker. For something you control end-to-end, two sturdier backends exist (x.backend): twscrape (scrapes X's GraphQL API with a burner-account login) or rsshub (your own RSSHub instance). Trade-offs in docs/RESEARCH.md.

Your source lists

Everything you follow lives in the sources/ directory — one .md file per platform, one entry per line, optional display name after |, # comments allowed. Scales from three follows to three thousand:

sources/x.md           karpathy
sources/youtube.md     UCrDwWp7EBBv4NwvScIpBDOA | Anthropic
sources/substack.md    astralcodexten
sources/rss.md         https://example.com/feed.xml | Example Blog

Every list supports the same two extras (legacy .txt lists still work, .md wins when both exist):

• ## group headers — split the entries below into interest groups, each its own digest/recap section (e.g. ## finance in x.md, ## ai in youtube.md).
• focus: / aim: lines — one sentence stating your motive for following these accounts, per group (under a header) or for the whole platform (above any header). The LLM recap stresses it and /chat keeps it in mind (e.g. focus: oil trades under ## finance).

(sources.yaml holds only settings: schedule, Telegram, X backend.) Two importers fill the lists from the exports platforms actually give you (re-running merges, never duplicates):

• Any RSS reader (Feedly, Inoreader, FreshRSS…): export OPML, then rss40 import opml subscriptions.opml → sources/rss.md

• YouTube: Google Takeout → select only YouTube and YouTube Music → subscriptions → gives subscriptions.csv, then rss40 import youtube-takeout subscriptions.csv → sources/youtube.md

• X: there's no clean export (the official archive contains numeric IDs, not handles). Easiest: open x.com/<you>/following, scroll to the bottom so everyone is loaded, then paste this in the browser console (F12) — it copies all handles to your clipboard, ready to paste into sources/x.md:

  copy([...new Set([...document.querySelectorAll('[data-testid="UserCell"] a[href^="/"]')]
    .map(a => a.getAttribute("href").slice(1)).filter(h => !h.includes("/")))].join("\n"))

Fetching is politely rate-limited (10 concurrent requests), so a thousand sources takes a couple of minutes, not a ban. Note that with very large lists you'll want digest.max_per_source low — and topic-clustered summaries (roadmap) are where huge digests get tamed.

The X caveat

X has no free read API in 2026, so every X route is community-maintained and can break when X changes its internals: public Nitter instances get rate-limited or disappear, scrapers need fixes after GraphQL changes. RSS4.0 treats that as a design constraint — the X backend is swappable in one config line, and a broken X source only costs you the X section of a digest, never the digest itself. The full survey of what works in 2026 (live-tested) is in docs/RESEARCH.md.

Inspiration

Inspired by meridian — AI-generated daily intelligence briefs from hundreds of scraped sources. RSS4.0 borrows its RSS-first ingestion and brief format, but stays deliberately small: local-first, Python, no cloud dependency, aggregation before AI. See docs/RESEARCH.md for the full analysis and a survey of similar projects.

Local LLM recap & chat (optional)

pip install -e .[llm]                  # llama-cpp-python (CPU wheels included)
# download any instruct-tuned GGUF, e.g. Qwen2.5-3B-Instruct Q4_K_M (~2 GB)

# sources.yaml
digest:
  mode: llm
llm:
  model: models/qwen2.5-3b-instruct-q4_k_m.gguf   # resolved next to sources.yaml

Every digest then also produces digests/…-recap.md — 3-6 link-free bullets per section, built for phone-speed skimming — and is sent to Telegram after the link digest. Sources live one question away: /chat (or /chat x/finance yesterday) starts a conversation grounded in the fetched posts, and every chat answer cites the post URLs it drew on. If the model or dependency is missing, the link digest ships unaffected.

Choosing / upgrading the model

Recap and chat quality scale directly with model size. Any instruct-tuned GGUF works — swapping is one file download plus the llm.model line. Q4_K_M quants are the sweet spot; RAM needed ≈ file size + ~1.5 GB.

Tier	Models (GGUF, Q4_K_M)	Size	What you get
Baseline (default)	Qwen2.5-3B-Instruct	~2 GB	Fine bullets; occasionally cites the wrong URL when one author has many posts, may garble details
Sweet spot	Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Qwen3-8B	~4.5–5 GB	Accurate citations, faithful summaries, better topic grouping — the cheapest big jump
Best local	Qwen3-14B, Phi-4 (14B), Gemma-3-12B	~8–9 GB	Nuanced synthesis and reliable trending/rare-poster weighting; slow on CPU — use n_gpu_layers

Rules of thumb: if recaps invent details, go 7–8B. If sections are long (50+ posts) also raise n_ctx to 16384 (costs RAM). Multilingual sources? Qwen and Gemma handle non-English posts noticeably better than Llama at equal size.

GPU is not optional in practice. CPU generation runs at ~5 tok/s — a multi-section recap takes minutes. n_gpu_layers now defaults to -1 (whole model on GPU), but that only works if llama-cpp-python itself was built with GPU support; the default PyPI wheel is CPU-only, and rss40 logs a loud warning at model load when that's the case. Fix with a CUDA build:

pip install llama-cpp-python --force-reinstall --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124

The timing logs (llm generation: … tok/s) tell you immediately whether the offload worked — expect 40+ tok/s on a mid-range GPU vs ~5 on CPU.

Roadmap

• Optional LLM summarization (digest.mode: llm) — local per-section recaps + /chat, via llama-cpp-python
• Content-quality filtering (retweets/replies dropped, keyword blocklist) — Shorts detection still open
• Echo-chamber study: optional serendipity sources
• Retrieval over the raw archive (raw/*.md item anchors are already stable ids)
• Local AI agent appliance: run the whole pipeline as a self-contained agent — Docker on a Raspberry Pi, with the LLM served by LM Studio (OpenAI-compatible endpoint, GPU on the desktop) and a Dolphin-class model; needs an llm.backend: openai option next to the in-process llama.cpp backend

Documentation

• docs/RESEARCH.md — prior art, Meridian analysis, per-platform feed strategies
• docs/ARCHITECTURE.md — pipeline design, data model, adapter contract
• docs/DEPLOYMENT.md — local, scheduled, and GitHub Actions setups

License

See LICENSE.