Medium Daily Digest
Project Overview
Medium's recommendation feed is noisy, so I built a personal aggregator that pre-judges it for me: every day it fetches the RSS feeds I follow, has an LLM compress each article into a structured summary — a one-line hook, three key points, and a worth-reading verdict — and delivers the result as a single email I can triage in under a minute. It doubles as a test bench for comparing a frontier cloud model (Claude) against a local open-source model (Gemma via Ollama) on the same summarization task. Paywalled articles are handled honestly: truncated content is treated as a known input, and the model judges from the preview whether the full read is worth the click.
Tech Stack & Links
Backend / Core
- Python — stdlib-first (sqlite3, smtplib, email) to keep the dependency surface minimal
- feedparser — handles Medium's quirky feed formats
LLM Layer
- Anthropic Claude API — stable structured JSON output
- Ollama + Gemma (27B / 4B) — local inference path: 27B for final summaries and scoring, 4B for lightweight coarse screening
- nomic-embed-text (via Ollama) — embeddings for semantic interest matching
Storage & Delivery
- SQLite — zero-config single file, perfect for a single-user dedup table
- SMTP (Gmail App Password) — the delivery channel least likely to break, no third-party lock-in
Config & Ops
- python-dotenv — secrets centralized in config.py
- GitHub Actions (planned) — free scheduled execution
Key Features
Pluggable summarizer architecture
A Summarizer abstract base class plus a get_summarizer(name) factory — Claude and Gemma swap with a single environment variable.
Three-gate funnel pipeline
fetch → dedupe → Gate 1 (embedding interest filter) → Gate 2 (4B coarse screen) → Gate 3 (27B summary + score) → sort by score → Top N → deliver.
Structured JSON summaries with graceful degradation
Every summary carries a one-line hook, three key points, and a worth-reading verdict; JSON repair and retry cover the occasional format break from the local model.
Embedding-based semantic interest matching
nomic-embed-text vectorizes my interest description and each article; the cosine similarity threshold is configurable.
I/O isolation for testability
Network, database, LLM, and SMTP each live in independent modules; pure functions stay easy to test, and the pipeline dry-runs with a FakeSummarizer — CI makes zero API calls.
Honest paywall handling
No circumvention: truncated content is treated as a known input, and the LLM judges from the preview whether an article deserves the click.
Algorithm Flow
Every article flows through a cost-shaped funnel: cheap, recall-protecting gates run first, the expensive model runs last, and a score-based post-filter makes the final cut. Gates 1 and 2 are deliberately loose — their job is only to discard obvious noise — while the 27B summarizer produces both the structured summary and the 1–10 score that ultimately decides what makes the email.
- 1
Fetch & Deduplicate
feedparser pulls every followed Medium RSS feed; a SQLite table records what has already been sent, so only never-seen articles enter the pipeline.
- 2
Gate 1 — Embedding Interest Filter
nomic-embed-text embeds my interest description and each article, and cosine similarity screens out clearly off-topic items. The threshold stays deliberately loose (0.45) — tightening it to 0.6 discarded relevant articles.
- 3
Gate 2 — 4B Coarse Screen
A local Gemma 4B model flags obviously low-quality pieces. Because a 4B's judgment is unstable, the rule is keep-when-unsure — this gate protects recall rather than cost.
- 4
Gate 3 — 27B Summary + Score
Gemma 27B (or Claude, switched by one environment variable) produces a structured JSON summary — one-line hook, three key points, worth-reading verdict — plus a 1–10 relevance/quality score. JSON repair and retry handle occasional format breaks from the local model.
- 5
Post-Filter, Rank & Deliver
Anything below a score of 6 is dropped, survivors sort by score descending, the top 10 make the cut, and the digest goes out via Gmail SMTP.
Challenges & Solutions
Problem
The funnel's first two gates weren't filtering anything, so costs never came down. By design, Gate 1 (embedding interest filter) and Gate 2 (4B coarse screen) should block most noise before the expensive 27B summarization. In practice, Medium articles are titled and abstracted to sound relevant to everything: at a cosine threshold of 0.45 almost everything squeaked past Gate 1, and Gate 2 ran on a keep-when-unsure rule to avoid false kills — so it dropped almost nothing. 28 of 30 articles walked straight through to the 27B stage, and token costs stayed high.
What I tried
Raising the cosine threshold from 0.45 to 0.6 discarded genuinely relevant articles — recall collapsed. Making the 4B gate more aggressive was no better: a 4B model's judgment is too unstable, and good articles started getting thrown away.
Final approach
Move the real filtering decision one gate later. Gates 1 and 2 stay loose to protect recall, while the 27B model emits a 1–10 relevance/quality score alongside each summary — judgment from the strongest model, produced as a near-free byproduct since it was running anyway. A post-filter then applies a MIN_SCORE = 6 floor, sorts survivors by score descending, and caps the email at MAX_ITEMS = 10.
Key insight
Cheap upstream filters simply lack judgment, and forcing their thresholds harder trades away recall. But the strongest model has to run regardless — asking it for a score turns a required expense into a free, high-quality filtering signal. The real selection decision moved from the coarse gates to the model with the most judgment, and the daily digest went from thirty undifferentiated items to a stable 8–10 high-scoring picks.