MyNews

A Ruby gem that transforms 200+ RSS feeds into themed bulletins published 3x daily. The pipeline fetches, normalizes, summarizes (via LLM), deduplicates, and publishes — outputting to both FreshRSS and local Markdown/HTML files.

Key Features Async concurrent fetching with ETag caching and rate limiting Full-text extraction and HTML-to-Markdown normalization LLM-powered summarization (OpenAI, Anthropic, Gemini via `ruby_llm`) SimHash deduplication and recurring topic detection Themed bulletin assembly with scheduled publishing FreshRSS Fever API integration SQLite persistence with FTS5 full-text search Mli> Proxy support for restricted feeds

Pipeline

FETCH

▶

NORMALIZE

▶

SUMMARIZE

▶

CLUSTER

▶

PUBLISH

Stage	What it does
Fetch	Async HTTP with ETag caching, rate limiting, Tor proxy support
Normalize	Readability full-text extraction, HTML-to-Markdown conversion
Summarize	LLM summarization via ruby_llm (OpenAI, Anthropic, Gemini)
Cluster	SimHash deduplication, recurring topic detection
Publish	Themed bulletin assembly, FreshRSS Fever API, Markdown+HTML output

Quick Links

• Installation -- get up and running
• Quick Start -- first pipeline run
• CLI Reference -- all commands
• Configuration -- defaults.yml, feeds.yml, bulletins.yml
• API Reference -- module documentation
• Examples -- 7 runnable examples

Requirements

• Ruby >= 3.2.0
• SQLite3
• LLM API key (OpenAI, Anthropic, or Gemini) for summarization