last_checked stored epoch seconds as REAL; SQLite's version-specific
REAL->text float printing rewrote ~every row on dump, so a locally
regenerated inventory.sql diverged from CI's. Normalize last_checked to
int on the entry before both the SQL record and the YAML dump, and make
the column INTEGER. All readers do coarse day-level staleness checks, so
sub-second precision is unused.
Verified: load->save idempotent for inventory.sql AND inventory.yaml, and
local (SQLite 3.40.1) output byte-identical to CI-equivalent python:3.11
(SQLite 3.46.1). The deterministic data regenerates on the next pipeline
save; committing only the code avoids conflicting with pipeline data churn.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
load_inventory_channels() selected entries via 'youtube_mosaic' in entry
(key presence). save_inventory() always writes a youtube_mosaic column
that the SQL round-trip materializes as an empty dict {} on EVERY entry,
so the filter matched all 18,647 inventory entries and emitted ~18.6k
inline <img> links on a single line. That produced a 4MB v2-docs/index.md
(and 2.9MB docs/index.md), hanging both V1 and V2 index pages in the
browser. A prior revert masked it; the next SQL-backed sync re-broke it.
Fix: only treat an entry as a mosaic channel when youtube_mosaic is a
non-empty dict with a logo image (136 curated channels). Regenerated both
index files via reorganize_mosaic — docs/index.md is now byte-identical
to the last known-good state (52KB); v2-docs/index.md back to 41KB.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The date field gh_pushed was written as the literal string 'N/A' when
GitHub data was unavailable (fast_enrich, gemini_utils). That non-date
string then reached datetime.fromisoformat() downstream. Now writes None
(which all readers already handle); gh_license keeps its 'N/A' string
sentinel since it is a displayed value, not a date.
Also hardens the twin fromisoformat() call in agentic_curator's MVQ
penalty check (guards N/A/None and wraps parsing) so a bad value can't
crash a curation batch.
Adds scripts/normalize_gh_pushed.py to clean the 281 existing inventory
entries (run in CI to avoid SQLite float-serialization churn — see
script docstring).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The V2 safety audit emitted hundreds of WARN lines and then aborted
entirely with 'NoneType object has no attribute lower':
- validate_mvq_compliance: gh_pushed of 'N/A' (and historic None/empty)
was passed to datetime.fromisoformat() -> 'Invalid isoformat string';
and gh_stars/stars stored as null hit 'None * int'. Now skips entries
with no usable push date and coalesces None before the star math.
- validate_linguistic_tagging: a null 'language' field caused
None.lower(), which aborted the whole report. Now coalesces to English.
- generate_audit_report: each mandate now runs in isolation so a single
bad entry can no longer kill the entire audit; failures are logged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each (category × period) cell in news_digest.json now carries a hash of its
entry pool (sorted URLs). Before calling Gemini, the engine checks:
1. Hash matches stored hash (same inventory entries)
2. Last analyzed within MAX_STALENESS_DAYS=30
Both conditions true → reuse existing ranked list, 0 Gemini API calls.
Only cells where the inventory actually changed (new/removed entries) trigger
a fresh Gemini ranking.
Expected savings: re-runs with code-only changes → 0 API calls (down from 66).
New ingestion batch → only affected categories call Gemini (5-10 of 26 typical).
Stale refresh (>30d) → full refresh at most once per month.
Also:
- Remove day-level GitHub Actions cache for digest (redundant: news_digest.json
is committed to the repo and is the persistent store for _meta)
- Always run news_digest.py in both 04.1 and 09 workflows (it's cheap when 0
Gemini calls are needed)
- Trending badge now reads _meta.last_updated (actual analysis date) instead of
file mtime (which changes on every render/commit)
- Remove invalid extra_head key from v2-mkdocs.yml (left over from rollback)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1. Skip page deletion and nav sync in --render-only mode
The CI publisher always uses --render-only but the pruning phase was
deleting pages not regenerated in a given pass (e.g. low-hit pages),
breaking nav references and corrupting the MkDocs build.
2. Protect dimension pages from deletion in full mode
Even in a full (non-render-only) run, pages defined in self.dimensions
are never deleted. Truly orphaned pages (not in dimensions AND not
generated) are the only ones pruned.
3. _sync_enterprise_navigation returns True/False
Deletion is gated on nav sync success. If nav sync fails, deletion
is skipped to prevent inconsistency (deleted files + stale nav).
Also fixes the fragile re.sub(r'nav:.*') regex by using string
indexing instead, preventing accidental truncation of extra_css etc.
4. Deploy workflow V2 sanity check
If V2 build produces fewer than 50 HTML pages or no index.html,
deploy falls back to V1-only instead of overwriting V1 with a
broken V2 build.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_render_single_link (line 872) crashed with TypeError: '>=' not supported
between NoneType and int. Replaced ALL remaining .get('stars', 0) patterns
with .get('stars') or 0 throughout v2_optimizer.py to prevent further
crashes from null stars values in inventory.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three sites in v2_optimizer where `-x.get("stars", default)` crashes
with TypeError when stars field is present but null in inventory.
Fixed with `-(x.get("stars") or default)` pattern consistently.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
item.get("resource_type", "Reference") returns None when the field
exists with a null value in the inventory. Use `or "Reference"` instead.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1. geo_region inference: _infer_geo_from_url() infers Americas/Europe/España/Asia-Pacific
from URL TLD (.es .de .fr .uk .jp .cn etc.) as fallback when geo_region field is empty.
Industry digest now shows real content instead of empty categories.
2. Last-updated badge: trending section header now shows "Updated Jun 19, 2026" pill
derived from news_digest.json mtime — gives readers confidence in freshness.
3. Search boost: tech-digest and industry-digest pages now have search.boost: 2
frontmatter so they rank higher in MkDocs Material site search.
4. RSS feed (src/rss_generator.py): generates v2-docs/feed.xml with top-20 curated
picks from 3-month digest. Runs after news_digest in publisher. Autodiscovery
<link rel="alternate"> added to v2-mkdocs.yml extra_head.
5. Weekly cron (09.weekly_digest.yml): runs every Monday 06:00 UTC, generates digest
with Gemini, renders digest pages, commits, then triggers publisher.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The 5-item link list was redundant with the 6 trending cards block
that the publisher generates. Cleaner UX: hero card (amber) → trending
cards → /tech-digest/ for full 22-category view.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New hero card (amber color) between AI & MCP Agents and Agentic Video
Hub, linking to /tech-digest/ with emoji icon and subtitle
- Dynamic digest-preview block above Agentic Pulse: shows top 1 link
from 5 priority categories (Kubernetes, AI, Security, IaC, Observability)
pulled from data/news_digest.json on each optimizer run
- New CSS: hero-badge-card--amber, hero-badge-icon, digest-preview
component with category chips and hover states
- v2_optimizer now generates both the card and preview automatically
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Inventory entries can have stars=null or discovered_at=null. Replace
.get("stars", 0) with .get("stars") or 0 in all three modules so that
None values are safely coerced to 0/"" before numeric/string comparison.
Fixes TypeError crashes in:
- v2_optimizer.py:632 _calculate_tags
- dedup.py:74,108 title dedup and entry scoring
- news_digest.py:322 category pool sorting
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
detect_license_changes() iterated all repos with gh_license set, causing
hundreds/thousands of GitHub API calls. Apply the same 200-repo cap as
the activity enrichment module to keep total enrichment under ~3 min.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Consolidate 2 GitHub API calls per repo into 1: open_issues_count
already includes PRs on GitHub, second /pulls call was redundant
- Reduce MAX_REPOS_DEFAULT 500→200: sufficient for meaningful enrichment,
200 × 0.5s = ~100s vs 500 × 1.5s = ~12.5min in CI
- Reduce GITHUB_RATE_DELAY 0.75s→0.5s: still safely under 5000 req/hr
- Add 429 rate-limit backoff (5s sleep instead of crashing)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The legacy landscape.cncf.io/api/items endpoint now returns HTML (SPA)
instead of JSON. Switch to GitHub Search API querying cncf-graduated,
cncf-incubating, and cncf-sandbox topics — reliable and uses existing
GH_TOKEN auth.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Change ` ## Category` to ` **Category**` inside MkDocs Material
tabs to avoid MD023 linter errors (headings must start at col 0)
- Sanitize pipe characters in "why" text to prevent broken markdown tables
- Regenerated tech-digest.md (1328 lines, 3 tabs: 333/443/552 lines)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Variable items per period: 3 months=10, 6 months=15, 12 months=20
so each tab shows progressively more content
- Use year field as fallback in _is_within_period when discovered_at
doesn't discriminate (backfilled entries)
- Fix NoneType comparison in _fallback_items for entries with null stars
- Regenerated digest: 3m=220, 6m=330, 12m=440 items across 22 categories
- tech-digest.md now 1353 lines with differentiated tabs (339/449/559 lines)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Update curator Gemini prompt with Phase 5: Company & Geo Classification
to extract company name and geo_region (americas/europe/spain/asia_pacific)
for industry digest categories
- Store company and geo_region from AI response in eval_data
- Add Intelligence Digest nav entries to _sync_enterprise_navigation so
digest pages persist across nav rebuilds
- Add dedup scan and enrichment pipeline steps to monthly cron workflow
(01.1.agentic_cron.yml) in addition to publish workflow
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add src/enrichment.py (395 lines): CNCF landscape API integration for
project graduation status, GitHub issue/PR velocity tracking with
community health scoring, and license change detection with history
- Add src/dedup.py (194 lines): URL normalization dedup, content-hash
dedup, title-similarity dedup (85% threshold via SequenceMatcher),
and resolution strategy keeping highest-quality entry
- Enable PWA/offline plugin in v2-mkdocs.yml for cached offline reading
- Expand video hub categorization with MLOps, security, IaC, GitOps,
observability, FinOps, cloud providers, and language-specific routing
- Add dedup scan and enrichment pipeline steps to CI publish workflow
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add last_ai_eval timestamp to eval_data in both fast-track and grounded-track
evaluation paths, plus SQL schema and curator
- Entries enriched >6 months ago are automatically flagged for re-evaluation
instead of being skipped (stale content detection)
- Enhance "See Also" links with cross-dimension references based on shared
tags between pages, in addition to same-dimension related links
- Add _collect_tags_from_tree() helper for recursive tag extraction
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add 26-category news digest engine (src/news_digest.py) with Gemini AI ranking
for 3/6/12 month temporal panels across tech, cloud, and geo categories
- Add discovered_at, company, geo_region fields to inventory schema with backfill
script populating 18K+ existing entries
- Fix critical v2-mkdocs.yml bug: plugins were nested under theme (silently disabled)
- Add MkDocs Material features: instant nav, breadcrumbs, footer, announce bar
- Add trending cards CSS grid and replace Agentic Pulse with dynamic Trending Now
- Generate tech-digest.md and industry-digest.md with tabbed 3/6/12 month views
- Merge 12 stub pages (<40 lines each) into parent categories with redirects
- Replace 50 bare except:pass patterns with contextual logging across all pipeline files
- Expand autonomous discovery from 6 to 14 GitHub search queries
- Add stale health re-check for online entries older than 30 days
- Track addition_method by source type (rss, twitter, github_trending)
- Add digest generation step to CI publish workflow
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
