Architecture
A single, end-to-end map of every component in the repository: what it is, what it reads, what it writes, and how it talks to the others. If you are debugging an unexpected commit or onboarding a new operator, start here.
One picture
┌──────────────────────────┐
│ Claude Code routine │
│ (cloud, scheduled) │
│ prompt: "Read │
│ prompts/daily-cti- │
│ brief.md and execute" │
└─────────────┬────────────┘
│ git push
▼
reads ┌──────────────────────────────────────────────────────────┐
──────► │ repository │
│ │
│ prompts/ state/ │
│ ├ daily-cti-brief.md ├ covered_items.json │
│ ├ weekly-summary.md ├ cves_seen.json │
│ └ CHANGELOG.md ├ deep_dive_history.json │
│ └ run_log.json │
│ sources/ │
│ briefs/ └ sources.json │
│ ├ YYYY-MM-DD.md tools/ │
│ └ weekly/YYYY-Www.md ├ check_brief.py (Phase 5.5) │
│ └ fetch_source.py │
│ docs/ │
│ ├ workflow.md │
│ ├ verification.md │
│ ├ routine-setup.md │
│ ├ architecture.md (this file) │
│ └ improvements.md │
└──────────────┬─────────────────────────────────┬─────────┘
│ │
push to claude/**│ (fallback) push to main │
▼ ▼
┌──────────────┐ ┌──────────────────┐
│ auto-merge │ ff-merges │ deploy-site.yml │
│ workflow │───────────────▶│ runs build.py │
└──────────────┘ to main │ uploads to Pages │
└────────┬─────────┘
│
▼
GitHub Pages reader
(real HTML pages emitted
by site/build.py — no SPA)Components
prompts/ — the agent's instructions
Two prompts. Each is the entire runtime contract for a routine; the routine is invoked with a one-line wrapper ("Read this prompt and execute it").
prompts/daily-cti-brief.md— the daily brief. Phases 0–6 + a 5.5 self-check gate (preflight → parallel research → verification → deep dive → compose → state update → self-check → commit/push). Spawns four parallel research sub-agents.prompts/weekly-summary.md— the weekly consolidating summary. Reads the past 7 days of dailies, plus two long-horizon sub-agents (W1 long-running campaigns + annual reports; W2 policy + regulatory).prompts/CHANGELOG.md— the version history of the prompt itself. Treat as the audit trail for editorial-policy changes.
briefs/ — the canonical output
One Markdown file per day at briefs/YYYY-MM-DD.md, one per ISO week at
briefs/weekly/YYYY-Www.md. Sections 0–8 per the structure pinned in
briefs/README.md:
0 TL;DR · 1 Immediate Actions (often absent) · 2 Active Threats / Trending
Actors / Notable Incidents & Disclosures · 3 Trending Vulnerabilities ·
4 Research & Investigative Reporting · 5 Updates to Prior Coverage ·
6 Deep Dive · 7 Action Items · 8 Verification Notes. Each individual H3
item carries a structured metadata footer (— *Source: … · Tags: … ·
Region: … [· CVE: …] [· CVSS: …] [· Vector: …] [· Auth: …] [· Status: …]*)
parseable by the build. These files are immutable once committed —
corrections happen in the next brief, not by editing past ones.
state/ — rolling memory across runs
The agent re-reads these every run before writing.
state/covered_items.json— full coverage records for every CVE / actor / campaign / incident / tool / annual report ever referenced. Each item has anappearances[]array — the site uses this to render the "story timeline" on each topic page.state/cves_seen.json— flat fast-lookup CVE index for sub-agent dedup. A subset ofcovered_items.json(CVEs only) with a tighter schema.state/deep_dive_history.json— rolling 30-day list of{date, topic, category}entries used by Phase 3 to apply the deep-dive category-rotation rule.state/run_log.json— rolling 90-day per-run record: model, sub-agent source allocation (sources_attempted/sources_used/items_returnedper S1–S4),fetch_failures,items_published,deep_dive. Surfaced on the operations dashboard at/ops/.
sources/ — the curated source list
sources/sources.json — ~80 entries spanning
national CERTs, vendor TI, journalism, breach trackers. Schema:
{
"id": "stable-id-never-changes", // referenced from covered_items.json
"publisher": "Display name",
"url": "https://...",
"category": ["ch-eu", "vulns", ...],
"reliability": "HIGH | MEDIUM | LOW",
"language": ["en", "de", ...],
"status": "active | candidate | demoted",
"last_successful_fetch": "YYYY-MM-DD | null",
"consecutive_failures": 0,
"notes": "history of changes, dated"
}The agent maintains this file autonomously per the lifecycle in the top-level README.
tools/ — small operator-shipped helpers
tools/fetch_source.py— stdlib-only Python bridge that re-issues HTTP requests with a stable desktop-Chrome User-Agent. Solves the recurring 403 / 302-to-login that the routine container hits on a handful of high-signal publishers (CISA pages, the Swiss NCSC Cyber Security Hub) where the upstream WAF is filtering the agent's default UA. Mandatory every run for CISA + NCSC.ch — do not even attemptWebFetchon those hosts; go straight to the bridge. Read-only by design: no auth, no JS execution, no third-party deps, host allow-list enforced.tools/check_brief.py— the institutionalised Phase 5.5 self-check gate. Stdlib-only Python script that bundles every pre-commit consistency check (state JSON parses, CVE sync, H3 footer presence and field completeness, taxonomy validation, UPDATE citations, multi-CVE / multi-source / primary-source-quality checks,tools/fetch_source.py-for-CISA/NCSC.ch enforcement,covered_items.jsonappearance heuristic,run_log.jsonOps-dashboard population,sources.jsonlast-fetched bookkeeping, IOC heuristic scan with version-string suppression) plus runs the build-side smoke tests insite/test_build.py. Imports the footer parser + taxonomy loader fromsite/build.pyso script and build agree on parsing rules. Read-only — the agent fixes drift, the script reports it. Non-zero exit aborts the commit. Maintained as part of the agent's self-evolution authority.
docs/ — operator-facing documentation
docs/workflow.md— phase-by-phase execution of both routines.docs/verification.md— the editorial / fake-news defence policy. The agent's quality gates are derived from this.docs/routine-setup.md— one-time GitHub App / Pages / branch-permission setup.docs/architecture.md— this file.docs/improvements.md— recommended improvements to the agentic workflow and the site, with rationale.
.github/workflows/ — CI
auto-merge-claude.yml— triggers on push toclaude/**. Fast-forwardsmainfrom the feature branch and deletes the branch. Belongs to the agent's publish chain; do not edit unless you understand the publishing fallback indocs/routine-setup.md.deploy-site.yml— triggers on push tomainwhenever the site inputs change. Runssite/build.py, uploads the bundle to GitHub Pages.
The two workflows are independent. The site is a consumer of the agent's output and never writes back.
site/ — the public reader
A stdlib-only Python static-site generator (site/build.py) emits a real
HTML page for every URL — home, every brief, every per-item block, every
CVE / source / topic page, every tag and region index, the operations
dashboard, the about pages. JavaScript only enhances (topbar search
autocomplete via data/search.json, GitHub-stars badge, brief-page filter
chips, theme cycle, copy-link, SPA-redirect bootstrap on /). With JS
disabled the site is fully readable. See site/README.md
for the internal layout. The site is read-only with respect to the rest
of the repo:
- It only reads
briefs/,state/,sources/,README.md,docs/*.md,prompts/CHANGELOG.md, andsite/taxonomy.yaml. - It writes nothing back — its build artifact lives entirely under
site/_site/(gitignored locally; force-pushed to thegh-pagesbranch by the CI workflow). - It emits three RSS feeds:
/feed.xml(daily, last 30),/feed-weekly.xml(weekly, last 30),/feed-items.xml(per-item granular, last 50). All three use the actual git-commit timestamp of the underlying brief as<pubDate>(not midnight-of-brief-date). - The unified search index at
_site/data/search.jsoncovers briefs, CVEs, topics, and sources. - The operations dashboard at
/ops/is rendered server-side fromstate/run_log.jsonat build time.
site/taxonomy.yaml is the controlled vocabulary
for every metadata-footer value (themes / sectors / regions / nexus /
cve_types / cve_vectors / cve_auth / cve_status / sections). The build
refuses any post-cut-over item using a value not in this file.
Data flow per routine run
┌──────────────┐ preflight ┌──────────────────────────────┐
│ routine │─────────────▶│ load sources.json (active) │
│ fires │ │ load past 7 days of briefs │
│ (operator- │ │ load covered_items.json │
│ scheduled) │ │ load cves_seen.json │
│ └──────────┬───────────────────┘
│ │
▼ spawn 4 sub-agents in parallel │
┌──────────────────────────────┐ │
│ S1 active threats / vulns │ │
│ S2 CH/EU & public sector │ │
│ S3 research & journalism │ │
│ S4 incidents & disclosures │ │
└──────────┬───────────────────┘ │
│ flexible Markdown returns │
▼ │
┌──────────────────────────────┐ │
│ verify (two-source / CERT) │ │
│ dedup vs preflight context │ │
│ rank, apply deep-dive │ │
│ category-rotation rule │ │
└──────────┬───────────────────┘ │
▼ │
┌──────────────────────────────┐ │
│ Write briefs/YYYY-MM-DD.md │ │
│ (with prompt-version badge) │ │
└──────────┬───────────────────┘ │
▼ │
┌──────────────────────────────┐ │
│ Phase 4.5 — verification │ │
│ sub-agent loop (≤3 iters): │ │
│ truth gate │ │
│ ─ every URL fetched │ │
│ ─ every claim grounded │ │
│ editorial-quality gate │ │
│ ─ relevance to CH/EU SOC │ │
│ ─ vendor advisory ≻ NVD/ │ │
│ CERT as primary │ │
│ ─ drop low-relevance │ │
│ ─ deepen unclear items │ │
│ (≤3 follow-up subagents) │ │
│ ─ surface contradictions │ │
│ ─ pursue missed angles │ │
│ ship at iter cap; residuals │ │
│ logged in § Verification │ │
└──────────┬───────────────────┘ │
▼ │
┌──────────────────────────────────────────────────────────────┐
│ Update state/covered_items.json, state/cves_seen.json, │
│ state/deep_dive_history.json, state/run_log.json (full │
│ sub-agent allocation + fetch_failures + verification_ │
│ iterations + verification_residual_count — Ops dashboard │
│ depends on this), sources/sources.json (last-seen, demotions, │
│ candidates) │
└──────────┬───────────────────────────────────────────────────┘
▼
┌──────────────────────────────┐
│ Phase 5.5 — self-check gate │ python3 tools/check_brief.py
│ (institutionalised script): │ ─ state JSON parses
│ │ ─ every brief CVE in cves_seen
│ │ ─ core sections vs covered appear-
│ │ ances heuristic
│ │ ─ every UPDATE has inline cite
│ │ ─ every H3 has a v2 footer
│ │ (Source ≥1 link + Tags + Region)
│ │ ─ CVE entries carry CVE / Vector /
│ │ Auth / Status
│ │ ─ multi-CVE: shared CVSS or per-
│ │ CVE breakdown
│ │ ─ primary-source quality (NVD /
│ │ CERT as sole primary → WARN)
│ │ ─ tools/fetch_source.py used for
│ │ CISA + NCSC.ch when 403 hit
│ │ ─ run_log.json today fully
│ │ populated (Ops dashboard)
│ │ ─ ≥1 source fetched today
│ │ ─ heuristic IOC scan
│ │ ─ taxonomy validation
│ │ ─ site/test_build.py passes
│ exit != 0 → abort commit; │
│ brief stays on disk; next │
│ run rebuilds state from it │
└──────────┬───────────────────┘
▼
┌──────────────────────────────┐ one of:
│ git commit + push │ ① push origin HEAD:main
│ │ ② push claude/<name>; CI ff
└──────────────────────────────┘The agent never bypasses any of these phases — Phase 0 is a hard prerequisite for Phase 1, Phase 5 (state update) is a hard prerequisite for Phase 6 (commit). If a phase fails, the prompt instructs the agent to stop and surface the error rather than silently continuing.
Adding a new component
A safe pattern for extending the system without affecting the agent:
- Site-only feature (new view, new search facet). Edit
site/. The agent's run is untouched. - New data field (e.g. add a
severitytocovered_items.json). Update the prompt's Phase 5 instructions, then re-flow the new field throughsite/build.pyand the renderers. Old briefs stay valid because the field is optional. - New source category. Edit
sources/sources.json(add the entry) and the category list inprompts/daily-cti-brief.mdPhase 1 (so a sub-agent picks it up). The site's category filter picks it up on the next build automatically. - New routine (e.g. monthly horizon scan). Add a prompt in
prompts/, create a new Claude Code routine pointing at it, and add a parallel workflow in.github/workflows/if you want CI to react to its output.
Anything more invasive (new state file, new repo layout) — write down the
reasoning in docs/improvements.md before making the
change. The agent's prompts are the load-bearing part of the system; small
contract changes are easy to ship by accident and hard to roll back.