Abstrahierte Entity-Relationship-Konstellation über einem Radar-Pin auf Bonn

Architektur
& Datenmodell

Statische Übersicht des Lagebild-Systems auf nostromo (Produktion): die Container-Pipeline von der externen Quelle bis zum Browser und das auf der Platte liegende Datenmodell. Das Datenmodell unten wird aus store.py und models.py generiert — es kann nicht still vom Code abweichen.

01 Architektur

Container-Pipeline C4 · Level 2

C4 Level 2 Container-Diagramm: Collectors → Kafka → Consumers → PostgreSQL → api + web → Ingress

Tragende Entscheidungen

Entscheidung	Wahl	Begründung
Datenbank	`PostgreSQL (CNPG)`	CloudNativePG statt SQLite: shared zwischen Deployments, HA-fähig, LVM-Storage.
Collector-Split	`feed-collector · nina · allris`	Separate Deployments: unabhängige Skalierung, Restart-Isolation, je eigener ServiceMonitor.
Nachrichten-Bus	`Kafka (Strimzi KRaft)`	Entkoppelt Collector von Consumer; Append-Only-Buffer; Ordering-Invariante per Single-Partition.
Push an Browser	`SSE über mtime-Polling`	Keine inotify-Abhängigkeit; portabel.
Dedup	`In-Memory Fuzzy-Cluster`	Dasselbe Ereignis aus 3 Quellen kollabiert.
Signal-Watchers	`Sidecars im api-Pod`	Gemeinsamer prometheus_multiproc-Dir; metrics-exporter-Sidecar aggregiert alle Watchers auf :9200.
Web-Skalierung	`2 Replicas, RollingUpdate`	Stateless SSR, kein geteilter State; Zero-Downtime-Deploy.

02 Datenmodell

Vier Tabellen in PostgreSQL (CNPG). Beziehungen sind dokumentiert (das Schema ist für günstige Abfragen denormalisiert; Fremdschlüssel sind nicht deklariert).

Tabelle 5 Spalten

sources

Feed registry. Seeded from sources.yaml, then DB-owned. Full pydantic Source JSON lives in `config`; hot columns are denormalized out.

id PK	VARCHAR
kind	VARCHAR
enabled	INTEGER
poll_minutes	INTEGER
config	TEXT

Tabelle 28 Spalten

items

Every seen item. Dedup + Atom source of truth. `fingerprint` is a stable SHA-256 over guid/link.

fingerprint PK	VARCHAR
source_id	VARCHAR
source_title	VARCHAR
category	VARCHAR
guid	VARCHAR?
title	VARCHAR
summary	VARCHAR?
link	VARCHAR?
source_published_at	VARCHAR?
collected_at	VARCHAR
event_at	VARCHAR?
first_seen	VARCHAR
ortsteil	VARCHAR?
stadtbezirk	VARCHAR?
lat	FLOAT?
lon	FLOAT?
geo_source	VARCHAR?
cluster_id	VARCHAR?
severity	TEXT?
expires	TEXT?
onset	TEXT?
geometry	TEXT?
count	INTEGER?
hours_reported	INTEGER?
planned_end	TEXT?
service_code	TEXT?
curation_locked	BOOLEAN
updated_at	VARCHAR?

Indizes stadtbezirkcluster_idcollected_atsource_published_at

source_id → sources.id emitted by · many-to-one
cluster_id → items.cluster_id clustered with · self (dedup)

Tabelle 7 Spalten

fetch_state

Per-source fetch cadence and health. Drives the Quellenstatus panel.

source_id PK	VARCHAR
last_fetch	VARCHAR
last_status	VARCHAR?
last_error	VARCHAR?
last_ok	VARCHAR?
fail_count	INTEGER
last_items	INTEGER?

source_id → sources.id tracks · one-to-one

Tabelle 8 Spalten

signals

Append-only state log per watcher. One row on each state change (`source` is a watcher name, not a feed source). Drives the Signals panel and /api/signals; severity is derived by the API when absent.

id PK	INTEGER
source	VARCHAR
state	VARCHAR
previous_state	VARCHAR?
emitted_at	VARCHAR
severity	VARCHAR?
detail	TEXT?
info_url	VARCHAR?

Indizes source, emitted_at

Quellen-Modell Union über kind

sources.config speichert das vollständige Pydantic-Source-JSON. Die Variante wird über das Feld kind diskriminiert.

rss RSSSource

A standard RSS/Atom feed.

id	str
poll_minutes?	int
enabled?	bool
authoritative?	bool
auto_approve?	bool \| str
kind?	'rss'
title	str
category	'blaulicht' \| 'news' \| 'amtlich' \| 'wissenschaft' \| 'events' \| 'verkehr' \| 'wetter' \| 'radverkehr' \| 'Baustelle'
url	str
stadtbezirk?	str \| None
bonn_only?	bool
user_agent?	str \| None

autobahn AutobahnSource

Autobahn roadworks/warning JSON API for a set of roads near Bonn.

id	str
poll_minutes?	int
enabled?	bool
authoritative?	bool
auto_approve?	bool \| str
kind?	'autobahn'
title?	str
roads?	list

dwd DWDSource

DWD municipality-level weather warnings (WFS).

id	str
poll_minutes?	int
enabled?	bool
authoritative?	bool
auto_approve?	bool \| str
kind?	'dwd'
title?	str
gemeinde?	str

radiobonn RadioBonnSource

Radio Bonn news via listing-page allowlist + monthly sitemap for dates. No RSS feed exists. Strategy: scrape the local-news listing page to get Bonn-area article IDs, then cross-reference with the monthly sitemap for lastmod timestamps. /dpa/ URLs skipped per robots.txt.

id	str
poll_minutes?	int
enabled?	bool
authoritative?	bool
auto_approve?	bool \| str
kind?	'radiobonn'
title?	str
category?	'blaulicht' \| 'news' \| 'amtlich' \| 'wissenschaft' \| 'events' \| 'verkehr' \| 'wetter' \| 'radverkehr' \| 'Baustelle'
listing_url?	str
sitemap_base?	str
bonn_only?	bool

swb-bus-und-bahn SWBBusUndBahnSource

SWB Bus und Bahn (Stadtwerke Bonn) live ÖPNV disruptions. Collected from the BONNmobil disruption API (``bc-drawings-extra-data``) — the same structured, real-time feed the official disturbances map renders. Each entry carries a headline, affected lines, validity window, and a full description. Drives the ÖPNV health signal. The endpoint is configurable.

id	str
poll_minutes?	int
enabled?	bool
authoritative?	bool
auto_approve?	bool \| str
kind?	'swb-bus-und-bahn'
title?	str
category?	'blaulicht' \| 'news' \| 'amtlich' \| 'wissenschaft' \| 'events' \| 'verkehr' \| 'wetter' \| 'radverkehr' \| 'Baustelle'
url?	str
stadtbezirk?	str \| None
bonn_only?	bool

nina NINASource

BBK NINA civil-protection warnings (Kafka-decoupled collector).

id	str
poll_minutes?	int
enabled?	bool
authoritative?	bool
auto_approve?	bool \| str
kind?	'nina'
title?	str
category?	'blaulicht' \| 'news' \| 'amtlich' \| 'wissenschaft' \| 'events' \| 'verkehr' \| 'wetter' \| 'radverkehr' \| 'Baustelle'
url?	str

allris AllrisSource

Bonn ALLRIS council-information system feed (Kafka-decoupled collector).

id	str
poll_minutes?	int
enabled?	bool
authoritative?	bool
auto_approve?	bool \| str
kind?	'allris'
title	str
category?	'blaulicht' \| 'news' \| 'amtlich' \| 'wissenschaft' \| 'events' \| 'verkehr' \| 'wetter' \| 'radverkehr' \| 'Baustelle'
url	str

open311 Open311Source

Bonn Mängelmelder via Open311 / GeoReport v2.

id	str
poll_minutes?	int
enabled?	bool
authoritative?	bool
auto_approve?	bool \| str
kind?	'open311'
title?	str
category?	'blaulicht' \| 'news' \| 'amtlich' \| 'wissenschaft' \| 'events' \| 'verkehr' \| 'wetter' \| 'radverkehr' \| 'Baustelle'
endpoint?	str
max_entries?	int

bonn-radzaehlung BonnRadzaehlungSource

Bonn cycling count stations (Fahrradmessstellen) via Offene Daten Bonn.

id	str
poll_minutes?	int
enabled?	bool
authoritative?	bool
auto_approve?	bool \| str
kind?	'bonn-radzaehlung'
title?	str
category?	'blaulicht' \| 'news' \| 'amtlich' \| 'wissenschaft' \| 'events' \| 'verkehr' \| 'wetter' \| 'radverkehr' \| 'Baustelle'
url?	str
counts_url?	str
locations_url?	str

baustelle BaustelleSource

Bonn UDP "Baustellen tagesaktuell" roadworks WFS (standalone Collector). ``auto_approve`` MUST stay False: every Baustelle is *planned* construction with an explicit validity window — ADR-0003 D-1's ``ConstructionProject`` case, not an ``Event``. A battle-test review found the originally proposed ``auto_approve: true`` would let the correlator's existing ``Construction`` -token derivation auto-confirm these as public Events with no human gate, which D-1 forbids (see design.md Non-Goals). Do not flip this to True.

id	str
poll_minutes?	int
enabled?	bool
authoritative?	bool
auto_approve?	bool \| str
kind?	'baustelle'
title?	str
category?	'blaulicht' \| 'news' \| 'amtlich' \| 'wissenschaft' \| 'events' \| 'verkehr' \| 'wetter' \| 'radverkehr' \| 'Baustelle'
endpoint?	str
provider_url?	str

manual ManualSource

Hand-curated items backed by a committed JSON file (no upstream API). ``auto_approve`` MUST stay False: a manual entry is a human already typing facts straight into the registry — there is no independent second source to corroborate it, so it still needs ordinary curation review before it becomes a public Event (same rationale as ``BaustelleSource`` above).

id	str
poll_minutes?	int
enabled?	bool
authoritative?	bool
auto_approve?	bool \| str
kind?	'manual'
title?	str
category?	'blaulicht' \| 'news' \| 'amtlich' \| 'wissenschaft' \| 'events' \| 'verkehr' \| 'wetter' \| 'radverkehr' \| 'Baustelle'
data_file	str

Kategorien Vokabular ↗

blaulichtnewsamtlichwissenschafteventsverkehrwetterradverkehrBaustelle

scripts/gen-datenmodell.py (just update-datenmodell)

Architektur& Datenmodell