Architektur
& Datenmodell
Statische Übersicht des Lagebild-Systems auf nostromo (Produktion): die Container-Pipeline
von der externen Quelle bis zum Browser und das auf der Platte liegende Datenmodell. Das
Datenmodell unten wird aus store.py und models.py generiert — es
kann nicht still vom Code abweichen.
01 Architektur
| Entscheidung | Wahl | Begründung |
|---|---|---|
| Datenbank | PostgreSQL (CNPG) | CloudNativePG statt SQLite: shared zwischen Deployments, HA-fähig, LVM-Storage. |
| Collector-Split | feed-collector · nina · allris | Separate Deployments: unabhängige Skalierung, Restart-Isolation, je eigener ServiceMonitor. |
| Nachrichten-Bus | Kafka (Strimzi KRaft) | Entkoppelt Collector von Consumer; Append-Only-Buffer; Ordering-Invariante per Single-Partition. |
| Push an Browser | SSE über mtime-Polling | Keine inotify-Abhängigkeit; portabel. |
| Dedup | In-Memory Fuzzy-Cluster | Dasselbe Ereignis aus 3 Quellen kollabiert. |
| Signal-Watchers | Sidecars im api-Pod | Gemeinsamer prometheus_multiproc-Dir; metrics-exporter-Sidecar aggregiert alle Watchers auf :9200. |
| Web-Skalierung | 2 Replicas, RollingUpdate | Stateless SSR, kein geteilter State; Zero-Downtime-Deploy. |
02 Datenmodell
Vier Tabellen in PostgreSQL (CNPG). Beziehungen sind dokumentiert (das Schema ist für günstige Abfragen denormalisiert; Fremdschlüssel sind nicht deklariert).
sources
Feed registry. Seeded from sources.yaml, then DB-owned. Full pydantic Source JSON lives in `config`; hot columns are denormalized out.
| id PK | VARCHAR |
| kind | VARCHAR |
| enabled | INTEGER |
| poll_minutes | INTEGER |
| config | TEXT |
items
Every seen item. Dedup + Atom source of truth. `fingerprint` is a stable SHA-256 over guid/link.
| fingerprint PK | VARCHAR |
| source_id | VARCHAR |
| source_title | VARCHAR |
| category | VARCHAR |
| guid | VARCHAR? |
| title | VARCHAR |
| summary | VARCHAR? |
| link | VARCHAR? |
| source_published_at | VARCHAR? |
| collected_at | VARCHAR |
| event_at | VARCHAR? |
| first_seen | VARCHAR |
| ortsteil | VARCHAR? |
| stadtbezirk | VARCHAR? |
| lat | FLOAT? |
| lon | FLOAT? |
| geo_source | VARCHAR? |
| cluster_id | VARCHAR? |
| severity | TEXT? |
| expires | TEXT? |
| onset | TEXT? |
stadtbezirkcluster_idcollected_atsource_published_atsource_id→sources.idemitted by · many-to-onecluster_id→items.cluster_idclustered with · self (dedup)
fetch_state
Per-source fetch cadence and health. Drives the Quellenstatus panel.
| source_id PK | VARCHAR |
| last_fetch | VARCHAR |
| last_status | VARCHAR? |
| last_error | VARCHAR? |
| last_ok | VARCHAR? |
| fail_count | INTEGER |
| last_items | INTEGER? |
source_id→sources.idtracks · one-to-one
signals
Append-only state log per watcher. One row on each state change (`source` is a watcher name, not a feed source). Drives the Signals panel and /api/signals; severity is derived by the API when absent.
| id PK | INTEGER |
| source | VARCHAR |
| state | VARCHAR |
| previous_state | VARCHAR? |
| emitted_at | VARCHAR |
| severity | VARCHAR? |
| detail | TEXT? |
| info_url | VARCHAR? |
source, emitted_atsources.config speichert das vollständige Pydantic-Source-JSON. Die
Variante wird über das Feld kind diskriminiert.
rss RSSSourceA standard RSS/Atom feed.
| id | str |
| poll_minutes? | int |
| enabled? | bool |
| kind? | 'rss' |
| title | str |
| category | 'blaulicht' | 'news' | 'amtlich' | 'wissenschaft' | 'events' | 'verkehr' | 'wetter' |
| url | str |
| stadtbezirk? | str | None |
| bonn_only? | bool |
| user_agent? | str | None |
autobahn AutobahnSourceAutobahn roadworks/warning JSON API for a set of roads near Bonn.
| id | str |
| poll_minutes? | int |
| enabled? | bool |
| kind? | 'autobahn' |
| title? | str |
| roads? | list |
dwd DWDSourceDWD municipality-level weather warnings (WFS).
| id | str |
| poll_minutes? | int |
| enabled? | bool |
| kind? | 'dwd' |
| title? | str |
| gemeinde? | str |
radiobonn RadioBonnSourceRadio Bonn news via listing-page allowlist + monthly sitemap for dates. No RSS feed exists. Strategy: scrape the local-news listing page to get Bonn-area article IDs, then cross-reference with the monthly sitemap for lastmod timestamps. /dpa/ URLs skipped per robots.txt.
| id | str |
| poll_minutes? | int |
| enabled? | bool |
| kind? | 'radiobonn' |
| title? | str |
| category? | 'blaulicht' | 'news' | 'amtlich' | 'wissenschaft' | 'events' | 'verkehr' | 'wetter' |
| listing_url? | str |
| sitemap_base? | str |
| bonn_only? | bool |
swb-bus-und-bahn SWBBusUndBahnSourceSWB Bus und Bahn (Stadtwerke Bonn) live ÖPNV disruptions. Collected from the BONNmobil disruption API (``bc-drawings-extra-data``) — the same structured, real-time feed the official disturbances map renders. Each entry carries a headline, affected lines, validity window, and a full description. Drives the ÖPNV health signal. The endpoint is configurable.
| id | str |
| poll_minutes? | int |
| enabled? | bool |
| kind? | 'swb-bus-und-bahn' |
| title? | str |
| category? | 'blaulicht' | 'news' | 'amtlich' | 'wissenschaft' | 'events' | 'verkehr' | 'wetter' |
| url? | str |
| stadtbezirk? | str | None |
| bonn_only? | bool |
nina NINASourceBBK NINA civil-protection warnings (Kafka-decoupled collector).
| id | str |
| poll_minutes? | int |
| enabled? | bool |
| kind? | 'nina' |
| title? | str |
| category? | 'blaulicht' | 'news' | 'amtlich' | 'wissenschaft' | 'events' | 'verkehr' | 'wetter' |
| url? | str |
allris AllrisSourceBonn ALLRIS council-information system feed (Kafka-decoupled collector).
| id | str |
| poll_minutes? | int |
| enabled? | bool |
| kind? | 'allris' |
| title | str |
| category? | 'blaulicht' | 'news' | 'amtlich' | 'wissenschaft' | 'events' | 'verkehr' | 'wetter' |
| url | str |
scripts/gen-datenmodell.py (just update-datenmodell)