DevRel Ships

One ingestion system that pulls "ships" from many sources (events, videos, blogs, demos, workshops, recipes) and upserts them into the DevRel Ships Notion database — so the bi-weekly report is a saved view, not a writing task.

A thin app on durable-sync: the generic Temporal sync engine (durable entity workflow, idempotent upsert, unattended OAuth refresh, rate-limit backoff) is the library; this repo is just the connectors + the DevRel-specific mapping and roster matching.

Notion auth without an admin (the interesting part)

Most DevRel teams don't own their company's Notion workspace, so they can't create the internal integration token the REST API requires. So we talk to Notion through its hosted MCP server with OAuth-as-an-individual + dynamic client registration: authorize once in a browser, as yourself — no admin, no IT ticket.

A long-running Temporal workflow (OAuthTokenWorkflow, in the library) owns the refresh token and serves fresh access tokens via query, so the worker runs unattended and the rotation race can't happen.

ONE-TIME (browser, as yourself)        ALWAYS-ON                  PER SOURCE (rolling)
  notion.bootstrap ─ refresh_token ─► OAuthTokenWorkflow  ◄─query─ sync_records (MCP)
                                      (refreshes on a timer)            ▲
                                                                   SourceSyncWorkflow
                                                                   └─ ShipsSource.fetch

Layout (the app — everything generic is in durable-sync)

devrel_ships/
├── config.py            # property names, data-source IDs, TYPE_ICONS, secrets
├── models.py            # ShipInput — a connector's normalized shape (pre-Record)
├── ships_source.py      # ShipsSource (durable_sync Source): ShipInput -> Record
├── roster_enrich.py     # roster matching as the destination-side session_enrich hook
├── pipeline.py          # wires ShipsSource -> NotionDestination (the one retarget point)
├── connectors/          # the ONLY source-specific code
│   ├── registry.py      #   source name -> fetch fn  (CONNECTORS)
│   ├── luma/            #   implemented: client.py + fetch.py (+ pure _to_ship)
│   ├── contentful/      #   implemented: scoped by content type
│   └── youtube/         #   stub
├── worker.py            # thin: run_worker(SOURCE, DESTINATION)
└── bootstrap.py         # thin: start_sources(SOURCE) — one entity workflow per connector
tests/                   # connector normalizers + the retarget tests (roster_enrich, to_record)

Setup

1. Install the engine — pip install -e ../durable-sync (or pip install -e ".[dev]").

2. Notion data source IDs — put them in .env as NOTION_DEVREL_SHIPS_DS / NOTION_ROSTER_DS. Idempotency keys on the existing Source ID column (no new column). Manual "Third Party" rows keep Source ID blank → never touched.

3. Luma API key — Luma calendar → Settings → API keys → Create key (needs Luma Plus).

Run it (local dev)

pip install -e ../durable-sync
cp .env.example .env                       # data-source IDs + LUMA_API_KEY (+ CONTENTFUL_*)
temporal server start-dev                  # terminal 1

# one-time Notion auth (no admin), from the library:
python -m durable_sync.connectors.notion.bootstrap   # opens browser, consent as yourself
python -m durable_sync.connectors.notion.prove       # OPTIONAL: headless proof

python -m devrel_ships.worker              # terminal 2
python -m durable_sync.connectors.notion.start        # launch the token-owner workflow (once)
python -m devrel_ships.bootstrap           # terminal 3: start one entity workflow per connector
pytest

Encrypt payloads at rest (recommended for shared/Cloud clusters):

python -m durable_sync.codec               # prints a key; put it in .env as DURABLE_SYNC_ENC_KEY

Why this shape

• The generic engine is a dependency, not copy-paste. Dedup, roster-relation writes, OAuth, backoff live once in durable-sync; this repo can't reinvent them.
• connectors/<source>/ is the only per-source code: a client + a pure _to_ship normalizer. ships_source.py turns those into Records.
• Roster matching is the destination-side session_enrich seam — it needs the open Notion session (to read Team Members), which is exactly what that library hook provides.

Adding a source

1. connectors/<source>/fetch.py — a plain fetch(after_iso) -> list[ShipInput] (NO @activity.defn) plus a pure normalize function.
2. Add one line to connectors/registry.py (CONNECTORS).
3. Unit-test the normalizer in tests/. Upsert, dedup, roster scoping, scheduling come from the library + pipeline.py.

Contentful note: scoped by content type (config.CONTENTFUL_DEVREL_TYPES), not per-author — DevRel-owned types are ingested wholesale; Marketing-shared types are never fetched.

Known seams (intended)

• Generic hosts (e.g. "Temporal Team") match no roster person → Team Member left empty → excluded from the Bi-Weekly Report. Source Author is always written. (The per-run unmatched_owner count from the old shape isn't surfaced anymore.)
• The OAuth path authorizes as an individual: if that person loses Notion access, re-run the library's notion.bootstrap + notion.start.
• Two Bi-Weekly Report filters are Notion-UI-only: Ship Date within past 2 weeks and Emeritus? is unchecked.