Skip to content

Redesign nemar.org: Phases 1+2+3a on Cloudflare Pages + Astro - #26

Merged
neuromechanist merged 30 commits into
mainfrom
feature/issue-1-epic-nemar-redesign
May 18, 2026
Merged

neuromechanist merged 30 commits into
mainfrom
feature/issue-1-epic-nemar-redesign

Conversation

@neuromechanist

Copy link
Copy Markdown
Contributor

Closes #1 (in part — see Deferred section).

Summary

Lands the nemar.org redesign epic to main. 27 commits, 104 files, +11k lines. Currently deployed to nemar-website.pages.dev from the epic branch; merging here flips production to main once CF Pages production-branch setting is updated.

What ships

Phase 1 — Foundation + Discover (#2 / #7)

  • Brain hero landing with CSS-only twinkle particles, themed light/dark
  • Hero stat tiles, Browse / Contribute CTAs
  • Discover: filter sidebar (modality OR/AND, participants range, has-HED), always-on search, server-side offset pagination across 65 pages

Phase 2 — Dataset detail (#3 / #8 / polish #9)

Phase 3a — Data quality (#24)

  • QualityPanel + 4 charts (pipeline success, 3 histograms for data/channels/IC retention, line-noise distribution)
  • Runtime aggregator (src/lib/qa-aggregate.ts) walks data.nemar.org/<id>/qa/ and assembles per-file dataqual.json into the QaAggregates shape (hallu doesn't pre-aggregate). Bounded concurrency, edge-cached per Cache-Control: s-maxage=300, stale-while-revalidate=86400

Cross-cutting polish + perf

Deliberately deferred

Phase 4 — Citation Dashboard + Community + Documentation (#5) — separate effort; stays on the issue tracker
Phase 5 — Live-features architecture (#6) — production branch cutover from feature/issue-1-epic-nemar-redesign to main is the only post-merge step; CF Pages dashboard setting flip
Half B of Phase 3 — FileVis/FileView modals (#25) — file-level click integration with the BIDS tree; needs render-tree.ts to emit Vis/View buttons; tracked separately

Open follow-ups

  • nemar-cli#529 — 17 of 51 on* mirrors have a GitHub tag but no registered version (publish webhook race); RCA + sweep+retry design in the issue
  • nemar-cli#532 — data.nemar.org listing returns children: [] for deep paths (sub/ses/eeg) despite S3 having the files. Affects QualityPanel rendering on session-structured datasets; aggregator handles gracefully today
  • website#21 — Phase 3 rework against current epic architecture (closed in part by phase 3: QualityPanel with runtime aggregator from per-file dataqual #24)
  • website#25 — File modals (Half B of Phase 3)

Verification

  • Per-PR reviews on Sonnet via /review-pr for every PR in the stack; all findings addressed before merge
  • 107/107 unit tests pass; typecheck clean; biome clean
  • Live verified on each preview deploy. Final epic deploy: nemar-website.pages.dev serves landing, discover, detail (including QualityPanel for on002718), ds→on redirect

Post-merge checklist

  • Flip CF Pages production branch from feature/issue-1-epic-nemar-redesign to main (Dashboard → Settings → Builds & deployments)
  • Verify production nemar-website.pages.dev still serves
  • Confirm DNS / CNAME if any custom domain is wired

* scaffold: Astro 6 + Cloudflare Pages adapter

Bun-managed Astro project targeting Cloudflare Pages. SSR by default for
filter + detail pages, with a static landing. Adds biome (lint/format),
vitest (unit), playwright (e2e), wrangler.toml, and a typed
@astrojs/cloudflare adapter config.

Phase 1 of #1; closes part of #2.

* phase1: base layout, design tokens, nav, footer

- design tokens (colors, type, spacing, radii) with light/dark via
  prefers-color-scheme + manual override persisted to localStorage
- Base.astro layout with inline FOUC-prevention bootstrap, preconnect
  to api/data.nemar.org
- Brand component using the legacy NEMAR logo SVG (text + brain +
  electrodes) extracted from the Joomla home
- Nav with sticky header, primary links, theme toggle, login modal stub
- Footer with funding line and section nav
- placeholder hero on / pointing to /discover

Part of #2.

* phase1: API client, types, filter state, format helpers

- typed api client for api.nemar.org/datasets list + get
- DatasetQuery / SortOption / ModalityCode types
- filter state library: URL serialize/deserialize, modality AND/OR,
  reserved-keyword auto-flip (EEG/MEG/iEEG/EMG/HED), client-side
  multi-modality + participant-range filtering
- format helpers: bytes, count, date, relative time, modality split
- vitest coverage: 22 filter tests + 15 format tests, all green

Part of #2.

* phase1: landing hero + Discover page + stub routes

- Hero (/): gradient orb, two-line search input, three stat tiles
  (datasets/participants/total size) populated from api.nemar.org with
  graceful legacy-figure fallback when the API is unreachable
- Discover (/discover): SSR fetch with server-side search + sort + single-
  modality filter, client-side multi-modality AND/OR and participant range,
  URL-bound state via filterStateFromURL/toURL
- DatasetCard with modality badges, BIDS-aware ID/title pair, facts table
  (participants, size, version, updated), OpenNeuro cross-link
- FilterSidebar with chip selector, AND/OR pill, range inputs, soon-pills
  on Phase-3 toggles (has-QA, has-10-20)
- Pagination with ellipses, focusable nav links
- Stub pages: /about /support /community /citation-dashboard /docs /404
- Renders ≥10 cards from live api.nemar.org/datasets; typecheck clean
  across 26 files

Part of #2.

* design: hide empty card facts + fix giant external-link SVG

DatasetCard: when participants/file_size/version/updated are null or 0,
omit the corresponding fact instead of rendering '—' placeholders. With
the OpenNeuro catalog imports returning sparse metadata, the old layout
made every card look broken (four dashes in a row).

Also pull the OpenNeuro source link out of the body into a footer band
so the card has a clear two-zone layout (content + provenance), and
constrain the external-link arrow SVG to 12x12. The reset.css
'display: block; max-width: 100%' on svg was letting the icon stretch
to 126x126 because the scoped logical-property width on .ds-card__ext
didn't apply to the SVG's intrinsic size — fixed by adding explicit
width/height attributes and switching CSS to physical width/height.

FINDING-001 (HIGH), FINDING-002 (HIGH)
* phase2: neuroschema types + data.nemar.org client + provenance + bids-tree + markdown

Lib layer for the upcoming dataset detail page.

- neuroschema.ts: types for NeuroschemaDataset, LandingPayload, Manifest
  (mirrors the live data.nemar.org responses verified this session)
- data-api.ts: typed getLanding/getMetadata/getManifest + fetchManifestEntryText
  with timeouts. Honors PUBLIC_DATA_BASE_URL override; 404 -> null, network
  errors throw so callers can render the right empty state
- provenance.ts: detect on*/ds* parity. nm* is native; on* is derived with
  best-effort original ds id resolution (related_identifiers DERIVED relations,
  URL host pattern, DOI pattern, last-ditch on->ds prefix swap). The mirror
  version is the highest vN.0.0 per the NEMAR policy that every OpenNeuro pull
  is a major version bump (see nemarOrg/nemar-cli#448 comment thread).
  listMirrorVersions returns every vN.0.0 newest-first.
- bids-tree.ts: fold manifest into nested TreeNode with size aggregates,
  numeric+locale-aware sort (sub-2 before sub-10). classifyFile gates the
  per-file Vis/View buttons by extension.
- markdown.ts: dep-free CommonMark subset (headings, paragraphs, lists,
  code fences + inline code, bold/italic, links + autolinks, hr). HTML-escapes
  everything and rejects javascript:/data: URLs. Scope deliberately small;
  follow-ups can add tables/blockquotes when a real README needs them.

69/69 unit tests passing.

* phase2: dataset detail page + provenance, version switcher, BIDS tree, README

Components:
- DetailHeader-equivalent inline in [id].astro (ID + lineage label + title +
  lede + modality chips + action row + DOI badge)
- ActionBar: Download zip, Compute (stub modal pointing at Phase 5 + NSG),
  Issues, GitHub, OpenNeuro for on* datasets
- ProvenanceCard (on* only): title 'NEMAR-curated copy of OpenNeuro ds00xxxx',
  2-cell DOI parity grid (NEMAR / OpenNeuro), View-original + View-mirror CTAs,
  All mirror snapshots <details> when vN.0.0 count > 1. The mirror version
  pointer points to highest vN.0.0 per the policy in nemar-cli#448 comment.
- VersionSwitcher: segmented tab strip + overflow dropdown when > 4 versions,
  'mirror' chip on vN.0.0 entries, hidden when only one version exists
- AuthorList: name + ORCID icon link + first affiliation (with +N more), expandable
- DoiBadge: copy-on-click with success state, label + value + small icon
- Chip: reusable label chip (neutral/accent/warning tones)
- BidsTree + BidsDirChildren: recursive collapsible tree from manifest paths,
  auto-opens shallow trees, EEG/.set + .tsv files tagged 'Vis · soon' / 'View · soon'
  pointing at Phase 5 visualizer work
- DetailRail: right-rail metadata (license, BIDS version, modalities/tasks/datatypes,
  sessions, publish date with relative time, authors, funding, keywords, related
  identifiers DOIs + URLs)
- Readme: server-side markdown rendering with empty-state for datasets without one

Route:
- pages/dataset/[id].astro SSR: parallel fetch of landing + metadata, then manifest
  for selected version (?v=... param falls back to latest). Defensive 404 when
  dataset is not public or doesn't exist. README pulled from manifest entry's
  presigned URL with 2.5s timeout. Cache-Control: max-age=60 to keep within
  presigned-URL TTL.

Verified live:
- /dataset/nm000104 (native, multi-thousand-file manifest)
- /dataset/on005262 (derived, exercises ProvenanceCard with DOI parity grid)
- /dataset/nm000103 (manifest 404 -> graceful 'Manifest unavailable' panel + rail
  still populated from metadata.json)
- /dataset/does-not-exist (404)
- 69/69 unit tests passing, 0 typecheck errors across 46 files

* phase2: cap README height with expand/collapse toggle + tighter tree auto-open

README:
- Wrapped in a 32rem max-block-size container with a bottom fade gradient
- 'Expand README' / 'Collapse README' button overlays the fade; jumps to
  full-height on click
- Auto-hides the toggle entirely when the README fits within the cap (short
  READMEs render unchanged)
- aria-expanded + aria-controls wired for keyboard / screen-reader users

BIDS tree:
- MAX_AUTO_OPEN_DEPTH 2 -> 1 + AUTO_OPEN_CHILD_CAP 12 -> 6 so large datasets
  (nm000104 with 100+ subjects, on005262 with 12+ subjects) render with
  only top-level subjects open. Users expand the ones they want.

Verified live against nm000104 (large tree) and on005262 (derived dataset).
…ords) (#9)

* style(design): FINDING-001 — set font-family on html root to stop Times New Roman fallback

Top-layer elements (dialog) and any element where the cascade from body
breaks were falling back to the UA serif default because <html> itself
had no font-family rule. Setting it on the root fixes it everywhere at
once.

Also adds color-scheme: light dark on the root so the browser draws form
controls and scrollbars in the right theme without per-element overrides.

* style(design): FINDING-006 — render keyword chips from neuroschema 0.3.0 {term} objects

Keywords on the live data.nemar.org/<id>/metadata.json are {term, subject_scheme,
value_uri} objects, not bare strings. The rail was rendering them as
[object Object]. Updated NeuroschemaDataset.keywords to the union type and
DetailRail to normalize both shapes, linking the chip to value_uri when
present so MeSH-coded keywords resolve to nlm.nih.gov.

* style(design): FINDING-005 — restore row layout for nested BIDS tree levels

Astro scopes <style> per component, so the .tree__row flex layout declared in
BidsTree.astro never reached BidsDirChildren.astro's summary rows. Without
display:flex, the chevron, icon, name, counts, and size each rendered on
their own line — see the captured screenshot showing 'ses-1597766237/ 1 dirs · 5
files 200 MB' broken across multiple lines.

Duplicates the row styles inside BidsDirChildren.astro. Marked with a comment
to keep both in sync if either file changes.

* style(design): FINDING-002 — DOI badge tap targets reach 44px

Three changes to push the copy + value link past 44px without re-skinning
the badge:
- Default badge min-block-size 36px -> 44px
- Compact variant min-block-size 28px -> 36px (still smaller, used inside
  the provenance card where space is tight)
- Value link gets display:inline-flex, align-self:stretch, padding-block,
  so the link's hit area fills the badge height rather than wrapping just
  the 18px text
- Copy button bumped from 28x28 to 36x36

Paired-target rule covers the remaining gap: the whole badge reads as one
control because the label, value, and copy button sit on a shared row.

* style(design): FINDING-003,004 — provenance buttons + brand link reach 44px

Two small touch-target bumps:
- ProvenanceCard .prov__btn: min-block-size 36 -> 44, removed redundant
  vertical padding so the button is exactly 44px tall instead of 39px.
- Brand component: min-block-size 44 on the anchor wrapper, padding-block
  to keep the logo visually centered without resizing the SVG.

Both now pass WCAG AAA + iOS HIG touch-target rules without changing the
visual chrome.

* style(design): FINDING-003 — provenance buttons reach 44px

The earlier commit only included the Brand fix because the
ProvenanceCard edit didn't persist on the first pass. Re-applying the
min-block-size 36 -> 44 + zero vertical padding so the buttons are
exactly 44px tall.

* design: README fallback chain + remove redundant tree icon

Two findings from the post-PR-9 spot check:

FINDING-007 (HIGH) — README unavailable on on005262 because manifests carry
git-annex content only, not git-tracked files at root. Added a three-stage
README resolution in pages/dataset/[id].astro:
  1. data.nemar.org manifest README* (preferred; version-locked)
  2. GitHub raw README on the dataset repo (HEAD, main, master across
     README.md/README/README.txt/readme.md); SSR-only so no CORS concerns
  3. neuroschema description (last-resort so the section is never blank)
Readme component renders a small banner explaining the source when the
content didn't come from the manifest. Verified live: on005262 now shows
the GitHub README ('ArEEG: Arabic EEG Dataset...') with a 'README from
GitHub repository' banner above it.

FINDING-008 (MED) — Redundant directory icons. Every tree row had both
the rotating chevron SVG and a static unicode '▸' next to the name. Two
arrows pointing right was confusing. Dropped the static icon; the chevron
alone signals expandability, and directory names already carry a trailing
'/' so the file-vs-dir distinction stays.

Also made data-api jsonFetch resilient to 429 (single retry honoring
Retry-After) + 5xx (return null instead of throwing) so the page renders
an empty state when data.nemar.org rate-limits the dev server instead of
blowing up with a 500.

* design: adopt legacy nemar.org brain hero, themed for light + dark

Replaces the abstract electrode-net SVG with the original wireframe brain
illustration from legacy nemar.org (public/hero-brain.png, 368KB),
composited the way the legacy did — but themed for both light and dark.

How it works:
- Same brain.png used in both themes
- Dark mode: mix-blend-mode: screen so the black background drops out and
  the teal wireframe glows. Matches the legacy hero look pixel-for-pixel.
- Light mode: filter: invert + hue-rotate + multiply blend, turning the
  teal-on-black into a dark-navy-on-light outline. Opacity 0.45 keeps it
  readable as a backdrop without competing with the text.
- Sparkles: pure CSS (12 + 9 radial-gradient dots in two stacked layers
  with offset twinkle animations), no particles.js dep. Reduced-motion
  honored — animation drops out, sparkles stay visible static.

Theme variables (--hero-bg-1/--hero-glow/--hero-spark/...) are declared
on .hero with light defaults, overridden in @media + [data-theme=dark].
Stat tiles and search input get matching glass treatments.

Composition matches the legacy reference screenshot in
website/screenshots/01-landing-page.png. Brain on the right, tagline on
the left, sparkles fill, stat tiles below the search.

* design: anchor hero brain to content container on wide viewports

The brain was absolutely positioned inside .hero (viewport-bound), so on
1920px+ screens the empty margin beside the .container (1200px max) grew
and shoved the brain way past the right edge of the readable content.

Moved the brain layer inside .hero__grid (which inherits .container's
max-width and centering), so it stays paired with the copy column at any
viewport. Reduced inline-size clamp from clamp(560px, 75%, 1100px) to
clamp(480px, 60%, 820px) — the smaller cap reads better when anchored to
content.

Sparkles stay viewport-wide (still in .hero, not .hero__grid) since the
twinkle field should fill the whole atmospheric backdrop.

* design: hero copy + dual CTA reflects native uploads + OpenNeuro mirrors

- Lede rewritten to call out three sources: NEMAR-native datasets,
  OpenNeuro mirrors, and user uploads. Removes the implicit framing that
  every dataset came through OpenNeuro.
- New CTA pair under the search bar: 'Browse datasets' (primary, dark
  pill) and 'Contribute your dataset' (ghost with arrow, points at
  /contribute). The arrow signals 'goes somewhere new' for the upload
  flow that's coming as a separate epic (researcher dashboard).
- Stub /contribute target lands on a follow-up issue and will resolve
  to the upload-flow landing once the dashboard epic ships.

Verified at 1440x900 light + dark.

* design: Compute button -> 'Compute instructions' with 3-route modal

Old behavior: 'Compute' button opened a single-paragraph 'coming with
Phase 5' dialog that pointed at nemar-cli and the NSG portal without
specifics.

New behavior: button label is 'Compute instructions' (sets expectation —
this gives you what you need to run compute today, not a launch button).
Modal lays out three concrete routes:

1. NSG portal — link with the dataset's S3 prefix prefilled in the copy
2. nemar-cli — copy-pasteable 3-line shell snippet using {datasetId}
3. Just the files — rclone/aria2c/curl against data.nemar.org/{id}/

Below the steps, a styled note: 'Direct compute access is coming soon.
One-click NSG submission from this page is scoped for a follow-up
phase. Tracked on #6.'

Verified the dialog opens with all three steps + the coming-soon note.
The 'compile' setting still bundled sharp into the SSR worker, which the
Workers runtime can't execute — deploys failed with:

  Uncaught Error: [unenv] process.report.getReport is not implemented yet!

We don't use Astro's <Image> component or astro:assets anywhere — every
visual asset is a plain <img> referencing /public/*.{svg,jpg,png}.
Passthrough lets the original assets ship as-is, keeps sharp out of the
worker bundle, and shaved the deploy from failing to ~3s.
…se-rail fallback

Three fixes triggered by /browse QA on the deployed preview:

1. Discover pagination only covered 200 of 645 datasets because the page
   fetched limit=200 once and client-paginated over it. Switched to
   server-side offset pagination: limit=pageSize, offset=(page-1)*pageSize.
   All 65 pages are now reachable. Bumped the Pagination window from ±1
   to ±3 around current so the strip shows [1, 2, 3, 4, 5, ..., 65]
   instead of [1, 2, ..., 65].

2. formatDate(null) / formatRelativeTime(null) threw on null inputs —
   catalog-only rows (ds*) ship with updated_at: null, which caused Astro
   to silently skip those cards mid-render. Pages 3+ rendered fewer
   cards than the count suggested. Both helpers now accept nullish and
   return empty string so callers' truthy checks skip the row cleanly.

3. ActionBar's compute-instructions dialog was rendering at top-left
   because the .compute-dialog selector was declared twice and the
   second declaration only set max-inline-size — the dialog inherited
   no inset/margin from native <dialog> auto-centering. Collapsed both
   into one with inset:0 + margin:auto + max-block-size:90vh.

4. Right-rail enrichment for on* datasets: when the neuroschema
   metadata.json has empty modalities/tasks/authors (true for every
   OpenNeuro import today — see nemarOrg/nemar-cli#512), fall back to
   the api.nemar.org/datasets/<id> catalog row and split its CSV strings
   into the array shape DetailRail expects. getDataset() also fixed to
   unwrap the {dataset: ...} envelope the single-id endpoint returns.

Verified live: all 65 discover pages render full card counts; compute
dialog centers; on005262 detail page no longer crashes.
…eservation

Tree links previously pointed at the manifest's presigned S3 URL (f.url),
which sends users straight to the content-addressed object — browsers
then download with the SHA-named filename
('SHA256E-s197576448--c70cae6e4a043e2124d7e5ee94422d02.bdf') rather than
the BIDS-shaped name they expect ('sub-01_task-typing_emg.bdf').

Fix: route every file link through data.nemar.org/<id>/<v>/<bids-path>
so the Worker's redirect is in the chain. Also adds the HTML 'download'
attribute as a hint with the basename.

This alone won't fix the SHA-naming today — the Worker still 302s to the
same presigned URL without setting response-content-disposition. But
once nemarOrg/nemar-cli#513 ships (filing now), the Worker will add
response-content-disposition=attachment; filename="<bids-basename>" to
the presign and filenames will start being correct. This commit puts
the frontend in position to benefit immediately when that lands.
ProvenanceCard was a full-width banner with title, paragraph, DOI parity
grid, two CTAs, and a mirror-versions <details> — way too much visual
weight for what should be 'switch the view' affordance. The user called
it out: most of the time you just want to know 'is this a NEMAR copy
and which version am I on'.

Replaced with ProvenanceToggle: a single chip next to the dataset ID
('NEMAR copy of ds005262' + chevron). Click reveals a small popover
with the DOI parity, an accent 'View as-imported mirror (vN.0.0)' /
'Switch to NEMAR latest' button (the label flips based on current
version), and 'Open on OpenNeuro'. Click outside closes.

Also adds SearchBar.astro and surfaces it on the Discover header so
users don't have to hand-edit the ?q= URL param. The hero search on /
still works; this is the always-on search inside the catalog view.

Filing fuzzy/context search engine as a separate issue — for now the
SearchBar submits as a plain GET that the existing api.nemar.org
LIKE-based search handles.
Retroactive init using the cross-agent vibe-rules templates from
init-project. Should have run before Phase 1; capturing the established
stack and conventions now so future agent sessions (Claude, Codex,
Cursor) pick up the project consistently.

AGENTS.md — project-wide rules:
- Stack: Astro 6 + Bun + Cloudflare Pages (SCCN)
- Architecture map (src/ layout + lib helpers + data flow)
- Critical principles: imageService passthrough (sharp doesn't run in
  Workers), reuse backend (don't reinvent), theme-aware tokens, null-
  safe data helpers, download links via data.nemar.org
- Backend dependency map (nemar-cli#511/512/513 with frontend fallbacks)
- Project-specific gotchas (Astro scoped styles, multi-modality
  pagination tradeoff, brain-hero blend modes)

CLAUDE.md — Claude-specific adapter: @AGENTS.md import + worktree paths,
skill triggers, deploy auth notes, links to .context/handoff.md as the
session-state pointer.

.rules/ — kept ci_cd, code_review, documentation, git, self_improve,
serena_mcp, testing. Removed python.md (this is a Bun/JS project).
Added astro.md covering Astro 6 + Bun + Cloudflare Pages standards,
component conventions (scoped styles per-file), theme tokens, deploy
flow.

.context/:
- handoff.md — most recent session state (where we left off)
- plan.md — phase status + backend dependency map + worktree paths
- ideas.md — design decisions (Astro vs Next, ProvenanceToggle redesign,
  brain hero theming, hand-rolled SVG charts, etc.)
- research.md — verified API shapes, hallu QA layout, versioning
  policy, Cloudflare gotchas
- scratch_history.md — failed attempts + lessons (the Astro silent-
  render-drop, the sharp-in-worker fail, mock module bleed in Bun)
OpenNeuro source ids (ds*) now redirect to their NEMAR canonical (on*)
when a mirror exists in the catalog. /dataset/ds002718 lands on
/dataset/on002718 with the query string preserved; ds* without a mirror
fall through to the existing 404 path.

Adds resolveCanonical() helper around /datasets/resolve/:id with null
fallback on non-2xx so a transient backend failure leaves the request
on the original id instead of erroring.

Refs #17
…ntries

Re-released OpenNeuro datasets carry both the original and the canonical
id as IsDerivedFrom in metadata.json. on002718 was originally released
as ds000117 then re-released as ds002718; both ids appear in
related_identifiers and the old loop locked onto whichever came first
with ??= (ds000117 in this case).

Collect candidate matches into a list, then prefer the one whose
extracted ds id matches the canonical on->ds swap. Fall back to first
candidate when nothing matches, and to the swap-only path when there
are no IsDerivedFrom entries at all (existing behavior).

Refs #19
…nical-pick

fix(provenance): prefer canonical ds id when multiple IsDerivedFrom entries
feat: 301 redirect ds*->on* on dataset detail page
…eware

perf: edge-cache via caches.default in middleware
feat: NEMAR-branded favicon, apple+android icons, web manifest
feat: OG/Twitter/canonical meta + brand OG card
Discover cards previously showed 'by yahya' (the uploader) for every
dataset. When the catalog row is enriched with author names, prefer
those: 'by FirstAuthor' for one, 'by FirstAuthor et al.' for two or
more. Fall back to the uploader, then to 'NEMAR' when neither is known.

Adds formatAuthorByline() helper + 4 unit tests covering nullish,
single, multi, and degenerate-comma input.

Refs #22
- format.ts: replace verbose docstring with one-line invariant comment;
  the previous docstring duplicated the code (WHAT) and only the API
  contract assumption is non-obvious (WHY).
- DatasetCard.astro: drop inline comment over fallback chain; the
  three-term short-circuit is self-evident TypeScript.
- format.test.ts: add whitespace-only first-slot test ("  , Richard N
  Henson" -> "Richard N Henson") covering a previously-untested branch
  where the filter must drop blank entries before counting.
feat: prefer authors over uploader in DatasetCard byline
Reworked Phase 3 against the current epic architecture (deferred README +
BIDS tree fetching). Ships the data-quality half: pipeline success, three
percent-retained histograms, line-noise distribution.

Phase 3 originally expected the hallu sync to produce a pre-aggregated
aggregates.json + hed-summary.json. Hallu instead emits per-file
dataqual.json files at every leaf path under <id>/qa/. New
lib/qa-aggregate.ts walks the data.nemar.org listing tree recursively,
fetches per-file dataquals in parallel, and assembles QaAggregates at
request time. [id].astro's Cache-Control absorbs the fan-out cost for
repeat visits.

QualityPanel renders for datasets with QA in S3 today (31 datasets
synced 2026-05-18; on002718 renders 18 files end-to-end). For
datasets without QA, the panel hides itself gracefully.

Out of scope:
- HED wordcloud (depends on hed-summary.json which hallu doesn't produce)
- Demographics (depends on participants.tsv ingestion)
- FileVis/FileView modals (file-level click integration; will close
  #21 separately)

Closes #21 in part — file modals tracked as a follow-up.
Refs #11 (the original Phase 3 PR; supersedes its scope
half-A).
Fixes from code-reviewer, silent-failure-hunter, and pr-test-analyzer:

- Cap per-file fan-out at 25 parallel fetches (was unbounded). Large
  datasets with hundreds of files could saturate data.nemar.org or
  graze Cloudflare's worker subrequest ceiling.
- Surface truncated listings: console.warn when the listing endpoint
  reports it served a partial subtree, instead of silently producing
  an under-counted aggregate.
- Distinguish non-404 listing/file failures: console.warn on 5xx or
  parse errors so a worker-side regression is visible across all
  datasets instead of silently hiding QualityPanel.
- Warn when more than zero per-file fetches fail; an aggregate over
  20 of 100 files looks plausible but is misleading.
- Stricter test router: exact-URL match, so a regression that drops
  the trailing-slash append is caught by the mocked tests (the bare
  form 308-redirects to a broken /data/ prefix in production).
- New tests: session-structured (sub/ses/eeg) recursion, partial
  mid-walk 404, 'cleaning' branch (no ICA percent), 'other' branch
  (line-noise only, no percent fields).

107/107 tests pass, typecheck clean.
phase 3: QualityPanel with runtime aggregator from per-file dataqual
Cloudflare Pages CI runs 'npm install' which strict-checks peer deps,
unlike bun. @astrojs/cloudflare@12.x declares peerDependencies of
astro@^5.7.0; with astro@6.3.3 npm ERESOLVE-fails. v13.5.1 supports
astro@^6.3.0 so npm install succeeds.

Every CF Pages build for epic-branch PRs since the astro@6 bump has
failed at the 'Installing project dependencies' step. Our manual
'wrangler pages deploy dist' bypasses the install because we ship a
pre-built dist/, which is why the deployed previews worked.

Local build verified, 'npm install --dry-run' verified, no breaking
changes from the adapter major bump in our usage.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented May 18, 2026 •

Copy link
Copy Markdown

Deploying nemar-website with  Cloudflare Pages  Cloudflare Pages

Latest commit: 93e9698
Status: ✅  Deploy successful!
Preview URL: https://527269a7.nemar-website.pages.dev

View logs

…-deps

CF Pages had been failing every webhook-triggered build at the 'npm install'
step due to a peer-dep mismatch between astro@6.3.3 and
@astrojs/cloudflare@12.x (which declares peer astro@^5.7.0). Earlier
attempt to bump to v13.5.1 worked for npm install, but v13 is the
Workers adapter ('Deploy your site to Cloudflare Workers' per its npm
description) and emits a Workers-style wrangler.json that Pages rejects
(reserved ASSETS binding, KV missing id, empty triggers).

Keep v12 (still works fine with astro 6 in practice; just unsupported
per peer-dep range) and add .npmrc with legacy-peer-deps=true so CF
Pages's npm install proceeds. bun is unaffected -- it ignores strict
peer-deps by default, which is why our local builds and 'wrangler pages
deploy' calls have always worked.

When the Cloudflare adapter ships a Pages-compatible v13.x (or when we
migrate the site to Workers deployment), we can drop the .npmrc and
bump.
@neuromechanist
neuromechanist merged commit 07b0b0d into main May 18, 2026
1 check passed
neuromechanist added a commit that referenced this pull request May 18, 2026
Post-cutover hygiene pass per pr-review-toolkit findings on PR #26.

- Replace literal SCCN account id in AGENTS.md/CLAUDE.md/.context/*
  with env-var references so the repo doesn't carry org-internal infra
  data as a hardcoded literal. Audit confirmed no actual credentials,
  emails, or tokens were tracked; only the account id was a literal.
  Deploy snippets now require CLOUDFLARE_ACCOUNT_ID exported in the
  shell.
- Drop ProvenanceCard.astro -- superseded by ProvenanceToggle, no
  remaining imports.
- Add console.warn context to four silent catches so worker outages,
  GitHub rate-limits, and catalog backend regressions leave triageable
  tail-log entries instead of silently degrading every detail page:
  - getDataset(id) catch in [id].astro
  - getQaAggregates(id) catch in [id].astro
  - fetchManifestEntryText catch in data-api.ts (manifest README path)
  - fetchGithubReadme catch in data-api.ts (AggregateError flattened
    into per-URL detail)
- Update deploy branch reference in AGENTS.md from the epic branch
  to main now that the cutover has merged.

107/107 tests pass, typecheck clean, build clean.
@neuromechanist
neuromechanist deleted the feature/issue-1-epic-nemar-redesign branch September 30, 2026 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Epic: Redesign nemar.org on Cloudflare Pages + Astro, reusing data.nemar.org

1 participant