Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -323,10 +323,13 @@ jobs:
# pnpm-lock.yaml and has them installed in its own folder, so the
# certification below typechecks, tests and builds against exactly
# those versions. Every lockfile, not a list: nothing to keep in sync.
# A drive app whose plugin folder is not a certified sandbox package
# (issue-tracker) keeps its lockfile in app/, where certify.mjs does not
# look for a package.json.
- name: Install plugin npm dependencies
working-directory: atomic-plugins
run: |
for lock in integrations/*/pnpm-lock.yaml; do
for lock in integrations/*/pnpm-lock.yaml integrations/*/app/pnpm-lock.yaml; do
[ -e "$lock" ] || continue
(cd "$(dirname "$lock")" && pnpm install --frozen-lockfile)
done
Expand Down Expand Up @@ -446,7 +449,7 @@ jobs:
- name: Install plugin npm dependencies
working-directory: atomic-plugins
run: |
for lock in integrations/*/pnpm-lock.yaml; do
for lock in integrations/*/pnpm-lock.yaml integrations/*/app/pnpm-lock.yaml; do
[ -e "$lock" ] || continue
(cd "$(dirname "$lock")" && pnpm install --frozen-lockfile)
done
Expand Down
111 changes: 111 additions & 0 deletions integrations/issue-tracker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ code (`adapter.ts`, `tracker-actions.ts`) stays here.

This folder also holds the plugin's other issue sources:

- `app/`: the **GitHub issues drive app**, the current host for the Devonian
bridge (#100). See [Drive app](#drive-app-app) below.
- `devonian/github-issues/`: the in-browser Devonian GitHub issues lens and
bridge (moved from the `devonian` package; see its README).
- `todoist.ts`: the read-only LocalThought projection of Todoist tasks onto
Expand Down Expand Up @@ -74,13 +76,122 @@ all, neither the projection nor this check (searched `browser/` and
host runs today. The host journey that runs the check, and exercises these
states end to end, is still to be built.

## Drive app (`app/`)

An iframe drive app, the same shape as `pets/app/` and `notion/app/`: one
ES module (`app/build.mjs` -> `dist/ui.js`, about 86 KB) whose
`view({ root, store })` runs in the host's null-origin frame. It hosts the
Devonian bridge from `devonian/github-issues/` for **one repository per app
install**, two-way for issue title, body (Markdown), Todo/Doing/Done status
and comments.

**Flow.** "Connect GitHub" asks the host for a connection
(`store.proxy.connect`, the host's consent bar, platform `github-issues`).
The app then asks for the repository as `owner/name` (typed; there is no
repository picker, see below), creates its columns and binds the repository
to the app for good. Sync runs when the view opens and on "Sync now".

**What it writes, and where.** Everything goes into the app's own subtree
(the only place a drive app may write): the Status select (Todo / Doing /
Done tags), GitHub issue number and GitHub source columns under the app's
ontology; one row per issue in the app's table, the body in Atomic's own
`description`; one Message per comment (`about` its row) in a "GitHub
comments" folder under the app; and one sync resource holding the bound
repository and the sync state as JSON text.

**Review before provider writes.** A pass never sends a create or update to
GitHub on its own. Every change the Bridge would send is held
(`devonian/github-issues/review.mjs`) and listed ("Update #1: status Todo →
Done (close it)"); "Send N changes to GitHub" approves exactly that content
for one pass. A change edited after review is held again. Imports into the
table are not gated, as for pets and notion. This is the app-level boundary
only: named actions, MCP exposure and the host-side approval journal of
[ACTIONS.md](../ACTIONS.md) remain #11's design, and this app does not
replace them.

**Recovery.**

- Same field changed on both sides since the last sync: sync pauses with the
fields named; "Keep GitHub's version" / "Keep this table's version"
settles only those fields (`Bridge.resolveConflict`) and syncs again. Keeping
this table's side becomes a held write, reviewed like any other.
- GitHub answers 401, or the host no longer has the connection: "Reconnect
GitHub". The refused connection is not offered again after the reload.
- A write whose response was lost: the next pass reads GitHub back first. If
the write landed, the operation just completes. If not, an update is offered
for review again, marked as unconfirmed; a create is never resent (the
transport's journal refuses: "Uncertain GitHub write"), and sync stays
paused. There is no in-app way out of that yet (design state 12), nor for a
record missing on one side (state 13): both need a person and a follow-up.
- Anything else fails the pass and "Sync now" retries it.

**Host behaviour it relies on or works around** (atomic-server `bae5cdbe3`,
read in `hostStore.ts`, `proxyConnections.ts`, `collection.ts`;
`app/frameStore.ts` has the detail):

- The rotating connection code never reaches the frame: the bridge's
`proxyTransport` runs with a `dispatch` over `store.proxy.request`, and its
write journal still guards uncertain writes. Host refusals that happen
before anything is sent are recognised by message and are not uncertain.
- A resolved `save`/`newResource` is an acknowledged `/app-write` commit.
- The frame's reads come from the host page's cache, which did **not** show
the app's own save within 5 s in the e2e. The adapter corrects its reads
for its own saves (per view) instead of waiting.
- `query` is answered from the page's local index first and can miss fresh
resources; subjects the app has seen are remembered in its sync state.
- No IndexedDB, localStorage or Web Locks in the frame, so
`background.mjs` is not used and nothing runs while the app is closed.

**Not verified, or not supported:**

- Only against the mock proxy's seeded repository (`atomic-fixture/tracker`:
two issues, one comment). Nothing has run against live GitHub, the real
integration proxy, or a repository beyond a handful of issues. The
Collection pages at 500; larger repositories are not tested.
- Two tabs or devices syncing the same app at once are not guarded: the sync
state syncs with the drive and `/app-write` has no compare-and-swap. One
syncing view per app is assumed. (The design's alternative, a per-browser
host storage op, needs atomic-server work.)
- The sync state grows with the write journal; it is not pruned.
- Comments made in the data-browser's own comment panel on a row are not
synced: they live outside the app's subtree, which the app cannot write.
- No repository picker: listing a token's repositories needs
`GET /user/repos` in the proxy's GitHub document, which is not verified.
Changing the bound repository means a new app.
- Labels other than `atomic:doing`, assignees, milestones and pull requests
are not synced; deletion on either side is never propagated.
- The view is a status line, the review list, the conflict choice and a
plain issue list. The designed board/list/detail views are pending #89
(design PR #103, not approved).
- **Install.** No catalog install flow for drive apps exists yet (#94), so
there is no catalog entry: one would advertise a runtime a user cannot
reach. The e2e installs the app test-side, as pets' and notion's do.

`app/package.json` pins `devonian@0.7.0` from npm (install it with
`pnpm install --frozen-lockfile` in `app/`), bundled as `devonian/atomic` plus
`reconcileRecord`; `@tomic/lib` is shimmed as in notion. It lives in `app/`
rather than here because `certify.mjs` treats a `package.json` in a plugin
folder as a sandbox package. `syncables` is not used: the Bridge's GitHub
port already pages GitHub, and bundling the GitHub OpenAPI document for
syncables would only add size.

## Verification

```sh
./browser/node_modules/.bin/vitest run --config integrations/issue-tracker/vitest.config.ts
./browser/node_modules/.bin/tsc -p integrations/issue-tracker/tsconfig.json
./browser/node_modules/.bin/tsc -p integrations/issue-tracker/app/tsconfig.json
node integrations/tooling/run-lane.mjs issue-tracker --tier e2e
```

The drive app's unit tests (`app/sync.test.ts`) run against an in-memory
host (`app/fakeStore.ts`) and the same GitHub fixture the mock proxy
serves, including a host whose reads never show the app's own saves;
`app/build.test.ts` checks the bundle and typechecks `app/`. The e2e
(`e2e/issue-tracker.spec.ts`) covers connect, import, reload with an
unchanged refresh, one reviewed update (closing #1) and a title conflict
settled for GitHub's side.

These check `adapter.ts`'s pagination, PR exclusion and mapping, the generic
event-to-JavaScript starter (`automation.test.ts`), the Todoist projection, and
every test under `devonian/github-issues/`. The latter import the `devonian`
Expand Down
64 changes: 64 additions & 0 deletions integrations/issue-tracker/app/build.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
// @wc-ignore-file
/**
* Builds the GitHub issues drive app to one ES module, `dist/ui.js`: the
* string an install flow stores as the App's `plugin-source`.
*
* node integrations/issue-tracker/app/build.mjs [--outfile path]
*
* Needs an atomic-server checkout's `browser/` beside `integrations/` (see
* AGENTS.md), for esbuild, and `pnpm install --frozen-lockfile` in
* integrations/issue-tracker/app/ for the npm `devonian` it bundles. The
* lens in `../devonian/github-issues/` resolves `devonian` from there too
* (`nodePaths`), not from this repo's `devonian/` sources.
*/
import { createHash } from 'node:crypto';
import { createRequire } from 'node:module';
import { fileURLToPath } from 'node:url';
import { mkdirSync, writeFileSync } from 'node:fs';
import { dirname } from 'node:path';

const path = relative => fileURLToPath(new URL(relative, import.meta.url));

/** Bundles in memory; writes only when `outfile` is given. */
export async function build({ outfile } = {}) {
const require = createRequire(path('../../../browser/package.json'));
const esbuild = require('esbuild');
const result = await esbuild.build({
entryPoints: [path('main.ts')],
bundle: true,
format: 'esm',
platform: 'browser',
target: 'es2022',
splitting: false,
legalComments: 'none',
write: false,
outfile: outfile ?? path('dist/ui.js'),
nodePaths: [path('node_modules')],
alias: {
'@tomic/lib': path('tomic-lib-shim.ts'),
// adapter.ts takes reconcileRecord from the host module the sandbox
// plugin was built against; devonian exports the same function.
'@integration-host/plugin-reconcile': 'devonian',
},
logLevel: 'silent',
});
const text = result.outputFiles[0].text;

if (outfile) {
mkdirSync(dirname(outfile), { recursive: true });
writeFileSync(outfile, text);
}

return {
text,
bytes: Buffer.byteLength(text),
sha256: createHash('sha256').update(text).digest('hex'),
};
}

if (process.argv[1] === fileURLToPath(import.meta.url)) {
const at = process.argv.indexOf('--outfile');
const outfile = at > 0 ? process.argv[at + 1] : path('dist/ui.js');
const out = await build({ outfile });
console.info(`${outfile}: ${out.bytes} bytes, sha256 ${out.sha256}`);
}
103 changes: 103 additions & 0 deletions integrations/issue-tracker/app/build.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
// @wc-ignore-file
import { spawnSync } from 'node:child_process';
import { fileURLToPath } from 'node:url';
import { core, dataBrowser, Datatype, validateDatatype } from '@tomic/lib';
import { describe, expect, it } from 'vitest';
// @ts-expect-error build.mjs is plain JS with no declaration file.
import { build } from './build.mjs';
import * as shim from './tomic-lib-shim.js';

describe('GitHub issues drive-app bundle', async () => {
const { text, bytes } = (await build()) as { text: string; bytes: number };

it('is one self-contained ES module exporting view()', async () => {
expect(text).not.toMatch(/^\s*import\s/m);
expect(text).not.toMatch(/\bimport\(/);
expect(text).not.toMatch(/\brequire\(/);
const mod = await import(
`data:text/javascript;base64,${Buffer.from(text).toString('base64')}`
);
expect(Object.keys(mod)).toEqual(['view']);
expect(typeof mod.view).toBe('function');
// Stored as a string property on a resource; keep an eye on the size.
expect(bytes).toBeLessThan(128 * 1024);
});

it('has no storage or network access of its own', () => {
expect(text).not.toMatch(/localStorage|sessionStorage|indexedDB/);
expect(text).not.toMatch(/\bfetch\(/);
expect(text).not.toMatch(/navigator\.locks/);
expect(text).not.toMatch(/node:/);
// proxyTransport's own code-spending branch is bundled but unreachable:
// sync.ts always passes `dispatch`, and its getCode/setCode throw.
expect(text).toMatch(/The host holds the connection code/);
});

it("bundles devonian's Atomic Data API and reconcileRecord only", () => {
expect(text).toMatch(/Conflicting identity mapping/);
expect(text).not.toMatch(/automerge/i);
expect(text).not.toMatch(/DevonianClient|DevonianTable|BackgroundSync/);
});

it("uses the real library's values through the shim", () => {
for (const [key, value] of Object.entries(shim.Datatype))
expect(Datatype[key as keyof typeof Datatype]).toBe(value);
for (const [key, value] of Object.entries(shim.core.properties))
expect(core.properties[key as keyof typeof core.properties]).toBe(value);
for (const [key, value] of Object.entries(shim.core.classes))
expect(core.classes[key as keyof typeof core.classes]).toBe(value);
for (const [key, value] of Object.entries(shim.dataBrowser.properties))
expect(
dataBrowser.properties[key as keyof typeof dataBrowser.properties],
).toBe(value);
for (const [key, value] of Object.entries(shim.dataBrowser.classes))
expect(dataBrowser.classes[key as keyof typeof dataBrowser.classes]).toBe(
value,
);
});

it("validates like the real library's validateDatatype", () => {
const samples = ['text', '', 0, 1.5, 3, true, false, ['a'], {}];

const outcome = (fn: () => void) => {
try {
fn();

return 'ok';
} catch (error) {
return (error as Error).message;
}
};

for (const datatype of [
Datatype.STRING,
Datatype.MARKDOWN,
Datatype.INTEGER,
Datatype.FLOAT,
Datatype.BOOLEAN,
Datatype.TIMESTAMP,
Datatype.JSON,
])
for (const value of samples)
expect(outcome(() => shim.validateDatatype(value, datatype))).toBe(
outcome(() => validateDatatype(value as never, datatype)),
);
});
});

describe('GitHub issues drive-app types', () => {
// vitest strips types without checking them; the lane has no typecheck
// tier for app/, so its tsconfig is checked here.
it('typechecks', () => {
const tsc = fileURLToPath(
new URL('../../../browser/node_modules/.bin/tsc', import.meta.url),
);
const result = spawnSync(
tsc,
['-p', fileURLToPath(new URL('tsconfig.json', import.meta.url))],
{ encoding: 'utf8' },
);
expect(result.stdout + result.stderr).toBe('');
expect(result.status).toBe(0);
}, 120_000);
});
Loading
Loading