Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ review bots read and write at the same time, written in Rust.
server. `OriginService` (`service.rs`) is the one API over a repository:
the HTTP routes (`http.rs`) and the command line's commands that the API
offers only translate into its calls, so neither duplicates the other.
`sync.rs` shares records with a remote (`remote.rs`), which answers the
same calls over the SDK or, for a local path, in-process.
- `tools/xtask`: development tasks, run with `cargo xtask`, such as timing
origin against git (`just bench-git`).

Expand All @@ -30,15 +32,18 @@ to a hot path (hashing, storage, snapshots, transfer) should come with a
benchmark, and its numbers in the pull request.

- `just bench`: the divan micro-benchmarks (encoding, store writes,
snapshots, input digests and promotion, landing, reading events, and the
server's request latency and long-poll wakeups).
snapshots, input digests and promotion, landing, reading events, the
server's request latency and long-poll wakeups, syncing a 20,000-file
tree over HTTP, and a fetch that brings nothing from a long history).
- `just bench-git`: times origin against git on a 20,000-file tree with
hyperfine, and rewrites `docs/benchmarks.md`.

End-to-end tests run the `origin` binary with `ORIGIN_CONFIG_DIR` set to a
temporary directory, which holds the user's `config.toml` and node key.
The HTTP tests drive every SDK method against a real server. One fails
while `docs/openapi.json` is stale: `just openapi` rewrites it.
while `docs/openapi.json` is stale: `just openapi` rewrites it. The sync
tests clone, push and land through `origin serve` and through local paths;
`tests/common` holds the helpers every end-to-end test shares.

## Commits & Pull Requests

Expand Down
42 changes: 37 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,13 @@ origin workspace list | rm <path>

origin op log [-n <count>] # what this node did, operation by operation, in every workspace
origin undo # move this workspace's working copy back one operation
origin watch [--kind <kind>] [--subject <id>] [--since <cursor>] # events, as they happen
origin watch [--kind <kind>] [--subject <id>] [--since <cursor>] [--local] # events, as they happen

origin clone <url> [<path>] [--target <target>] # follow a remote's target, starting a change on its tip
origin remote [<url>] # show or set the remote: http://host:port/<repo>, or a path
origin push [<change>...] # share changes and the ones below them; lands nothing
origin fetch [--target <t>] [--change <id>] # new acceptances, and the records of known changes
origin pull # fetch, then move the working copy onto its target's tip
```

Landing rebases a change onto the target's tip, file by file, checks the
Expand Down Expand Up @@ -81,6 +87,25 @@ op log, one a line starting with a cursor: `statement.issued`,
takes an event name or a statement kind (`core.conflict`, `ci.*`), and
`--since` resumes after a cursor.

A repository can have one remote, a server or another repository's path,
shared by its workspaces. A clone holds the chains of the targets it
follows, `policy` and one other, verified against their authorities' keys,
and the records of the changes it knows, which each `fetch` brings as they
move: it asks the remote's events what moved since the last fetch, so a
fetch that brings nothing is quick however long the history. It fetches
snapshots only when a command reads them, starting with its target's tip.
`push` shares revisions, their files and statements, and refuses a change
the remote has a newer revision of, even one pushed while it sends:
`fetch`, then `converge` it. A statement that needs one the remote lacks
stays, and the push says so. A landing is decided on the evidence the
remote holds when it's tried: send a statement together with the one it
supersedes, as `push` does, or a landing waiting on the first may take it.
`land`, `land --from` and `target
new` go to the remote's queue for every target the remote has, after pushing
what lands, and `watch` long-polls the remote's events (`--local` for this
node's). Fetching never changes your files; `pull` then moves the working
copy onto its target's new tip, like a landing elsewhere does.

Changes are named by a unique prefix of their id, or `@` for the working
copy's. A revision is `@`, a change (for its head), or a unique prefix of
the revision's id. Your name and node key live in `$ORIGIN_CONFIG_DIR`, or
Expand Down Expand Up @@ -119,11 +144,18 @@ key = "node.key" # its secret key, created if missing
demo = "repos/demo"
```

The HTTP API, under `/api/v1`, puts and fetches records in batches, lists
statements, revisions, targets and their logs, lands changes and targets,
and streams events by long-poll: `GET
The HTTP API, under `/api/v1`, puts records in batches, if the changes
they revise are where the client saw them, and fetches them by id, walks
snapshots for the trees and blobs a client lacks (the only way files leave
a server), in parts of at most 64 MiB each way, compares the records of
changes, lists statements, revisions, targets and their logs, lands
changes and targets, and streams
events by long-poll: `GET
/api/v1/repos/<repo>/events?cursor=<n>&wait=30s` answers as soon as there
are events after the cursor, with a JSON batch and the next cursor.
are events after the cursor, with a JSON batch, the next cursor, and the
generation of the op log it's of: pass it back as `generation`, and a
cursor of another log, such as from before the repository was restored,
is refused as expired.
[`docs/openapi.json`](docs/openapi.json) describes it, and
[`origin-client`](crates/origin-client) is its Rust SDK. Nothing
authenticates callers yet.
Expand Down
6 changes: 5 additions & 1 deletion apps/origin-src/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ anyhow.workspace = true
axum.workspace = true
clap.workspace = true
getrandom.workspace = true
origin-client.workspace = true
origin-platform.workspace = true
origin-shared.workspace = true
serde.workspace = true
Expand All @@ -35,7 +36,6 @@ zstd.workspace = true

[dev-dependencies]
divan.workspace = true
origin-client.workspace = true
# The benchmarks build their working trees with the fixtures.
origin-src = { workspace = true, features = ["fixture"] }
serde_json.workspace = true
Expand Down Expand Up @@ -65,6 +65,10 @@ harness = false
name = "packs"
harness = false

[[bench]]
name = "sync"
harness = false

[[bench]]
name = "workspace"
harness = false
Expand Down
Loading
Loading