Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,20 +21,30 @@ review bots read and write at the same time, written in Rust.
the HTTP routes (`http.rs`) and the command line's commands that the API
offers only translate into its calls, so neither duplicates the other.
`sync.rs` shares records with a remote (`remote.rs`), which answers the
same calls over the SDK or, for a local path, in-process.
same calls over the SDK or, for a local path, in-process. Every call
takes a `Caller`, authenticated by `auth.rs`; `permissions.rs` filters
what it reads and checks what it writes, by the rights `policy.rs`
resolves from `policy.toml`.
- `tools/xtask`: development tasks, run with `cargo xtask`, such as timing
origin against git (`just bench-git`).

## API changes

Every server feature lives in `OriginService`; transports like HTTP only
translate requests. A new HTTP endpoint also takes a line in the SDK's
`endpoints!` macro, and `just openapi` to update `docs/openapi.json`.

## Testing & Benchmarks

Performance is a primary goal, so benchmarks matter as much as tests. A change
to a hot path (hashing, storage, snapshots, transfer) should come with a
benchmark, and its numbers in the pull request.

- `just bench`: the divan micro-benchmarks (encoding, store writes,
snapshots, input digests and promotion, landing, reading events, the
server's request latency and long-poll wakeups, syncing a 20,000-file
tree over HTTP, and a fetch that brings nothing from a long history).
snapshots, input digests and promotion, landing, permission checks,
reading events, the server's request latency and long-poll wakeups,
syncing a 20,000-file tree over HTTP, and a fetch that brings nothing
from a long history).
- `just bench-git`: times origin against git on a 20,000-file tree with
hyperfine, and rewrites `docs/benchmarks.md`.

Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

73 changes: 68 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ origin remote [<url>] # show or set the remote: http://host:port/<
origin push [<change>...] # share changes and the ones below them; lands nothing
origin fetch [--target <t>] [--change <id>] # new acceptances, and the records of known changes
origin pull # fetch, then move the working copy onto its target's tip
origin whoami | permissions # who the remote thinks you are, and what you may do there
origin token issue --scope "write src" [--ttl 1h] # a short-lived token for an agent
```

Landing rebases a change onto the target's tip, file by file, checks the
Expand Down Expand Up @@ -110,9 +112,64 @@ Changes are named by a unique prefix of their id, or `@` for the working
copy's. A revision is `@`, a change (for its head), or a unique prefix of
the revision's id. Your name and node key live in `$ORIGIN_CONFIG_DIR`, or
else `origin` in `$XDG_CONFIG_HOME` or `~/.config`: set `user = "<name>"`
in its `config.toml` if `$USER` isn't the name you want. See
in its `config.toml` if `$USER` isn't the name you want, and the token to
send each server under `[tokens]`, by its URL
(`"http://127.0.0.1:7878" = "<token>"`), or in `$ORIGIN_TOKEN`. See
[the design](docs/design.md) for the model.

## Permissions

What each user may do lives in `policy.toml`, so changing it is a landing
on `policy`, which only `admin` may make:

```toml
[grants] # capabilities: annotate, land, admin
"group:maintainers" = ["admin", "land *", "annotate review"]
"group:runners" = ["annotate ci.*"]

[[paths]] # path access: none, read or write
path = "secrets"
grants = { all = "none", "group:maintainers" = "write" }

[[paths]] # CI's checks: only maintainers redefine them
path = "checks.toml"
grants = { all = "read", "group:maintainers" = "write" }
```

Under a path, the most specific rule that mentions you decides, and paths
no rule mentions you for are writable. Without `[grants]`, everyone lands
on every target but `policy` and issues every kind of statement. Content
you may not read never reaches your clone: those files are missing from
the working copy and stay as they were in your revisions, and fetching
restores them once you may read them. Revisions that only change what you
can't read, and the statements about them, are hidden from you, and you
may not issue statements about them, nor promote their results. Reading
nothing hides the repository whole. You may only push revisions that
change paths you write, and land what changes only paths you write. You
may take over someone else's change, and hand it on: where you can't
write, your revision must do exactly what theirs did, the same files
changed from the same contents to the same ones, with exactly the same
conflicts unresolved, on the same change below, and a new change lists
there only those of the revision it's on. Only a new change gets a root
revision. Anyone who reads a change, even with a token that only reads,
may still copy its head as it is: the change then diverges until someone
converges it again. The statements you push need their kind's `annotate`, promotions
included, but for the reserved `core.*` kinds, which come only from whom
core names: `core.abandon` from the change's creator, who made its root
revision, or an `admin`, so you can't abandon a change you took over;
`core.conflict` from a revision's author, or a node. Each user of a clone,
and each scope of their tokens, keeps their own rights on the remote, as
of their last fetch, so workspaces of one clone can act for different
users, or agents.

`origin token issue --scope <capability>... --ttl <period>` asks the remote
for a token acting for you with a narrower scope, such as `write src`,
`read /`, `annotate ci.*`, `land main` or `admin`; the server verifies it
without storing it. A server's operator signs one for any principal with
`origin token issue --config origin-server.toml --as user:agent-1 --scope
...`. A token never grants more than its principal's rights, lasts at most
7 days, and a token issued with a token expires with it.

`converge` merges the files of a change's heads. A text file two heads
changed holds both versions between conflict markers (`<<<<<<<`, `=======`,
`>>>>>>>`), and a binary one keeps one side. A `core.conflict` statement
Expand Down Expand Up @@ -142,23 +199,29 @@ key = "node.key" # its secret key, created if missing

[repos]
demo = "repos/demo"

[tokens] # static tokens, by principal
"user:alice" = "a long random secret"
```

The HTTP API, under `/api/v1`, puts records in batches, if the changes
they revise are where the client saw them, and fetches them by id, walks
snapshots for the trees and blobs a client lacks (the only way files leave
a server), in parts of at most 64 MiB each way, compares the records of
changes, lists statements, revisions, targets and their logs, lands
changes and targets, and streams
changes, lists statements,
revisions, targets and their logs, lists the directories and reads the
files of a target's tip or any revision, without its history, issues
scoped tokens, lands changes and targets, and streams
events by long-poll: `GET
/api/v1/repos/<repo>/events?cursor=<n>&wait=30s` answers as soon as there
are events after the cursor, with a JSON batch, the next cursor, and the
generation of the op log it's of: pass it back as `generation`, and a
cursor of another log, such as from before the repository was restored,
is refused as expired.
[`docs/openapi.json`](docs/openapi.json) describes it, and
[`origin-client`](crates/origin-client) is its Rust SDK. Nothing
authenticates callers yet.
[`origin-client`](crates/origin-client) is its Rust SDK. Callers send
`Authorization: Bearer <token>`; without one, they are anonymous, and only
read what everyone may.

## Development

Expand Down
7 changes: 6 additions & 1 deletion apps/origin-src/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,14 @@ fixture = []
[dependencies]
anyhow.workspace = true
axum.workspace = true
blake3.workspace = true
clap.workspace = true
getrandom.workspace = true
origin-client.workspace = true
origin-platform.workspace = true
origin-shared.workspace = true
serde.workspace = true
serde_json.workspace = true
similar.workspace = true
thiserror.workspace = true
tokio.workspace = true
Expand All @@ -38,7 +40,6 @@ zstd.workspace = true
divan.workspace = true
# The benchmarks build their working trees with the fixtures.
origin-src = { workspace = true, features = ["fixture"] }
serde_json.workspace = true
tempfile.workspace = true

[[bench]]
Expand All @@ -49,6 +50,10 @@ harness = false
name = "land"
harness = false

[[bench]]
name = "permissions"
harness = false

[[bench]]
name = "server"
harness = false
Expand Down
6 changes: 3 additions & 3 deletions apps/origin-src/benches/annotations.rs
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@ fn promotions_cached(bencher: Bencher<'_, '_>) {
let mut annotations = Annotations::open(&store).unwrap();
assert_eq!(
annotations
.promotions(RecordId::Revision(target), &issuer)
.promotions(RecordId::Revision(target), &issuer, |_| Ok(true))
.unwrap()
.len(),
9
Expand All @@ -193,7 +193,7 @@ fn promotions_cached(bencher: Bencher<'_, '_>) {
.with_inputs(|| Annotations::new(&store, index.clone()))
.bench_local_refs(|annotations| {
annotations
.promotions(RecordId::Revision(target), &issuer)
.promotions(RecordId::Revision(target), &issuer, |_| Ok(true))
.unwrap()
});
}
Expand All @@ -214,7 +214,7 @@ fn promotions_uncached(bencher: Bencher<'_, '_>) {
})
.bench_local_refs(|annotations| {
annotations
.promotions(RecordId::Revision(target), &issuer)
.promotions(RecordId::Revision(target), &issuer, |_| Ok(true))
.unwrap()
});
}
2 changes: 1 addition & 1 deletion apps/origin-src/benches/land.rs
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ impl Repo {
author: "user:bench".parse().unwrap(),
}
.encode();
let description = EncodedObject::blob(b"");
let description = EncodedObject::attachment(b"");
self.store.write_all(&[content, tree, description]).unwrap();
self.store.write_all(&[revision]).unwrap();
change
Expand Down
Loading
Loading