Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions src/openhuman/hosting/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ everything about a provider, and this module owns everything about OpenHuman.
- Resolve the configured hosting account (`Account::from_config`) from
`[hosting]`, falling back to the provider's own environment variables.
- Decide which directory an agent may deploy (`resolve_in_workspace`).
- Expose the six `hosting_*` agent tools and describe their results to a model.
- Expose the nine `hosting_*` agent tools and describe their results to a model.

Everything else — Vercel's endpoints, the upload-then-build deployment protocol,
how a marketplace database is provisioned and connected, the order a launch runs
Expand All @@ -21,7 +21,7 @@ mock of the provider's REST API. Nothing here knows the word `readyState`.
| File | Role |
| --- | --- |
| `mod.rs` | `Account` (credential resolution + the shared `dyn Host`) and `resolve_in_workspace`. |
| `tools.rs` | The six agent tools. |
| `tools.rs` | The nine agent tools. |
| `test.rs` | Account resolution, workspace containment, and each tool's contract. |

## Agent tools
Expand All @@ -30,11 +30,29 @@ mock of the provider's REST API. Nothing here knows the word `readyState`.
| --- | --- |
| `hosting_launch_site` | Deploys a workspace directory as a live site, optionally provisioning a database and wiring it in, setting environment variables, and attaching domains. External effect. |
| `hosting_deployment_status` | Whether a build has finished. Read-only. |
| `hosting_list_deployments` | A site's recent deployments, newest first, with status and target. Read-only. |
| `hosting_rollback` | Points production back at an earlier deployment that already built. External effect. |
| `hosting_list_sites` | The sites on the account. Read-only. |
| `hosting_set_env` | Sets environment variables on an existing site. External effect. |
| `hosting_add_domain` | Attaches a custom domain. External effect. |
| `hosting_domain_status` | Whether a site's domains are verified and serving. Read-only. |
| `hosting_analytics` | Traffic over the last N days. Read-only. |

### Why there is a rollback and no separate promote

`Host::promote` is both: a rollback *is* a promote of an older deployment, and
the crate models it once rather than twice. The tool is named for the reason an
agent reaches for it.

`hosting_rollback` reads the deployment before promoting it and refuses one that
did not finish building. `hosting_list_deployments` returns failed and
still-building deployments too — they are the history an agent is reading — so
the id it picks is not necessarily one that can serve traffic, and promoting a
failed build would take the site down during an attempt to bring it back up. It
does **not** check that the deployment belongs to the named site: a deployment is
looked up by id alone and the provider's response may omit the project name, so
that check would refuse legitimate rollbacks. The provider owns it.

## Gating

Two gates, and both matter:
Expand Down
6 changes: 4 additions & 2 deletions src/openhuman/hosting/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,11 @@
//!
//! # What the agent gets
//!
//! Six tools, in [`tools`]. `hosting_launch_site` is the one that matters: it
//! Nine tools, in [`tools`]. `hosting_launch_site` is the one that matters: it
//! turns a directory in the workspace into a live site with a database behind
//! it. The other five read or adjust what it produced.
//! it. `hosting_rollback` is its counterweight — it points production back at
//! an earlier deployment, so an agent that ships a broken site has a way back
//! rather than only a way forward. The rest read or adjust what they produced.
//!
//! # The credential
//!
Expand Down
265 changes: 264 additions & 1 deletion src/openhuman/hosting/test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -90,9 +90,12 @@ fn an_account_exposes_every_hosting_tool() {
[
"hosting_launch_site",
"hosting_deployment_status",
"hosting_list_deployments",
"hosting_rollback",
"hosting_list_sites",
"hosting_set_env",
"hosting_add_domain",
"hosting_domain_status",
"hosting_analytics",
]
);
Expand All @@ -109,7 +112,7 @@ fn only_the_tools_that_change_the_world_carry_an_external_effect() {
for tool in account.tools() {
let expected = matches!(
tool.name(),
"hosting_launch_site" | "hosting_set_env" | "hosting_add_domain"
"hosting_launch_site" | "hosting_set_env" | "hosting_add_domain" | "hosting_rollback"
);
assert_eq!(
tool.external_effect(),
Expand Down Expand Up @@ -246,3 +249,263 @@ async fn a_read_tool_reports_a_missing_argument_rather_than_calling_out() {

assert!(result.is_error);
}

// ── The deployment-history and rollback tools (issue opencompany#913) ────────

#[tokio::test]
async fn the_new_read_tools_report_a_missing_site_rather_than_calling_out() {
let workspace = tempfile::tempdir().expect("tempdir");
let config = config_with(workspace.path(), true, "token");
let account = Account::from_config(&config)
.expect("resolution does not fail")
.expect("an account");

let listed = tools::ListDeploymentsTool::new(account.host())
.execute(json!({}))
.await
.expect("the tool reports rather than panics");
assert!(listed.is_error);

let domains = tools::DomainStatusTool::new(account.host())
.execute(json!({}))
.await
.expect("the tool reports rather than panics");
assert!(domains.is_error);
}

#[tokio::test]
async fn a_rollback_missing_either_argument_is_refused_before_any_call() {
let workspace = tempfile::tempdir().expect("tempdir");
let config = config_with(workspace.path(), true, "token");
let account = Account::from_config(&config)
.expect("resolution does not fail")
.expect("an account");

// A rollback names two things and neither can be guessed: repointing
// production at an unnamed deployment is not a recoverable mistake.
for args in [
json!({}),
json!({"site": "shop"}),
json!({"deployment_id": "d1"}),
] {
let result = tools::RollbackTool::new(account.host())
.execute(args.clone())
.await
.expect("the tool reports rather than panics");
assert!(result.is_error, "{args} should have been refused");
}
}

// ── The rollback guard, against a mock of the provider's API ─────────────────

use std::sync::Arc as TestArc;

use wiremock::matchers::{method, path};
use wiremock::{Mock, MockServer, ResponseTemplate};

/// A real provider client pointed at a local mock rather than at Vercel.
///
/// `connect_to`'s own documentation names this case, and loopback `http://` is
/// accepted precisely so a test can carry a bearer token to a server that never
/// leaves the machine.
fn host_against(server: &MockServer) -> TestArc<dyn tinyhosts::Host> {
let base_url = server.uri();
TestArc::from(
tinyhosts::connect_to(
tinyhosts::ProviderKind::Vercel,
tinyhosts::Credentials::new("token").expect("credentials"),
Some(base_url.as_str()),
)
.expect("a client against the mock"),
)
}

/// **The guard that makes this tool a recovery path rather than a second way to
/// break the site.**
///
/// `hosting_list_deployments` returns failed and still-building deployments too
/// — they are part of the history an agent reads — so the id it picks is not
/// necessarily one that can serve traffic. Promoting a failed build would take
/// the site down during an attempt to bring it back up.
///
/// The assertion that matters is `expect(0)` on the promote route: the refusal
/// has to happen *before* the outward call, not be an error message reported
/// after production already moved.
#[tokio::test]
async fn rolling_back_to_a_deployment_that_never_built_does_not_touch_production() {
let server = MockServer::start().await;

Mock::given(method("GET"))
.and(path("/v13/deployments/dpl_broken"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"id": "dpl_broken",
"name": "shop",
"readyState": "ERROR",
"errorMessage": "build failed"
})))
.mount(&server)
.await;

// Vercel's promote is POST /v10/projects/{project}/promote/{deployment},
// reached only after GET /v9/projects/{site} resolves the id. Neither may
// be called at all.
Mock::given(method("GET"))
.and(path("/v9/projects/shop"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"id": "prj_1",
"name": "shop"
})))
.expect(0)
.mount(&server)
.await;

let result = tools::RollbackTool::new(host_against(&server))
.execute(json!({"site": "shop", "deployment_id": "dpl_broken"}))
.await
.expect("the tool reports rather than panics");

assert!(result.is_error, "a failed deployment must not be promoted");
// The status is named, so a model can pick a different deployment rather
// than retry the same one.
let rendered = result.text();
assert!(
rendered.contains("Failed"),
"the refusal should name the state it refused: {rendered}"
);
}

/// The ordinary path: a deployment that built is promoted, and the site's URL
/// comes back so the agent can say where production now points.
#[tokio::test]
async fn rolling_back_to_a_ready_deployment_promotes_it() {
let server = MockServer::start().await;

Mock::given(method("GET"))
.and(path("/v13/deployments/dpl_good"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"id": "dpl_good",
"name": "shop",
"url": "shop-abc.vercel.app",
"readyState": "READY",
"target": "production"
})))
.mount(&server)
.await;
Mock::given(method("GET"))
.and(path("/v9/projects/shop"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"id": "prj_1",
"name": "shop"
})))
.mount(&server)
.await;
Mock::given(method("POST"))
.and(path("/v10/projects/prj_1/promote/dpl_good"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({})))
.expect(1)
.mount(&server)
.await;

let result = tools::RollbackTool::new(host_against(&server))
.execute(json!({"site": "shop", "deployment_id": "dpl_good"}))
.await
.expect("the tool reports rather than panics");

assert!(!result.is_error, "{result:?}");
let rendered = result.text();
assert!(
rendered.contains("dpl_good") && rendered.contains("shop-abc.vercel.app"),
"the result should say what is serving and where: {rendered}"
);
}

/// The history read an agent uses to *find* the id above.
#[tokio::test]
async fn listing_deployments_reports_the_history_a_rollback_picks_from() {
let server = MockServer::start().await;

Mock::given(method("GET"))
.and(path("/v7/deployments"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"deployments": [
{"id": "dpl_new", "name": "shop", "readyState": "ERROR"},
{"id": "dpl_old", "name": "shop", "readyState": "READY"}
]
})))
.mount(&server)
.await;

let result = tools::ListDeploymentsTool::new(host_against(&server))
.execute(json!({"site": "shop"}))
.await
.expect("the tool reports rather than panics");

assert!(!result.is_error, "{result:?}");

// Both are reported, and each status is bound to its id. The failed one is
// why the agent is here and the ready one is where it is going, so a list
// that returned the right ids against the wrong statuses would send the
// rollback at the deployment that just broke the site.
let deployments: serde_json::Value =
serde_json::from_str(&result.text()).expect("the tool answers with JSON");
let rows = deployments.as_array().expect("an array of deployments");

assert_eq!(rows.len(), 2, "{deployments}");
// Newest first, as the crate documents and as the provider returned them.
assert_eq!(rows[0]["id"], "dpl_new", "{deployments}");
assert_eq!(rows[0]["status"], "failed", "{deployments}");
assert_eq!(rows[1]["id"], "dpl_old", "{deployments}");
assert_eq!(rows[1]["status"], "ready", "{deployments}");
}

/// A domain that is attached but unverified is not serving, and the tool has to
/// say so — that difference is the entire reason to read domains.
#[tokio::test]
async fn domain_status_distinguishes_a_verified_domain_from_a_pending_one() {
let server = MockServer::start().await;

Mock::given(method("GET"))
.and(path("/v9/projects/shop/domains"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"domains": [
{"name": "shop.example.com", "verified": true},
{"name": "www.example.com", "verified": false}
]
})))
.mount(&server)
.await;

let result = tools::DomainStatusTool::new(host_against(&server))
.execute(json!({"site": "shop"}))
.await
.expect("the tool reports rather than panics");

assert!(!result.is_error, "{result:?}");

// Bound to the name rather than checked for presence: asserting only that a
// `true` and a `false` both appear somewhere passes just as happily if the
// two are swapped, which is the one thing this tool must not get wrong.
let domains: serde_json::Value =
serde_json::from_str(&result.text()).expect("the tool answers with JSON");
let verified_for = |name: &str| -> bool {
let entry = domains
.as_array()
.expect("an array of domains")
.iter()
.find(|domain| domain["name"] == name)
.unwrap_or_else(|| panic!("{name} is missing from {domains}"));
entry["verified"]
.as_bool()
.expect("`verified` is a boolean")
};

assert!(
verified_for("shop.example.com"),
"the verified domain must report verified: {domains}"
);
assert!(
!verified_for("www.example.com"),
"and the pending one must not — that difference is the entire reason to \
read domains: {domains}"
);
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
Loading
Loading