Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions TESTING_COVERAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -644,6 +644,8 @@ Both matter because `iroh_transport` holds the router and node identity in
| Paired peer + its relay/direct addresses survive a kill | `lib/tests/identity_durability.rs` |
| Two whole servers pair via `POST /iroh-sync` and reconcile | `server/tests/it/iroh_pairing.rs` |
| `/iroh-sync` refuses malformed node ids with a UI-showable error | `server/tests/it/iroh_pairing.rs` |
| A peer confirms a drive with its stored file, and disconnecting during the next file write preserves the previous full sync date | `lib/src/sync/iroh_e2e.rs` |
| Sync settings shows the selected drive's confirmed date for one peer and sync activity for another | `browser/e2e/tests/peer-full-sync-status.spec.ts` |

### Glue

Expand Down
81 changes: 64 additions & 17 deletions browser/data-browser/src/helpers/managedServer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ export type ServerPeer = {
deviceName: string | null;
/** Whether it holds a connection to the server right now. */
live: boolean;
/** Unix millis of the last successful sync, if it has ever synced. */
/** Unix milliseconds of the latest sync activity reported for this link. */
lastSeen?: number;
/** Resources moved by that last sync, each way. Not lifetime totals. */
lastSent?: number;
Expand Down Expand Up @@ -346,6 +346,67 @@ export type NodeDriveUsage = {
managedBy?: string;
};

/** Unix milliseconds of the last full sync, keyed by peer node ID. */
export type PeerFullSyncs = Partial<Record<string, number>>;

async function fetchDriveUsageData(
serverUrl: string,
driveSubject: string,
agent: Agent,
peerFullSyncsOnly = false,
) {
if (!serverUrl || !driveSubject || !agent?.subject) return null;

try {
const url = new URL('/drive-usage', serverUrl);
url.searchParams.set('subject', driveSubject);
if (peerFullSyncsOnly) url.searchParams.set('peerFullSyncsOnly', 'true');

// Sign the full URL, including the query the server uses for verification.
const headers = await signRequest(url.toString(), agent, {
Accept: 'application/json',
});
const res = await fetch(url.toString(), { headers });

return res.ok ? await res.json() : null;
} catch {
return null;
}
}

/** Verified peer dates for the drive, read from the node that connected to them. */
export async function fetchPeerFullSyncs(
serverUrl: string,
driveSubject: string,
agent: Agent,
): Promise<PeerFullSyncs | null | undefined> {
const data = await fetchDriveUsageData(serverUrl, driveSubject, agent, true);

if (data === null) return null;

const dates: unknown = data?.peerFullSyncs;
// Older servers return ordinary usage data after ignoring the query flag.
if (dates === undefined) return undefined;

const result: PeerFullSyncs = {};

if (dates && typeof dates === 'object') {
for (const [nodeId, date] of Object.entries(dates)) {
if (
/^[0-9a-f]{64}$/.test(nodeId) &&
typeof date === 'number' &&
Number.isSafeInteger(date) &&
date > 0 &&
date <= 8.64e15
) {
result[nodeId] = date;
}
}
}

return result;
}

/**
* Per-drive usage (resource count + bytes) reported by the connected node's
* `GET /drive-usage`. Generic — works on any atomic-server, self-hosted
Expand All @@ -359,25 +420,11 @@ export async function fetchNodeDriveUsage(
driveSubject: string,
agent: Agent,
): Promise<NodeDriveUsage | null> {
if (!serverUrl || !driveSubject || !agent?.subject) return null;
const data = await fetchDriveUsageData(serverUrl, driveSubject, agent);

const url = new URL('/drive-usage', serverUrl);
url.searchParams.set('subject', driveSubject);
if (data === null) return null;

try {
// Sign the URL being fetched, not the drive. The server rebuilds the
// signed message from the request it received — query string and all — so
// signing anything else fails the auth check before routing, and the
// endpoint answers 500 rather than the usage it holds.
const headers = await signRequest(url.toString(), agent, {
Accept: 'application/json',
});
const res = await fetch(url.toString(), { headers });

if (!res.ok) return null;

const data = await res.json();

return {
driveName: typeof data?.name === 'string' ? data.name : null,
resourceCount: Number(data?.resourceCount ?? 0),
Expand Down
90 changes: 81 additions & 9 deletions browser/data-browser/src/routes/SyncRoute.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -115,8 +115,10 @@ import {
type ManagedInfo,
EMPTY_NODE_INFO,
fetchNodeDriveUsage,
fetchPeerFullSyncs,
forgetServerPeer,
type NodeDriveUsage,
type PeerFullSyncs,
} from '../helpers/managedServer';
import { isOriginWithoutNode } from '../helpers/originNode';
import { getDriveUsage } from '../helpers/managedUsage';
Expand Down Expand Up @@ -170,6 +172,18 @@ function rawToNodeDid(raw: string): string {
return nodeSubject(raw);
}

function peerFullSyncFact(timestamp: number | undefined) {
if (timestamp === undefined) return null;
const date = new Date(timestamp);
const ago = formatTimeAgo(date);

return (
<time dateTime={date.toISOString()} data-testid='peer-last-full-sync'>
{ago ? `Last full sync ${ago}` : 'Last full sync just now'}
</time>
);
}

function StatusIcon({ status }: { status: NodeStatus }) {
switch (status) {
case 'synced':
Expand Down Expand Up @@ -698,7 +712,7 @@ function SyncPage() {
() => localStorage.getItem('ws-debug') === '1',
);
const [clientDbOn, setClientDbOn] = useState(() => isClientDbEnabled());
const { setServer, setDrive, baseURL } = useSettings();
const { agent: peerAgent, setServer, setDrive, baseURL } = useSettings();
const { drive: requestedDrive } = SyncRoute.useSearch();
// The portal does not know the name of a drive this account only belongs to,
// so the links to it carry the name this app shows.
Expand Down Expand Up @@ -1248,6 +1262,62 @@ function SyncPage() {
// display-only: reaching them is the server's job, not this tab's.
const serverPeers =
!isNode && showServerConn ? (managedInfo.peers ?? []) : [];
const peerServer = isNode ? getLocalServerOrigin() : status.serverUrl;
const showPeerDates = isNode
? pairedPeers.length > 0
: serverPeers.length > 0;
const [peerDates, setPeerDates] = useState<
(ScopedDriveValue<PeerFullSyncs> & { agentSubject: string }) | null
>(null);
const peerFullSyncs =
peerDates?.agentSubject === peerAgent?.subject
? currentDriveValue(peerDates, status.drive, peerServer)
: null;

useEffect(() => {
const drive = status.drive;
const agentSubject = peerAgent?.subject;

if (
!drive ||
!peerServer ||
!peerAgent ||
!agentSubject ||
!showPeerDates
) {
return;
}

let cancelled = false;
let checking = false;

const refresh = () => {
if (checking) return;

checking = true;

void fetchPeerFullSyncs(peerServer, drive, peerAgent).then(value => {
checking = false;

if (!cancelled) {
if (value === undefined) clearInterval(timer);

setPeerDates(
value ? { drive, server: peerServer, agentSubject, value } : null,
);
}
});
};

const timer = setInterval(refresh, 5000);
refresh();

return () => {
cancelled = true;
clearInterval(timer);
};
}, [peerAgent, peerServer, showPeerDates, status.drive]);

const connectionCount =
(showServerConn ? 1 : 0) + pairedPeers.length + serverPeers.length;

Expand Down Expand Up @@ -2339,8 +2409,9 @@ function SyncPage() {
link that was live and exchanging data. The server knows whether
the peer is connected right now and when it last synced. */}
{pairedPeers.map(peer => {
const raw = nodeDidToRaw(peer.nodeId);
const reported = serverPeers.find(
p => nodeDidToRaw(p.nodeId) === nodeDidToRaw(peer.nodeId),
p => nodeDidToRaw(p.nodeId) === raw,
);
const lastActivity = reported?.lastSeen ?? peer.lastSync;

Expand All @@ -2364,9 +2435,10 @@ function SyncPage() {
}
subtitle='Paired device'
facts={[
lastActivity
? `Last sync activity ${formatTimeAgo(new Date(lastActivity)) ?? 'just now'}`
: null,
peerFullSyncFact(raw ? peerFullSyncs?.[raw] : undefined) ??
(lastActivity
? `Last sync activity ${formatTimeAgo(new Date(lastActivity)) ?? 'just now'}`
: null),
// Only what this side actually counted — the accepting node
// answers frames through the engine and does not tally what
// it served, and a fabricated 0 would be the same lie this
Expand Down Expand Up @@ -2410,10 +2482,10 @@ function SyncPage() {
}
subtitle={`Paired with ${serverLabel(status.serverUrl ?? '')}`}
facts={[
// Peer statistics cover the last pass across the link.
peer.lastSeen
? `Last sync activity ${formatTimeAgo(new Date(peer.lastSeen)) ?? 'just now'}`
: null,
peerFullSyncFact(raw ? peerFullSyncs?.[raw] : undefined) ??
(peer.lastSeen
? `Last sync activity ${formatTimeAgo(new Date(peer.lastSeen)) ?? 'just now'}`
: null),
// What that sync moved. Deliberately the last pass, not a
// lifetime total — see `KnownPeer::last_sent`.
!!peer.lastSeen &&
Expand Down
67 changes: 67 additions & 0 deletions browser/e2e/tests/peer-full-sync-status.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
import { standaloneTest as test, expect } from './deployment-fixtures';
import { before, FRONTEND_URL } from './test-utils';

test.beforeEach(before);

test('Sync shows confirmation only for the peer that verified the drive', async ({
page,
}) => {
const drive = await page.evaluate(() => window.store.getDrive());
const verifiedPeer = '1'.repeat(64);
const activePeer = '2'.repeat(64);
const timestamp = Date.now() - 60_000;
const property = 'https://atomicdata.dev/properties/peer/';

await page.route('**/server', async route => {
const response = await route.fetch();
const body = await response.json();
body['https://atomicdata.dev/properties/server/peers'] = [
{ id: verifiedPeer, name: 'Confirmed device' },
{ id: activePeer, name: 'Active device' },
].map(({ id, name }) => ({
[`${property}nodeId`]: `atomic:node:${id}`,
[`${property}deviceName`]: name,
[`${property}live`]: true,
[`${property}lastSeen`]: timestamp,
}));
await route.fulfill({ response, json: body });
});
await page.route(
url =>
url.pathname === '/drive-usage' &&
url.searchParams.get('peerFullSyncsOnly') === 'true',
async route => {
expect(new URL(route.request().url()).searchParams.get('subject')).toBe(
drive,
);
await route.fulfill({
json: { peerFullSyncs: { [verifiedPeer]: timestamp } },
});
},
);

await page.goto(`${FRONTEND_URL}/app/sync`);
await expect(
page.getByText('Confirmed device', { exact: true }),
).toBeVisible();
await expect(page.getByText('Active device', { exact: true })).toBeVisible();
const confirmation = page.getByTestId('peer-last-full-sync');
await expect(confirmation).toHaveCount(1);
await expect(confirmation).toHaveAttribute(
'datetime',
new Date(timestamp).toISOString(),
);
await expect(page.getByText(/^Last sync activity /)).toHaveCount(1);

if (process.env.ATOMIC_PEER_FULL_SYNC_SCREENSHOT) {
await expect(
page.locator('#boot-splash:not(.boot-splash-leaving)'),
).toHaveCount(0);
await page.setViewportSize({ width: 1280, height: 900 });
await confirmation.scrollIntoViewIfNeeded();
await page.screenshot({
animations: 'disabled',
path: process.env.ATOMIC_PEER_FULL_SYNC_SCREENSHOT,
});
}
});
1 change: 1 addition & 0 deletions docs/src/app/folders-and-drives.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ You can keep working while the app shows that the first backup is pending. If it
*Storage and hosting* shows the last verified full sync for Cloud Server or a self-hosted Atomic Server, and the last successful backup for Cloud Vault.
Atomic uses the newest of these times to check how recently it verified your data on an external server or in Cloud Vault. It shows a warning after 24 hours and a stronger warning after seven days or when it can’t verify either a full sync or a backup.
For server sync, Atomic checks that the server has the drive’s local changes. A full sync with an external server clears the warning even with Cloud Vault turned off.
In Sync settings, a paired device shows *Last full sync* when it has confirmed that it holds this drive’s resources and uploaded files. The date belongs to this drive and this device.
Use *Back up now* in Sync settings to make a fresh Cloud Vault backup, including when an unchanged drive has an older backup.

![The drive switcher and the sidebar tree](../assets/app/drives.png)
Expand Down
6 changes: 6 additions & 0 deletions docs/src/atomic-sync/transports.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,12 @@ A person with write rights on a Drive can also make a server dial a node with `P
A node id says _where_ to connect. It does not grant any access. See [permissions and identity](permissions-and-identity.md).
A browser tab can't accept incoming connections, so it can't be dialed over Iroh.

Peers that support full sync confirmation check the stored resource versions and uploaded file contents for the drive after they exchange data.
The sender records the date when it receives a matching confirmation for that drive from the peer.
The date describes the verified copy at that time. An interrupted transfer leaves the previous date in place.

Iroh nodes advertise the `peer-full-sync` capability. The `VERIFY_FULL_SYNC` frame (`0x54`) carries a JSON object with the drive identifier, a 32-character hex nonce, and the 64-character hex SHA-256 drive hash in version 2 format. The receiving peer checks its durable drive state and file contents, then echoes the object in `FULL_SYNC_CONFIRMED` (`0x55`) if they match.

Servers can also announce which Drives they hold through the pkarr relay network, so a client that only knows a Drive's identifier can find a node to dial. See [resolution](../identifiers.md#resolution).

## WebRTC between browsers
Expand Down
Loading
Loading