Repository navigation
feat(engine): keep every packet for up to 7 days, one SQLite file a day - #458
Merged
Merged
Conversation
The packet history's store (F1.41 behaviors 1-6, 12, 13): every packet the engine lists goes to a background writer and into history/<day>.db, so receiving never waits. Once a minute the days past the length go, then the oldest past the size limit; the status says when the limit keeps fewer days than asked for, or when today alone filled it. Settings › Data › History sets the length (7 days) and size (1,024 MB); [engine.history] in the gateway's configuration wins on every start. Turning it off, after a second Save, and Start over delete it. An unreadable day is set aside, and a second engine on the same folder keeps none. Refs #314
Contributor
Package reportVulnerable packagesNo known vulnerabilities in direct or transitive packages. Available package updates
Pinned versionsThese projects override the central version in
Checked at 67c0aac against nuget.org. |
…for late sizes on Windows The gateway's engine holds the packet history's files open, which Windows won't delete; an asynchronous dispose can reach Dispose(true) before the host stops. Windows also reports an open file's size late, so the days-fit estimate is checked as a range.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First part of #314 (spec F1.41).
Store (
PacketHistory, MITCore):history/under the data folder, throughMicrosoft.Data.Sqlite. WAL,synchronous=NORMAL, no pooling..invalidand named in the status. A.lockfile keeps a second engine on the same folder out (it shows InUseElsewhere).Settings:
AppSettings.History(HistorySettings(Days, MaxMegabytes), default 7 days within 1,024 MB), validated assettings.history-invalid.[engine.history] days / max_megabytesingateway.tomlwins on every start, and Settings shows it without letting it be changed. The commented example is indeploy/linux/gateway.toml.IEngineService.GetHistoryStatusAsync, operator only on a gateway.UI: Settings › Data › History, above capture. A switch, days, size, what's kept ("7 days kept, 412 MB, since …"), and the warning. Turning it off asks once more, because it deletes what's kept.
Not in this PR: the restart replay, long tracks and older packets, and the dashboard's 7 days. Those are the next PRs in the stack.
Checks:
PacketHistoryTests(12) andHistoryUiTests(9), plus gateway access checks.libe_sqlite3.sois in the linux-arm64 publish.