fix(hugegraph-dist): gate init-store on a dedicated init_store.enabled option - #3119
fix(hugegraph-dist): gate init-store on a dedicated init_store.enabled option#3119bitflicker64 wants to merge 12 commits into
Conversation
…g is false Honor the existing ServerOptions flag in InitStore so distributed/HStore can opt out of local backend and admin init. Unset keeps full init for standalone. Docker maps HG_SERVER_LOAD_FROM_LOCAL_CONFIG to rest-server.properties. Fixes apache#3118
dosubot
Bot
added
size:M
imbajin
left a comment
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: Disabling local graph loading also bypasses the Docker password-based admin bootstrap, so the requested credential is never installed. Evidence: exact-head static trace through docker-entrypoint.sh, InitStore.java, and GraphManager.java; 3/3 focused tests passed; visible checks are green.
…d option Replace the graph.load_from_local_config gate with a dedicated init_store.enabled option defaulting to true. That property already means whether GraphManager loads graph definitions from the local directory, and existing configs may materialize its declared false default, so reusing it made an explicit false skip backend and admin initialization for standalone RocksDB/HBase installs. The early return also skipped StandardAuthenticator.initAdminUserIfNeeded, so a Docker PASSWORD was piped into init-store.sh and discarded without creating the admin, leaving the server on the auth.admin_pa default. In skip mode the entrypoint now writes PASSWORD to auth.admin_pa instead, and does not create docker/init_complete since nothing was initialized. The gate is read back from rest-server.properties so a config mounted without the env var behaves the same way. Cover the lifecycle with a main-level test whose graphs directory would fail if the early exit were missed, and entrypoint smoke tests for default init, PASSWORD, skip, skip with PASSWORD, mounted property, and false to true restart. Run those in Docker CI and extend its path filter to the dist docker directory.
dosubot
Bot
added
size:L
bitflicker64
changed the title
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3119 +/- ##
============================================
- Coverage 39.13% 1.54% -37.60%
+ Complexity 264 21 -243
============================================
Files 770 748 -22
Lines 65772 63282 -2490
Branches 8725 8280 -445
============================================
- Hits 25741 975 -24766
- Misses 37272 62223 +24951
+ Partials 2759 84 -2675 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
imbajin
left a comment
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The skip path still fails admin bootstrap for the shipped HStore configuration and has inconsistent option/password handling; backend and plugin registration also occurs before the gate. Evidence: six independent review lanes, exact-head static traces, an uppercase FALSE regression test failing four assertions, and all visible latest-head checks passing.
… is true With init_store.enabled=false the built-in admin account is not created by init-store, and the only other component that creates it is GraphManager.initAdminUserIfNeeded(), reached from loadMetaFromPD() and so gated on usePD. That option defaults to false and the shipped HStore compose files do not set it, so enabling auth in skip mode would start a server that enforces authentication with no account to authenticate against. The entrypoint now refuses that combination and exits with an explanation, and InitStore logs the same condition for non-Docker installs. Skipping without auth is unaffected. usePD=true is already the documented pairing for distributed deployments in the cluster-test rest-server.properties template. Also align the shell with the server's boolean parsing, which is case-insensitive, so HG_SERVER_INIT_STORE_ENABLED=FALSE no longer means skip to Java and run to the entrypoint; read the property through the separators a properties file allows; escape the password for properties serialization so a backslash survives; and register backends and plugins only on the enabled path, since plugin registration runs every plugin's register() and propagates its failures. Add entrypoint coverage for these plus a test that disabled mode leaves backend options unregistered. Note a pre-existing arthas config key mismatch found while checking the option table.
|
Unrelated to this change, but noticed while checking the The arthas options are declared camelCase ( It is invisible today because each shipped value equals its default. It bites on any change, e.g. |
imbajin
left a comment
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The mounted-properties override path can create duplicate scalar keys and make init-store or server startup fail. Evidence: six independent review lanes, exact-head static tracing through docker-entrypoint.sh and HugeConfig.java, and all visible latest-head checks passing.
get_prop was widened to read the `=`, `:` and whitespace separators a properties file allows, but set_prop still matched only `key=value`. A mounted `init_store.enabled:false` overridden by HG_SERVER_INIT_STORE_ENABLED therefore gained a second definition instead of being replaced, and a key defined twice collects both values into a list that fails the scalar type check while the config is still loading. Both init-store and server startup would fail. The same applied to a colon-form auth.admin_pa overridden by PASSWORD. set_prop now matches the same separators and collapses every existing definition into one canonical `key=value` line, leaving comments alone. It is written in awk, so the key and value are matched and emitted literally and no regex escaping is involved. Cover colon-form and whitespace-form overrides asserting a single remaining definition, a colon-form auth.admin_pa override, and that commented defaults are not treated as definitions. Add a test loading a duplicated key through HugeConfig to pin why the collapse is required.
imbajin
left a comment
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The disabled init path can still report success for an unusable auth configuration, and two Docker config rewrite paths can fail or corrupt mounted/protected configs. Evidence: six independent exact-head review lanes; entrypoint smoke tests passed 57 assertions with 2 macOS-only skips; git diff --check passed; codecov/patch and codecov/project are failing.
…onf in place Four fixes to the skip path. InitStore returned zero for a configuration it had just reported as unusable: init_store.enabled=false with an authenticator set and usePD false means no component creates the built-in admin. Tarball and init-job callers see only the exit status, so they continued into an auth-enabled server nobody could reach. It now fails. Remote auth is exempt, since the auth manager is then an RPC client and StandardAuthenticator only bootstraps a local one; the entrypoint guard gains the same exemption, which it was missing. set_prop replaced the config by rename. A single-file bind mount cannot be replaced that way, so startup aborted under set -e, and the rename also discarded the original mode, which matters where auth.admin_pa is written. It now truncates and rewrites in place, and creates its scratch file under umask 077. On the skip path enable-auth.sh ran unconditionally, and it appends its keys without checking, so a mounted config that already enabled auth ended up with auth.authenticator and auth.graph_store defined twice, which the config parser rejects. It now runs only when auth is not configured yet, and both keys are collapsed to single definitions afterwards. The test guard that skipped two cases without GNU sed was stale once set_prop moved to awk. Removed, so those cases run everywhere. Cover the exit status and the remote-auth exemption in the CLI test, and add entrypoint cases for a mounted auth config, the remote-auth exemption, inode and mode preservation, and scratch file cleanup.
imbajin
left a comment
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The skip path can leave Gremlin authentication incomplete and rejects supported custom authenticators. Evidence: six independent exact-head review lanes, static traces through enable-auth.sh and HugeAuthenticator.loadAuthenticator(), 74/74 entrypoint assertions passing, git diff --check passing, and all visible latest-head workflows passing.
dosubot
Bot
added
size:XL
…n check enable-auth.sh also adds the authentication block to gremlin-server.yaml and switches hugegraph.properties to the auth proxy. Skipping it whenever auth.authenticator already existed left both undone, so a mounted config that set only that key could start with REST auth on and Gremlin open. Apply each of the three changes only where it is missing instead, and point the Gremlin block at whichever authenticator the REST config names. The admin-account requirement now applies only to the built-in authenticator and its subclasses. auth.authenticator takes any implementation class, and a custom LDAP/OIDC/plugin one manages identities elsewhere, so requiring usePD for it rejected a deployment that works. get_prop no longer deletes whitespace inside a value, only around it, as the properties parser does.
41259fb to
0dd5a7a
Compare
- get_prop uses `sed -E` rather than `-r`, which only GNU sed documents - the appended gremlin-server.yaml block no longer glues itself onto a last line that has no trailing newline - the option description, the InitStore comment and docker/README no longer claim every authenticator needs the built-in admin account, which stopped being true when the check was narrowed - InitStoreConfigTest is imported in suite order in UnitTestSuite - a test asserts the entrypoint's inlined auth block still names everything bin/enable-auth.sh does, so the two cannot drift unnoticed
|
Did a self-review pass over the whole diff and pushed 090763a with the follow-ups, no behaviour change beyond the two fixes below:
The rest is text and hygiene: the option description, the Since the entrypoint reapplies what Local run: 106 assertions across 30 entrypoint cases, unit suite 626 tests green on JDK 11, rat and editorconfig clean. |
# Conflicts: # hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/UnitTestSuite.java
imbajin
left a comment
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The skip path still has inconsistent admin-bootstrap decisions across Docker and Java, and several auth configurations can start unusable or incompletely protected. Evidence: six independent exact-head review lanes, static traces through docker-entrypoint.sh, InitStore, GraphManager, and StandardHugeGraph; 106/106 entrypoint assertions and the focused Java test passed; git diff --check passed; visible Actions passed but Codecov statuses failed.
|
Coverage infrastructure note for final head Both completed Linux server jobs generated and discovered four coverage XML reports, then reached the repository coverage upload step. The tokenless POST was rejected by Codecov with HTTP 429 in both jobs:
The uploader explicitly exited with status 0 after those errors, so GitHub shows the jobs as successful but Codecov has not produced final-head project or patch statuses. This reproduces the repository-level tokenless-upload failure seen on the baseline; it is not a test or coverage-generation failure. The focused suite has 17 passing tests and the full Java 11 unit profile has 609 passing tests with one existing skip. Could a maintainer retry coverage after the rate-limit window with the repository upload token, or explicitly waive the Codecov gate for this head? |
dosubot
Bot
added
size:XXL
2 participants
Fixes #3118
Kept in sync with hugegraph#171
Problem
InitStorealways performs local initialization: it scansconf/graphs,initializes non-HStore backends, and bootstraps the built-in admin account. That
is correct for standalone and tarball deployments, but a PD/HStore deployment
already owns its metadata. Its Kubernetes pods also cannot rely on the
container-local
docker/init_completeflag surviving a restart.Solution
This adds a dedicated
init_store.enabledoption, defaulting totrue.Existing standalone and tarball configurations therefore keep the current full
initialization path. Distributed deployments can set it to
false(or setHG_SERVER_INIT_STORE_ENABLED=falsein the server container) to makeInitStorereturn before backend registration, plugin registration, graphscanning, or local admin initialization.
The disabled path validates authentication before returning successfully:
StandardAuthenticatoror subclass requiresusePD=trueand anHStore-backed
auth.graph_store, so the PD-created admin is visible to theauthenticator.
built-in admin requirement.
bin/init-store.shwrapper.The Docker entrypoint now applies the same decision by invoking that Java
validation gate. It also:
an already complete mounted configuration;
HugeFactoryAuthProxyconfiguration for mountedauth configs, with or without
PASSWORD;PASSWORDtoauth.admin_paonly in disabled mode, encoding allvalid UTF-8 and Java-properties metacharacters without changing the value;
needs to be created, while always closing the auth graph;
rest-server.propertiesto mode0600before persisting thatsecret while preserving a mounted file's inode;
same-directory scratch file;
docker/init_completewhen no initialization occurred; andfollowed by one auth bootstrap when authentication is later mounted, using
either
PASSWORDor the configuredauth.admin_pa.auth.admin_pais used only when the PD-backed admin is first created. ChangingPASSWORDon a later restart does not rotate an existing account.Validation
906a3ef630294f8fad8e1d415aaa50d4af966a44.The entrypoint suite covers default and disabled initialization, mounted
properties, Boolean spelling and separators, duplicate-key normalization,
custom/remote/built-in auth, HStore eligibility, no-password mounted auth,
persisted no-auth-to-auth restarts, complete read-only mounts, helper no-op
behavior, wrapper exit propagation, secret file permissions, and exact Java
parsing of passwords containing backslashes, spaces, controls, non-ASCII
characters, embedded newlines, and a trailing newline.
Scope
This change does not alter
graph.load_from_local_config, generalGraphManagerbehavior, Helm job structure, or shipped ComposeusePDdefaults. Deployments opting out of init-store remain responsible for selecting
the PD/HStore metadata path described above.