Skip to content

[Global Ask] Verify KG, ontology, and semantic claims with public SearXNG evidence #272

Open

Description

@seonghobae

Buyer problem

The Buyer-surface stack #258 → #260 → #261 → #262 → #263 → #264 already assembles authorized source text, persisted Knowledge Graph facts, project/role/Keyman semantic facts, contextual-orchestrator answers, TEPP run metadata, and fast-mlsirm calibrated reports. The integration is incomplete in four material ways:

  1. Global Ask loads persisted semantic/KG evidence only after lexical candidate selection, so a buyer term that exists only in project, responsibility, affiliation, Keyman, edge type, endpoint label, or ontology IRI can miss the source entirely.
  2. The citation-bearing post-chat path requests contextual-orchestrator mode="route" although the accepted orchestration ADR reserves mode="verify" for checked, citation-bearing judgment.
  3. SearXNG is wired only to an admin relation-presence check; Global Ask cannot return FEVER-style supported / refuted / not_enough_information judgments for public KG/ontology/semantic claims.
  4. TEPP and fast-mlsirm artifacts can be confused with externally verifiable facts. Current TEPP main publishes an accepted-run contract, not a completed-result DTO; fast-mlsirm scores are calibrated measurement evidence, not web truth.

Required product slice

  • Add bounded, index-backed candidate nomination from persisted project, role/responsibility/affiliation, Keyman, KG edge, endpoint label, and ontology evidence.
  • Keep candidate IDs non-authoritative and re-run the existing source-post visibility/ABAC gate before any body or semantic evidence is exposed.
  • Change citation-bearing Global Ask/post chat to contextual-orchestrator mode="verify" with an explicit reasoning effort.
  • Add an opt-in verify_external Global Ask contract. Only claims backed by public source posts may leave the trust boundary. Never send private source text, raw source hints, credentials, PII, TEPP payloads, or fast-mlsirm respondent/item data to SearXNG.
  • Search a bounded number of public claims through configured SearXNG, then use contextual-orchestrator mode="verify" to classify each claim as supported, refuted, or not_enough_information against the retrieved evidence.
  • Keep external URLs separate from internal post citations and preserve provenance/truth status. Web evidence may corroborate or dispute a real-world claim; it must not promote an inferred edge to authoritative fact.
  • Surface verification status, rationale, and external citations in the Buyer Ask Agent with a clear next action.
  • Preserve the TEPP boundary: accepted transport evidence is not a completed measurement. Preserve the fast-mlsirm boundary: calibrated scores expose model/version/provenance but are never labeled web-verified.

TDD acceptance

  • semantic-only and KG/ontology-only terms nominate the authorized source;
  • private-only semantic facts never produce an external query;
  • external result URLs are bounded and reject search pages, localhost, and literal private-network hosts;
  • conflicting evidence can produce refuted; no usable evidence produces not_enough_information;
  • contextual-orchestrator requests contain literal "mode": "verify" for both citation chat and claim adjudication;
  • external citation IDs cannot become cited_post_ids;
  • TEPP accepted envelopes cannot become completed measurement claims;
  • fast-mlsirm report evidence cannot become externally verified facts;
  • frontend tests cover opt-in, unavailable state, three-way status, external links, and next-action copy;
  • changed production statement/branch coverage and public docstrings remain 100%.

References

  • Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018). FEVER: A large-scale dataset for fact extraction and verification. Proceedings of NAACL-HLT 2018, 809–819.
  • World Wide Web Consortium. (2013). PROV-O: The PROV ontology.
  • SearXNG. (2026). Search API documentation.

Stack gate

Implement as a draft PR stacked on #264. Do not merge or mark Ready until #264’s current cutoff-navigation review finding is repaired, the stack is semantically restacked on the current #258 head, all exact-head checks are terminal-success, and an independent current-head approval exists.

Activity

  1. seonghobae commented on Aug 20, 2026

    @seonghobae
    ContributorAuthor

    Implementation remains Draft PR #276 exact current head 84b42bb. Keep this issue open: the draft still needs full current-head workflow evidence, an independent approval, and protected stack merges before its public-verification contract can be called delivered.

  2. changed the title [-][Buyer Gap] Verify Global Ask KG/ontology/semantic claims with public SearXNG evidence[/-] [+][Global Ask] Verify KG, ontology, and semantic claims with public SearXNG evidence[/+] on Aug 25, 2026
  3. seonghobae commented on Aug 26, 2026

    @seonghobae
    ContributorAuthor

    Current-main audit correction (2026-08-26): the semantic candidate-nomination slice is active on PR #672, but public claim verification is still not protected-main delivery.

    The closed stack implementation from PR #641 must not be replayed unchanged. Its claim selection depends on question-token overlap and string-pattern classification, which are heuristic admission rules, and it forces mode="verify" despite the current ADR 0076 / contextual-orchestrator boundary assigning orchestration policy to the upstream gateway with reasoning_effort="auto".

    The replacement slice must therefore begin with a typed persisted public-claim envelope (claim kind, exact public evidence post, truth/provenance/time fields, and egress eligibility), apply current authorization again before dispatch, and let contextual-orchestrator own paper-grounded orchestration. Private/person/Keyman/TEPP/fast-mlsirm evidence remains ineligible; external URLs remain separate from cited_post_ids. Until that owning contract exists, the correct state is unavailable rather than a heuristic query.

  4. seonghobae commented on Aug 26, 2026

    @seonghobae
    ContributorAuthor

    Cycle 2026-08-26 09:13 KST: public-claim envelope slice is starting on current protected main (494b54e). It will persist a typed public-claim envelope (claim kind, public evidence post, truth/provenance/time, egress eligibility), re-check authorization before any SearXNG dispatch, keep private/person/Keyman/TEPP/fast-mlsirm ineligible, keep external URLs out of cited_post_ids, and fail closed to unavailable until the owning contract can classify. Independent of #672/#668/#657/#640. Will open a PR onto main after push. Do not self-approve.

  5. seonghobae commented on Aug 26, 2026

    @seonghobae
    ContributorAuthor

    Replacement slice is up as #679 (feat/public-claim-envelope-v2220, ADR 0224).

    Owning contract: typed persisted public_claim_envelope. Global Ask verify_external is opt-in. Admission is the envelope, not question-token overlap. Person / Keyman / TEPP / fast-mlsirm kinds cannot be stored. External URLs never enter cited_post_ids. Missing SearXNG is unavailable. This repository does not force mode="verify".

    Independent of #672 / #668 / #657 / #640. Independent APPROVE required; not self-approving.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions