English · 简体中文
Cross-domain scientific intelligence platform · Let researchers spend time on Thinking
Multi-source parallel retrieval · LLM per-paper scoring & summarization · Two-tier Markdown memory · Collaborative research · Knowledge graph
🌐 Try Online · 🖥️ Desktop Client · 📐 Architecture · 📜 LICENSE
ScholarPilot gives researchers back the time they spend "sifting through literature" so they can spend it on "thinking through problems."
You describe your research direction in natural language in a chat, and the system will:
- AI auto-plans retrieval keywords (per-source boolean syntax adapted to each data source)
- Parallel fetching from 14+ data sources (academic papers + global patents + Chinese patents)
- LLM 0-10 per-paper scoring + one-sentence summary + relevance justification
- Cutoff filtering → you only see high-relevance results
- 4-bucket feedback (highly relevant / relevant / unsure / irrelevant) → two-tier Markdown memory learning → next round more precise
- Once the library has documents, switch to Collaborative Research mode — AI reads, searches, and cites as it works
- One-click Deep Read for high-scoring papers (full PDF extraction + LLM deep-read)
- Knowledge Graph auto-extracts entities/relations, visualized with cytoscape force-directed layout
🌐 Try online: https://scholarpilot.top/ — Web version. Register and start a project to run a retrieval.
🖥️ Desktop: client/ Tauri 2, 100% BYOK, all LLM calls happen locally; artifacts in
client/src-tauri/target/release/bundle/{msi,dmg,deb,appimage}/.
Desktop client · Project "Transformer Inference Acceleration" — conversational project creation + keyword plan confirmation + 4-bucket feedback classification
Every flow (create project / retrieve / confirm keywords / 4-bucket classification / collaborative research / deep read) is embedded in the chat as a rich message bubble, driven by natural language. The bubble itself is interactive UI (KeywordConfirmPanel / BucketCardView / SourcePlanCard, etc.) — not just chat-box text instructing the user. A state machine manages 7 conversation scenes, and the AI infers timing to enter/exit flows automatically, with no slash commands to memorize.
User-level MEMORY.md (identity / profession / overall research direction) + Project-level MEMORY.md (research direction / sub-questions for this project). One index file + multiple detail .md files, readable and writable by the AI, viewable and editable by the user. More suitable for "research-direction profiles" than vector embedding RAG — interpretable, hand-editable, low iteration cost, prevents cross-project contamination.
QueryPlanAgent automatically plans three-tier degrading keywords: complex boolean (A OR B) AND (C OR D) → medium AND → simple fallback. Each source adapts to its API syntax independently (OpenAlex 5 words, arXiv 2-3 words, DBLP 2-word prefix, OpenAlex_zh dual-format |||, EPO CQL). Users can edit them in the confirmation bubble before executing.
LLM extracts entities/relations (document / author / concept / topic / journal); cytoscape-fcose force-directed layout renders 500+ nodes smoothly; community detection + hubs/gaps analysis. Naturally suited to literature-survey workflows.
The desktop client writes the LLM key to OS Keychain. All scoring / summarization / collaborative research happens locally; not a single line of backend code is changed. The web version uses platform-owned keys; the desktop side gets zero-cost integration with the user's own Anthropic / OpenAI / DeepSeek / Moonshot / jiekou.ai relay / Ollama local.
Academic: OpenAlex (250M+) / arXiv / EuropePMC / Crossref / DBLP / PubMed / ClinicalTrials / OpenAlex_zh Patents: EPO OPS (Europe) / Lens.org (90+ jurisdictions, 160M+) / PatentHub (China, tiered paid + PDF)
asyncio.gather parallel fetch → LLM scoring → chord-style summarization dispatch. A single round returns results in 30-60 seconds. Paid sources (PatentHub) go through a budget gate; the user can confirm again to override the soft cap.
| Form | Best for | Notable |
|---|---|---|
| 🌐 Web scholarpilot.top | Try it / occasional use | Register and go; platform-hosted; Cloudflare Tunnel — no exposed origin port |
| 🖥️ Desktop Tauri 2 | Heavy long-term use | OS Keychain stores tokens; local SQLite; BYOK self-pay; native PDF fetch bypasses anti-scrape |
This
main0repo is a frozen public-version branch (2026-05-08). It's the engineering- and product-stage open-source snapshot of ScholarPilot, intended for learning and reference.
| Form | Path | Purpose |
|---|---|---|
| Web Frontend (frozen) | frontend/ |
Vue 3 + Element Plus, browser SPA |
| Desktop Client (active) | client/ |
Tauri 2 + Vue 3, BYOK, local SQLite + file sandbox |
| Backend (CN node) | backend/ |
FastAPI + Celery + PostgreSQL, full-stack API for the web SPA |
| sp-api (HK node) | sp-api/ |
Client-only lightweight gateway, fetcher orchestration + paid-source budget gate, no LLM dependency |
Architectural evolution: from single web app → "Web + Desktop + Dual-backend" trio. Web uses platform LLM keys (self-managed cost); desktop client is 100% BYOK (cost belongs to the user). See
docs/architecture/.
- Chat-as-subject: every flow (create / retrieve / collaborate / push) happens in chat bubbles, driven by natural language
- Agent-first keyword planning: QueryPlanAgent picks the strategy automatically; per-source keyword adaptation (each source has its own API syntax)
- Two-phase user confirmation: AI generates keyword plan → user edits and confirms → execute retrieval
- LLM per-paper scoring: ScoringAgent gives 0-10 + one-sentence summary per paper; cutoff filter applied
- LLM summary: for papers above the cutoff, generates summary, key points, and relevance to project
- 4-bucket classification: highly relevant / relevant / unsure / irrelevant, persisted across rounds
- Three-tier keyword degradation: complex boolean → medium AND → simple fallback, per-source API adaptation
- 14+ data sources: academic (OpenAlex / arXiv / EuropePMC / Crossref / DBLP / PubMed / ClinicalTrials / OpenAlex_zh) + patent (EPO OPS / Lens.org / PatentHub)
- Select existing papers to enter collaborative mode; deep Q&A grounded in the library
- ResearchAgent reads, searches, and cites as it works; section-level probes for fine-grained reading
- Collaboration context persisted to the conversation state machine
- Inspired by Claude Code's MEMORY.md design
- User-level: identity / profession / overall research direction, shared across projects
- Project-level: research direction / sub-questions for this project only
- AI can read and write; user can view and edit; prevents cross-project contamination
- LLM extracts entities and relations; cytoscape.js force-directed (fcose) layout
- Node types: document / author / concept / topic / journal
- Community detection, hubs / gaps analysis
- DuckDB + SQLite FTS5; supports offline retrieval over millions of papers
- Three modes: static / API / hybrid
- Tauri 2 native window (Windows / macOS / Linux)
- OS Keychain stores OAuth2 dual tokens — never localStorage
- Local SQLite (11 tables aligned with backend) + file-system sandbox
- BYOK: user-supplied LLM key in OS Keychain; all scoring / summarization / collaborative research happen locally
- Native PDF fetching via Rust (reqwest + cookie store) to bypass some academic site anti-scraping
- Background process crawls latest progress on a schedule based on MEMORY
- Desktop notification / Email / Telegram push (planned)
Inspired by Claude Code's 12-layer harness, the system ships with an extensible AI infrastructure:
| Component | Description |
|---|---|
| Tool Registry | 14+ fetchers registered as unified tools; runtime stats + reliability reports |
| Hook Engine | 11 lifecycle hooks (ROUND_START / POST_SEARCH / PRE_FEEDBACK, ...) |
| Skill System | 8 reusable workflows: Deep Dive / Trend Spotter / Gap Finder / Quality Audit / Outline Synth, etc. |
| Agent Pipeline | IntentAgent → QueryPlanAgent → ScoringAgent → MemoryAgent → CollaborationAgent → GraphAgent end-to-end |
| Memory | Two-tier Markdown (user + project) + bucket-driven structured memory |
| Layer | Tech |
|---|---|
| Desktop client | Tauri 2 + Vue 3 + Element Plus + cytoscape.js + p-queue + zod |
| Web frontend (frozen) | Vue 3 + TypeScript + Element Plus + Vite + vis-network |
| Client Rust backend | reqwest + reqwest_cookie_store + sqlx + tauri-plugin-sql v2 |
| Server | FastAPI + SQLAlchemy (async) + Alembic + Pydantic v2 |
| Task queue | Celery + Redis |
| Database | PostgreSQL 16 (primary) + DuckDB + SQLite FTS5 (local KB) |
| AI / LLM | Anthropic Claude SDK / OpenAI SDK / DeepSeek / Moonshot / Ollama / OpenAI-compatible relay |
| Document parsing | markitdown (PDF / DOCX → Markdown) |
| Deployment | Docker Compose + Cloudflare Tunnel + nginx |
0. Try it without installing → scholarpilot.top
Open https://scholarpilot.top/ and register (invite code required — comment on Issues to ask for one, or self-host). Go to the Journal homepage → describe your research direction in natural language → complete a retrieval → give feedback in the 4 buckets → watch the next round adjust. The web version uses platform LLM keys — zero config, ready to use.
git clone https://github.com/sstar16/scholarpilot
cd scholarpilot
cp .env.example .env
# Edit .env — at minimum set POSTGRES_PASSWORD, SECRET_KEY, DISABLED_SOURCES
docker compose up -dOpen http://localhost, register an account (first run needs a manual invite code or invite-code logic tweak in backend), and configure your LLM in the top-right settings.
cd client
npm install
npm run tauri:dev # start Tauri dev window (first cargo build is 5-10 min)
# Or package:
npm run tauri:build # artifacts in src-tauri/target/release/bundle/{msi,dmg,deb,appimage}/The client connects to the sp-api gateway by default. For local dev, edit client/.env's VITE_API_BASE_URL to point at your own backend.
cd sp-api
docker compose -f docker-compose.sp-api.yml up -d
# In the client, set client/.env's VITE_API_BASE_URL to this sp-apischolarpilot/
├── backend/ # FastAPI + Celery + Postgres (CN node)
│ ├── app/
│ │ ├── api/ # FastAPI routes
│ │ ├── models/ # SQLAlchemy ORM
│ │ ├── schemas/ # Pydantic schemas
│ │ ├── services/
│ │ │ ├── fetchers/ # 14+ data-source Fetchers
│ │ │ ├── core/ # LLM manager + Redis persistence
│ │ │ └── ...
│ │ ├── harness/ # AI agent infrastructure (10 agents)
│ │ ├── knowledge_base/ # DuckDB + SQLite FTS5
│ │ ├── prompts/ # per-source + per-agent prompts
│ │ └── workers/ # Celery tasks
│ └── alembic/versions/ # Database migrations
├── frontend/ # Vue 3 web SPA (frozen)
├── client/ # Tauri 2 desktop client (active)
│ ├── src/ # Vue 3 source (11-phase pipeline ported from backend workers)
│ ├── src-tauri/ # Rust backend (commands / fs sandbox / pdf fetcher)
│ ├── e2e/ # Playwright tests
│ └── README.md
├── sp-api/ # HK client gateway (fetcher only, no LLM)
├── nginx/ # nginx config (reverse proxy for frontend / backend)
├── docs/
│ └── architecture/ # 5 Mermaid architecture diagrams + viewer
├── scripts/ # General scripts (backup, invite-code generation, etc.)
├── docker-compose.yml
├── docker-compose.sp-api.yml
├── .env.example
├── CLAUDE.md # Project guide for AI coding agents
└── LICENSE
# Backend
docker compose restart backend worker beat flower
docker compose logs -f backend
docker compose exec backend alembic upgrade head
# Web frontend
docker compose build frontend && docker compose up -d frontend && docker compose restart nginx
# Desktop client
cd client && npm test # vitest
cd client && npx vue-tsc --noEmit # type check
cd client && npx playwright test # e2e
cd client/src-tauri && cargo check # Rust type checkScholarPilot benchmarked, borrowed from, and directly depends on many outstanding open-source projects during design and implementation. Sincere thanks to them all.
- LearningCircuit/local-deep-research — Local-first deep-research assistant. We borrowed its anti-hallucination prompt constraints (
Never make up sources), report-generation deduplication, three-tier citation handler strategy, News Subscriptions pattern, and MCP-server exposure design.
Desktop client:
- Tauri 2 — Rust native-window framework
- Vue 3 + Element Plus — UI framework
- Vite — Build tool
- cytoscape.js + cytoscape-fcose — Knowledge-graph visualization (500+ nodes, smooth)
- p-queue — LLM concurrency queue (priority + intervalCap + pause/resume)
- zod — TS-first schema validation
- @tauri-apps/plugin-sql v2 — sqlx-backed SQLite migration
- reqwest + reqwest_cookie_store — Rust HTTP (academic PDF fetching)
Server:
- FastAPI — async Python web framework
- SQLAlchemy + Alembic — ORM + migrations
- Celery + Redis — async task queue
- PostgreSQL 16 — primary DB
- DuckDB + SQLite FTS5 — local knowledge base
- Pydantic v2 — validation and serialization
- markitdown — PDF/DOCX → Markdown
LLM providers:
- Anthropic Claude SDK
- OpenAI SDK
- DeepSeek
- Moonshot Kimi
- Ollama — local LLM
- Any OpenAI-compatible relay
Data-source APIs (huge thanks to their open data):
- OpenAlex — 250M+ academic works
- arXiv — CS / physics preprints
- EuropePMC — full PubMed mirror
- Crossref — citation data
- DBLP — CS conference papers
- PubMed — biomedical
- ClinicalTrials.gov — clinical trials
- EPO OPS — European patents
- Lens.org — global patent-academic aggregator
- PatentHub — Chinese patents
Deployment and infrastructure:
- Docker / Docker Compose
- Cloudflare Tunnel (cloudflared) — zero-exposure HTTPS
- nginx
- Claude Code — The MEMORY.md index + multiple-
.md-detail interpretable-memory mechanism inspired our two-tier Markdown memory design; it also inspired the multi-layer harness pattern (Hook / Tool Registry / Skill) used in Harness Engineering. - Anthropic Engineering Blog — Reference for agent design patterns, prompt caching, and tool-use protocol.
We did detailed comparison work at the selection stage:
- Knowledge-graph visualization: cytoscape.js (~6.6M weekly downloads) vs vue-flow (200K) vs vis-network (269K, replaced)
- LLM concurrency: p-queue vs p-limit (p-queue wins on priority + intervalCap)
- TS schema: zod vs typia vs ajv (zod wins on DX)
- Rust PDF: reqwest + cookie store + optional rquest (boring-tls) fallback
- SQLite migration: tauri-plugin-sql v2 forward-only (accept the no-Down constraint)
Most major features are done or in progress:
- ✅ Retrieval flow (11-phase state machine, client-side)
- ✅ DevTools log-visualization platform
- ✅ Local knowledge base (DuckDB + SQLite FTS5)
- ✅ Chat-as-subject / collaborative research / two-tier Markdown memory
- ✅ Knowledge graph (cytoscape)
- ✅ Desktop client (M1 / M2 / M3 BYOK 100%)
- 🚧 sp-api refactor (CN/HK dual backend, BYOK takes over business/LLM)
- 🚧 Scheduled push (MEMORY-based background monitoring)
- ⬜ MCP server (expose fetchers + agents to Claude Desktop / Cursor)
- ⬜ Multi-user / team-shared projects
- ⬜ Client-side SQLCipher encryption (enterprise audience)
PRs, issues, and benchmark discussion are welcome. Before submitting, we recommend running npx vue-tsc --noEmit (client) / pytest (backend) to make sure nothing breaks. Commit format: feat: / fix: / refactor: / docs: / chore:.
This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0) — see LICENSE.
- You may read, use, modify, and distribute the code freely.
- Any derivative work, including SaaS / network deployments based on a modified copy, MUST be released under the same AGPL-3.0 terms.
- For commercial SaaS or proprietary use without AGPL copyleft, please contact the author for a commercial license.
© 2026 sstar16