Skip to content

About

AI-powered academic research assistant

Resources

Stars

4 stars

Watchers

0 watching

Forks

Repository files navigation

English · 简体中文

ScholarPilot · Let researchers spend time on thinking

ScholarPilot

Cross-domain scientific intelligence platform · Let researchers spend time on Thinking

Multi-source parallel retrieval · LLM per-paper scoring & summarization · Two-tier Markdown memory · Collaborative research · Knowledge graph

Live Demo License: AGPL-3.0 Tag Tauri 2 Vue 3

🌐 Try Online · 🖥️ Desktop Client · 📐 Architecture · 📜 LICENSE


What it does

ScholarPilot gives researchers back the time they spend "sifting through literature" so they can spend it on "thinking through problems."

You describe your research direction in natural language in a chat, and the system will:

  1. AI auto-plans retrieval keywords (per-source boolean syntax adapted to each data source)
  2. Parallel fetching from 14+ data sources (academic papers + global patents + Chinese patents)
  3. LLM 0-10 per-paper scoring + one-sentence summary + relevance justification
  4. Cutoff filtering → you only see high-relevance results
  5. 4-bucket feedback (highly relevant / relevant / unsure / irrelevant) → two-tier Markdown memory learning → next round more precise
  6. Once the library has documents, switch to Collaborative Research mode — AI reads, searches, and cites as it works
  7. One-click Deep Read for high-scoring papers (full PDF extraction + LLM deep-read)
  8. Knowledge Graph auto-extracts entities/relations, visualized with cytoscape force-directed layout

🌐 Try online: https://scholarpilot.top/ — Web version. Register and start a project to run a retrieval.

🖥️ Desktop: client/ Tauri 2, 100% BYOK, all LLM calls happen locally; artifacts in client/src-tauri/target/release/bundle/{msi,dmg,deb,appimage}/.

ScholarPilot Desktop · Conversational project management

Desktop client · Project "Transformer Inference Acceleration" — conversational project creation + keyword plan confirmation + 4-bucket feedback classification


✨ Highlights

🎯 Truly "chat-as-subject", not a ChatGPT wrapper

Every flow (create project / retrieve / confirm keywords / 4-bucket classification / collaborative research / deep read) is embedded in the chat as a rich message bubble, driven by natural language. The bubble itself is interactive UI (KeywordConfirmPanel / BucketCardView / SourcePlanCard, etc.) — not just chat-box text instructing the user. A state machine manages 7 conversation scenes, and the AI infers timing to enter/exit flows automatically, with no slash commands to memorize.

🧠 Two-tier Markdown memory — inspired by Claude Code's interpretable memory

User-level MEMORY.md (identity / profession / overall research direction) + Project-level MEMORY.md (research direction / sub-questions for this project). One index file + multiple detail .md files, readable and writable by the AI, viewable and editable by the user. More suitable for "research-direction profiles" than vector embedding RAG — interpretable, hand-editable, low iteration cost, prevents cross-project contamination.

🤖 Agent-first retrieval + per-source keyword adaptation

QueryPlanAgent automatically plans three-tier degrading keywords: complex boolean (A OR B) AND (C OR D) → medium AND → simple fallback. Each source adapts to its API syntax independently (OpenAlex 5 words, arXiv 2-3 words, DBLP 2-word prefix, OpenAlex_zh dual-format |||, EPO CQL). Users can edit them in the confirmation bubble before executing.

📊 Knowledge graph — graphAgent + cytoscape.js visualization

LLM extracts entities/relations (document / author / concept / topic / journal); cytoscape-fcose force-directed layout renders 500+ nodes smoothly; community detection + hubs/gaps analysis. Naturally suited to literature-survey workflows.

💸 100% BYOK — the platform doesn't pay for LLM

The desktop client writes the LLM key to OS Keychain. All scoring / summarization / collaborative research happens locally; not a single line of backend code is changed. The web version uses platform-owned keys; the desktop side gets zero-cost integration with the user's own Anthropic / OpenAI / DeepSeek / Moonshot / jiekou.ai relay / Ollama local.

🌍 14+ academic + patent sources in parallel

Academic: OpenAlex (250M+) / arXiv / EuropePMC / Crossref / DBLP / PubMed / ClinicalTrials / OpenAlex_zh Patents: EPO OPS (Europe) / Lens.org (90+ jurisdictions, 160M+) / PatentHub (China, tiered paid + PDF)

asyncio.gather parallel fetch → LLM scoring → chord-style summarization dispatch. A single round returns results in 30-60 seconds. Paid sources (PatentHub) go through a budget gate; the user can confirm again to override the soft cap.

📦 Desktop + Web dual form sharing the same backend

Form Best for Notable
🌐 Web scholarpilot.top Try it / occasional use Register and go; platform-hosted; Cloudflare Tunnel — no exposed origin port
🖥️ Desktop Tauri 2 Heavy long-term use OS Keychain stores tokens; local SQLite; BYOK self-pay; native PDF fetch bypasses anti-scrape

This main0 repo is a frozen public-version branch (2026-05-08). It's the engineering- and product-stage open-source snapshot of ScholarPilot, intended for learning and reference.


Forms

Form Path Purpose
Web Frontend (frozen) frontend/ Vue 3 + Element Plus, browser SPA
Desktop Client (active) client/ Tauri 2 + Vue 3, BYOK, local SQLite + file sandbox
Backend (CN node) backend/ FastAPI + Celery + PostgreSQL, full-stack API for the web SPA
sp-api (HK node) sp-api/ Client-only lightweight gateway, fetcher orchestration + paid-source budget gate, no LLM dependency

Architectural evolution: from single web app → "Web + Desktop + Dual-backend" trio. Web uses platform LLM keys (self-managed cost); desktop client is 100% BYOK (cost belongs to the user). See docs/architecture/.


Core capabilities

Retrieval (done)

  • Chat-as-subject: every flow (create / retrieve / collaborate / push) happens in chat bubbles, driven by natural language
  • Agent-first keyword planning: QueryPlanAgent picks the strategy automatically; per-source keyword adaptation (each source has its own API syntax)
  • Two-phase user confirmation: AI generates keyword plan → user edits and confirms → execute retrieval
  • LLM per-paper scoring: ScoringAgent gives 0-10 + one-sentence summary per paper; cutoff filter applied
  • LLM summary: for papers above the cutoff, generates summary, key points, and relevance to project
  • 4-bucket classification: highly relevant / relevant / unsure / irrelevant, persisted across rounds
  • Three-tier keyword degradation: complex boolean → medium AND → simple fallback, per-source API adaptation
  • 14+ data sources: academic (OpenAlex / arXiv / EuropePMC / Crossref / DBLP / PubMed / ClinicalTrials / OpenAlex_zh) + patent (EPO OPS / Lens.org / PatentHub)

Collaborative research (done)

  • Select existing papers to enter collaborative mode; deep Q&A grounded in the library
  • ResearchAgent reads, searches, and cites as it works; section-level probes for fine-grained reading
  • Collaboration context persisted to the conversation state machine

Two-tier Markdown memory (done)

  • Inspired by Claude Code's MEMORY.md design
  • User-level: identity / profession / overall research direction, shared across projects
  • Project-level: research direction / sub-questions for this project only
  • AI can read and write; user can view and edit; prevents cross-project contamination

Knowledge graph (done)

  • LLM extracts entities and relations; cytoscape.js force-directed (fcose) layout
  • Node types: document / author / concept / topic / journal
  • Community detection, hubs / gaps analysis

Local knowledge base (done)

  • DuckDB + SQLite FTS5; supports offline retrieval over millions of papers
  • Three modes: static / API / hybrid

Desktop client (M1 / M2 / M3 done)

  • Tauri 2 native window (Windows / macOS / Linux)
  • OS Keychain stores OAuth2 dual tokens — never localStorage
  • Local SQLite (11 tables aligned with backend) + file-system sandbox
  • BYOK: user-supplied LLM key in OS Keychain; all scoring / summarization / collaborative research happen locally
  • Native PDF fetching via Rust (reqwest + cookie store) to bypass some academic site anti-scraping

Scheduled push (in development)

  • Background process crawls latest progress on a schedule based on MEMORY
  • Desktop notification / Email / Telegram push (planned)

Harness Engineering

Inspired by Claude Code's 12-layer harness, the system ships with an extensible AI infrastructure:

Component Description
Tool Registry 14+ fetchers registered as unified tools; runtime stats + reliability reports
Hook Engine 11 lifecycle hooks (ROUND_START / POST_SEARCH / PRE_FEEDBACK, ...)
Skill System 8 reusable workflows: Deep Dive / Trend Spotter / Gap Finder / Quality Audit / Outline Synth, etc.
Agent Pipeline IntentAgent → QueryPlanAgent → ScoringAgent → MemoryAgent → CollaborationAgent → GraphAgent end-to-end
Memory Two-tier Markdown (user + project) + bucket-driven structured memory

Tech stack

Layer Tech
Desktop client Tauri 2 + Vue 3 + Element Plus + cytoscape.js + p-queue + zod
Web frontend (frozen) Vue 3 + TypeScript + Element Plus + Vite + vis-network
Client Rust backend reqwest + reqwest_cookie_store + sqlx + tauri-plugin-sql v2
Server FastAPI + SQLAlchemy (async) + Alembic + Pydantic v2
Task queue Celery + Redis
Database PostgreSQL 16 (primary) + DuckDB + SQLite FTS5 (local KB)
AI / LLM Anthropic Claude SDK / OpenAI SDK / DeepSeek / Moonshot / Ollama / OpenAI-compatible relay
Document parsing markitdown (PDF / DOCX → Markdown)
Deployment Docker Compose + Cloudflare Tunnel + nginx

Quick start

0. Try it without installing → scholarpilot.top

Open https://scholarpilot.top/ and register (invite code required — comment on Issues to ask for one, or self-host). Go to the Journal homepage → describe your research direction in natural language → complete a retrieval → give feedback in the 4 buckets → watch the next round adjust. The web version uses platform LLM keys — zero config, ready to use.

1. Web + Backend (full self-host)

git clone https://github.com/sstar16/scholarpilot
cd scholarpilot
cp .env.example .env
# Edit .env — at minimum set POSTGRES_PASSWORD, SECRET_KEY, DISABLED_SOURCES
docker compose up -d

Open http://localhost, register an account (first run needs a manual invite code or invite-code logic tweak in backend), and configure your LLM in the top-right settings.

2. Desktop client (recommended, BYOK)

cd client
npm install
npm run tauri:dev   # start Tauri dev window (first cargo build is 5-10 min)
# Or package:
npm run tauri:build  # artifacts in src-tauri/target/release/bundle/{msi,dmg,deb,appimage}/

The client connects to the sp-api gateway by default. For local dev, edit client/.env's VITE_API_BASE_URL to point at your own backend.

3. sp-api (HK client gateway, optional)

cd sp-api
docker compose -f docker-compose.sp-api.yml up -d
# In the client, set client/.env's VITE_API_BASE_URL to this sp-api

Directory structure

scholarpilot/
├── backend/                    # FastAPI + Celery + Postgres (CN node)
│   ├── app/
│   │   ├── api/                # FastAPI routes
│   │   ├── models/             # SQLAlchemy ORM
│   │   ├── schemas/            # Pydantic schemas
│   │   ├── services/
│   │   │   ├── fetchers/       # 14+ data-source Fetchers
│   │   │   ├── core/           # LLM manager + Redis persistence
│   │   │   └── ...
│   │   ├── harness/            # AI agent infrastructure (10 agents)
│   │   ├── knowledge_base/     # DuckDB + SQLite FTS5
│   │   ├── prompts/            # per-source + per-agent prompts
│   │   └── workers/            # Celery tasks
│   └── alembic/versions/       # Database migrations
├── frontend/                   # Vue 3 web SPA (frozen)
├── client/                     # Tauri 2 desktop client (active)
│   ├── src/                    # Vue 3 source (11-phase pipeline ported from backend workers)
│   ├── src-tauri/              # Rust backend (commands / fs sandbox / pdf fetcher)
│   ├── e2e/                    # Playwright tests
│   └── README.md
├── sp-api/                     # HK client gateway (fetcher only, no LLM)
├── nginx/                      # nginx config (reverse proxy for frontend / backend)
├── docs/
│   └── architecture/           # 5 Mermaid architecture diagrams + viewer
├── scripts/                    # General scripts (backup, invite-code generation, etc.)
├── docker-compose.yml
├── docker-compose.sp-api.yml
├── .env.example
├── CLAUDE.md                   # Project guide for AI coding agents
└── LICENSE

Dev workflow

# Backend
docker compose restart backend worker beat flower
docker compose logs -f backend
docker compose exec backend alembic upgrade head

# Web frontend
docker compose build frontend && docker compose up -d frontend && docker compose restart nginx

# Desktop client
cd client && npm test                  # vitest
cd client && npx vue-tsc --noEmit      # type check
cd client && npx playwright test       # e2e
cd client/src-tauri && cargo check     # Rust type check

Acknowledgements

ScholarPilot benchmarked, borrowed from, and directly depends on many outstanding open-source projects during design and implementation. Sincere thanks to them all.

Peer projects benchmarked

  • LearningCircuit/local-deep-research — Local-first deep-research assistant. We borrowed its anti-hallucination prompt constraints (Never make up sources), report-generation deduplication, three-tier citation handler strategy, News Subscriptions pattern, and MCP-server exposure design.

Direct dependencies (core stack)

Desktop client:

Server:

LLM providers:

Data-source APIs (huge thanks to their open data):

Deployment and infrastructure:

Design and methodology inspiration

  • Claude Code — The MEMORY.md index + multiple-.md-detail interpretable-memory mechanism inspired our two-tier Markdown memory design; it also inspired the multi-layer harness pattern (Hook / Tool Registry / Skill) used in Harness Engineering.
  • Anthropic Engineering Blog — Reference for agent design patterns, prompt caching, and tool-use protocol.

Tech decisions and comparison research

We did detailed comparison work at the selection stage:

  • Knowledge-graph visualization: cytoscape.js (~6.6M weekly downloads) vs vue-flow (200K) vs vis-network (269K, replaced)
  • LLM concurrency: p-queue vs p-limit (p-queue wins on priority + intervalCap)
  • TS schema: zod vs typia vs ajv (zod wins on DX)
  • Rust PDF: reqwest + cookie store + optional rquest (boring-tls) fallback
  • SQLite migration: tauri-plugin-sql v2 forward-only (accept the no-Down constraint)

Roadmap

Most major features are done or in progress:

  • ✅ Retrieval flow (11-phase state machine, client-side)
  • ✅ DevTools log-visualization platform
  • ✅ Local knowledge base (DuckDB + SQLite FTS5)
  • ✅ Chat-as-subject / collaborative research / two-tier Markdown memory
  • ✅ Knowledge graph (cytoscape)
  • ✅ Desktop client (M1 / M2 / M3 BYOK 100%)
  • 🚧 sp-api refactor (CN/HK dual backend, BYOK takes over business/LLM)
  • 🚧 Scheduled push (MEMORY-based background monitoring)
  • ⬜ MCP server (expose fetchers + agents to Claude Desktop / Cursor)
  • ⬜ Multi-user / team-shared projects
  • ⬜ Client-side SQLCipher encryption (enterprise audience)

Contributing

PRs, issues, and benchmark discussion are welcome. Before submitting, we recommend running npx vue-tsc --noEmit (client) / pytest (backend) to make sure nothing breaks. Commit format: feat: / fix: / refactor: / docs: / chore:.


License

This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0) — see LICENSE.

  • You may read, use, modify, and distribute the code freely.
  • Any derivative work, including SaaS / network deployments based on a modified copy, MUST be released under the same AGPL-3.0 terms.
  • For commercial SaaS or proprietary use without AGPL copyleft, please contact the author for a commercial license.

© 2026 sstar16

About

AI-powered academic research assistant

Resources

Stars

4 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages