llm-chat-cpp — MCP-Powered Terminal Chat

A C++ terminal chat application that connects to llama.cpp and an MCP server for tool-assisted conversations.

Quick Start (Docker)

The recommended way to run llm-chat is Docker — no compilers or dev headers needed on your machine.

# 1. Build the image
docker build -t llm-chat .

# 2. Configure MCP servers
cp mcp.example.json mcp.json   # edit as needed

# 3. Run
docker compose run --rm llm-chat

llama.cpp must be running on the host (llama-server -m <model.gguf> --port 8080). The container reaches host services via host.docker.internal, which is already configured in docker-compose.yml.

Docker

Build

docker build -t llm-chat .

Run

The image is a multi-stage build on debian:bookworm-slim; the repo root is mounted read-only at /app so the app picks up your mcp.json.

docker run -it --rm \
  -e LLM_CHAT_LLM_URL=http://host.docker.internal:8080 \
  -e LLM_CHAT_MCP_URL=http://host.docker.internal:8000/mcp \
  -v "$PWD":/app:ro \
  llm-chat

Compose

docker-compose.yml already sets the TTY, host.docker.internal, environment defaults and the repo mount:

docker compose run --rm llm-chat

Pass extra flags after the service name:

docker compose run --rm llm-chat --llm-url http://10.0.0.1:8080 --mcp-config /app/mcp.json

Override defaults via environment variables in your shell:

LLM_CHAT_MAX_CONTEXT=4096 docker compose run --rm llm-chat

Native Build (optional)

Build from source only if you don't want Docker.

Requirements

• C++17 compiler (g++ ≥ 9, clang ≥ 10, or MSVC 2019+)
• CMake ≥ 3.16
• libcurl (development headers)
• GNU Readline (development headers, Linux only)
• nlohmann/json (development headers)
• vcpkg (Windows only)

Ubuntu/Debian

sudo apt install build-essential cmake libcurl4-openssl-dev libreadline-dev nlohmann-json3-dev

Arch Linux

sudo pacman -S base-devel cmake curl readline nlohmann-json

Fedora

sudo dnf install gcc-c++ cmake libcurl-devel readline-devel nlohmann-json-devel

Build

Linux

cmake -B build
cmake --build build

Binary: build/llm-chat

Windows (MSVC + vcpkg)

cmake -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=<vcpkg_root>\scripts\buildsystems\vcpkg.cmake
cmake --build build --config Release

Binary: build\Release\llm-chat.exe

Dependencies (curl, nlohmann-json) are installed automatically via vcpkg manifest (vcpkg.json).

Usage

1. Start llama.cpp

On the host machine:

llama-server -m <model.gguf> --port 8080

2. Configure MCP Server

Create a mcp.json file in the working directory (in Docker: the repo root, mounted at /app/mcp.json):

{
  "mcpServers": {
    "exa-search": {
      "type": "http",
      "url": "https://mcp.exa.ai/mcp"
    },
    "local-tools": {
      "type": "http",
      "url": "http://localhost:8000/mcp"
    }
  }
}

Docker note: URLs in mcp.json are resolved from inside the container. Use host.docker.internal instead of localhost for services running on the host (e.g. http://host.docker.internal:8000/mcp).

3. Run llm-chat

Docker:

docker compose run --rm llm-chat

Native:

./build/llm-chat

If MCP connection fails at startup, the app continues without tools. Use /mcp connect later to retry.

Command Line Options

Option	Description
--llm-url URL	llama.cpp server address
--mcp-url URL	MCP server address (overrides mcp.json)
--mcp-config PATH	Path to mcp.json configuration file
--help	Show usage

docker compose run --rm llm-chat --llm-url http://10.0.0.1:8080 --mcp-config /app/mcp.json

Native equivalent:

./build/llm-chat --llm-url http://10.0.0.1:8080 --mcp-config ./mcp.json

Environment Variables

Variable	Default	Description
LLM_CHAT_LLM_URL	http://localhost:8080	llama.cpp server address
LLM_CHAT_MCP_URL	http://localhost:8000/mcp	MCP server address
LLM_CHAT_MAX_CONTEXT	8192	Max context token count (for percentage display)

In Docker Compose the URL defaults point at host.docker.internal instead of localhost (see docker-compose.yml); override them the same way:

export LLM_CHAT_LLM_URL="http://host.docker.internal:8080"
export LLM_CHAT_MCP_URL="http://host.docker.internal:8000/mcp"
export LLM_CHAT_MAX_CONTEXT=4096
docker compose run --rm llm-chat

Native:

export LLM_CHAT_LLM_URL="http://10.0.0.1:8080"
export LLM_CHAT_MCP_URL="http://10.0.0.1:8000/mcp"
export LLM_CHAT_MAX_CONTEXT=4096
./build/llm-chat

Note: If the llama.cpp server provides n_ctx (or max_context_length) in the /v1/models endpoint, it overrides LLM_CHAT_MAX_CONTEXT.

Commands

Command	Description
/quit or /exit	Exit
/help	Show command list
/clear	Clear chat history
/tools	List MCP tools
/mcp	Show MCP connection status
/mcp connect [server]	Connect to MCP server (all, or one by name)
/mcp disconnect	Disconnect from MCP server
/read	Add file to context
/files	List attached files
/remove	Remove attached file
/clearfiles	Remove all attached files
/models	List / switch / unload models

Architecture

┌─────────────┐      ┌──────────────┐      ┌──────────────┐
│  llm-chat   │─────▶│  llama.cpp   │      │  MCP Server  │
│ (container  │      │  (host)      │      │  (HTTP)      │
│  or native) │      │  :8080       │      │  JSON-RPC    │
│             │      │  /v1/chat/   │      │  tools/list  │
│             │      │  completions │      │  tools/call  │
│             │      └──────────────┘      └──────────────┘
│             │ host.docker.internal       Supports SSE
│             │ mcp.json                   and JSON
│             │ auto-discovery
└─────────────┘

1. App searches for mcp.json in exe dir, then working dir
2. User message → llama.cpp API (with tool definitions)
3. If model returns tool_calls → execute via MCP
4. Tool result sent back to model
5. Model final response displayed to user
6. Stats shown after each response (tokens, time, t/s, context %)

MCP Configuration

The app reads mcp.json from (in order):

1. Same directory as the executable
2. Current working directory (in Docker: /app, i.e. the repo root)

You can also specify a custom path with --mcp-config.

Standard mcp.json Format

{
  "mcpServers": {
    "server-name": {
      "type": "http",
      "url": "https://example.com/mcp",
      "headers": {
        "Authorization": "Bearer token"
      },
      "enabled": true
    }
  }
}

Field	Required	Description
type	Yes	Must be "http"
url	Yes	MCP server endpoint URL
headers	No	Additional HTTP headers
enabled	No	Set false to skip (default: true)