Skip to content

Windows: 1bit.exe and a Vulkan llama-server.exe, cross-built on Linux - #99

Merged
bong-water-water-bong merged 3 commits into
mainfrom
win/core
Sep 25, 2026
Merged

bong-water-water-bong merged 3 commits into
mainfrom
win/core

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

scripts/build-windows.sh <out-dir> cross-builds the engine for Windows 10+ x64 on Linux. It produces two single static executables: 1bit.exe (serve with Vulkan, route, comfy) and llama-server.exe built from the Vulkan pin. 1bit.exe finds llama-server.exe beside it. See docs/windows.md.

Code changes

  • app/serve.cpp:
    • Backends start with CreateProcess inside a kill-on-close job object, so they die with 1bit.exe for any reason; this is Windows' counterpart to PR_SET_PDEATHSIG.
    • free_port uses Winsock, and signals go through std::signal.
    • The POSIX paths are unchanged.
  • npu/file_map.{h,cpp}: one read-only file mapping (mmap / CreateFileMapping) for npu/model.cpp and laya/safetensors.cpp.
  • app/main.cpp: 1bit comfy spawns and waits on Windows.
  • CMakeLists.txt:
    • Windows 10 API level for cpp-httplib;
    • its non-blocking getaddrinfo off (MinGW lacks GetAddrInfoExCancel, and the engine only connects to 127.0.0.1);
    • static linking with MinGW.

The script fetches pinned, sha256-checked inputs: llvm-mingw (clang 23: the engine is C++26), PCRE2 10.48, Vulkan-Headers, the Vulkan loader's vulkan-1.def (turned into the import library with llvm-dlltool, so no Vulkan SDK or Windows machine is needed) and SPIRV-Headers. The pinned llama.cpp uses std::function without <functional> in ggml-vulkan-types.h, which libc++ does not include transitively, so the script force-includes it.

Tested on Strix Halo under Wine 11, on the real Radeon 8060S via winevulkan and RADV, from a clean clone:

  • The script built both executables in about 2 minutes.
  • 1bit.exe serve -m Qwen3-0.6B-Q4_K_M.gguf, with no --llama-server, found llama-server.exe beside it, answered "The capital of France is Paris." at 132 tok/s, and streamed SSE.
  • taskkill /F /IM 1bit.exe left no backend running.
  • The Linux build of the same sources passes ctest 6/6, smoke_serve included.
  • Not yet run on a real Windows machine.

🤖 Generated with Claude Code

bong-water-water-bong and others added 3 commits September 25, 2026 18:29
… (scripts/build-windows.sh)

serve's child backends start with CreateProcess in a kill-on-close job object
on Windows (the Linux PR_SET_PDEATHSIG), free_port uses Winsock, 1bit comfy
spawns and waits. Model and safetensors files map through npu/file_map.{h,cpp}
(mmap / CreateFileMapping). httplib: Windows 10 API level, non-blocking
getaddrinfo off (MinGW lacks GetAddrInfoExCancel); MinGW links statically.
scripts/build-windows.sh fetches pinned, checksummed llvm-mingw, PCRE2,
Vulkan-Headers, the loader's vulkan-1.def and SPIRV-Headers, then builds
1bit.exe and llama-server.exe from the Vulkan pin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… into the logs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…how it was tested

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@context7

context7 Bot commented Sep 25, 2026

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan. —

Commit 9c53e1e

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Windows ComfyUI Execution

On Windows, the run_comfy function uses _spawnvp to execute comfyui_cpp.exe and waits for it. This approach may not handle all edge cases of process lifecycle management as gracefully as the POSIX fexecve version, especially in terms of error propagation and cleanup if the spawned process fails to start or exits unexpectedly.

#ifdef _WIN32
// Windows: $ONEBIT_COMFYUI (an absolute path), then this build's, then comfyui_cpp.exe on PATH;
// run as a child that 1bit waits for (no exec in place on Windows).
int run_comfy(int argc, char** argv) {
    if (argc < 1 || !std::strcmp(argv[0], "-h") || !std::strcmp(argv[0], "--help")) {
        std::printf("usage: 1bit comfy <workflow.json>\n"
                    "  runs a ComfyUI API-format workflow (SD1.5 txt2img/img2img, see docs/comfyui.md)\n");
        return argc < 1 ? 2 : 0;
    }
    std::string bin;
    if (const char* e = std::getenv("ONEBIT_COMFYUI"); e && *e) {
        if (!std::filesystem::path(e).is_absolute())
            throw std::runtime_error("ONEBIT_COMFYUI must be an absolute path, not " + std::string(e));
        bin = e;
    }
#ifdef ONEBIT_COMFYUI_BIN
    if (bin.empty() && std::filesystem::exists(ONEBIT_COMFYUI_BIN)) bin = ONEBIT_COMFYUI_BIN;
#endif
    if (bin.empty()) bin = "comfyui_cpp.exe";   // _spawnvp searches PATH
    std::vector<const char*> args{bin.c_str()};
    for (int i = 0; i < argc; ++i) args.push_back(argv[i]);
    args.push_back(nullptr);
    const intptr_t rc = ::_spawnvp(_P_WAIT, bin.c_str(), args.data());
    if (rc < 0) throw std::runtime_error("cannot run " + bin + ": " + std::strerror(errno));
    return int(rc);
}
Windows Process Management

The Child class on Windows uses CreateProcess within a job object to ensure child processes are killed when the parent dies. However, there's a potential issue with the AssignProcessToJobObject call happening after CreateProcess but before ResumeThread. If ResumeThread fails, the process might not be properly associated with the job, leading to orphaned processes.

#ifdef _WIN32
// Windows: CreateProcess into a job object that kills its processes when its last handle
// closes, so the backend dies with `1bit serve` for any reason (Linux: PR_SET_PDEATHSIG).
class Child {
public:
    Child(const std::vector<std::string>& argv, const std::vector<std::string>& env_extra, int port)
        : port_(port) {
        // the command line, each argument quoted the way CommandLineToArgvW reads it back
        std::string cmd;
        for (const auto& a : argv) {
            if (!cmd.empty()) cmd += ' ';
            cmd += '"';
            size_t bs = 0;
            for (char ch : a) {
                if (ch == '\\') { bs++; continue; }
                if (ch == '"') cmd.append(bs * 2 + 1, '\\');
                else cmd.append(bs, '\\');
                bs = 0;
                cmd += ch;
            }
            cmd.append(bs * 2, '\\');
            cmd += '"';
        }
        // our environment plus env_extra (a value the user set wins), as a double-NUL block
        std::vector<std::string> env_store;
        if (char* block = ::GetEnvironmentStringsA()) {
            for (char* e = block; *e; e += std::strlen(e) + 1) env_store.emplace_back(e);
            ::FreeEnvironmentStringsA(block);
        }
        for (const auto& e : env_extra) {
            const std::string key = e.substr(0, e.find('=') + 1);
            bool present = false;
            for (const auto& have : env_store) present = present || have.rfind(key, 0) == 0;
            if (!present) env_store.push_back(e);
        }
        std::string env_block;
        for (const auto& e : env_store) { env_block += e; env_block += '\0'; }
        env_block += '\0';
        job_ = ::CreateJobObjectA(nullptr, nullptr);
        if (job_) {
            JOBOBJECT_EXTENDED_LIMIT_INFORMATION li{};
            li.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;
            ::SetInformationJobObject(job_, JobObjectExtendedLimitInformation, &li, sizeof li);
        }
        STARTUPINFOA si{};
        si.cb = sizeof si;
        PROCESS_INFORMATION pi{};
        if (!::CreateProcessA(nullptr, cmd.data(), nullptr, nullptr, FALSE, CREATE_SUSPENDED, env_block.data(), nullptr,
                              &si, &pi))
            throw std::runtime_error("cannot start " + argv[0] + " (error " + std::to_string(::GetLastError()) + ")");
        if (job_) ::AssignProcessToJobObject(job_, pi.hProcess);   // before it runs: its children join too
        ::ResumeThread(pi.hThread);
        ::CloseHandle(pi.hThread);
        proc_ = pi.hProcess;
    }
    ~Child() {
        stop();
        if (job_) ::CloseHandle(job_);
    }
    Child(const Child&) = delete;
    Child& operator=(const Child&) = delete;

    void stop() {
        if (!proc_) return;
        ::TerminateProcess(proc_, 1);   // no SIGTERM on Windows; llama-server keeps no state to flush
        ::WaitForSingleObject(proc_, 5000);
        ::CloseHandle(proc_);
        proc_ = nullptr;
    }

    bool alive() const { return proc_ && ::WaitForSingleObject(proc_, 0) == WAIT_TIMEOUT; }
Windows File Mapping Error Handling

In map_file on Windows, if CreateFileMappingA or MapViewOfFile fails, the function returns nullptr and sets why. However, there's no explicit check for sz.QuadPart being zero before attempting to create the file mapping, which could lead to unexpected behavior if the file size is zero.

const uint8_t* map_file(const std::string& path, size_t& size, std::string* why) {
    size = 0;
#ifdef _WIN32
    const HANDLE f = ::CreateFileA(path.c_str(), GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING,
                                   FILE_ATTRIBUTE_NORMAL, nullptr);
    if (f == INVALID_HANDLE_VALUE) {
        if (why) *why = "cannot open " + path;
        return nullptr;
    }
    LARGE_INTEGER sz{};
    ::GetFileSizeEx(f, &sz);
    const HANDLE m = sz.QuadPart ? ::CreateFileMappingA(f, nullptr, PAGE_READONLY, 0, 0, nullptr) : nullptr;
    void* p = m ? ::MapViewOfFile(m, FILE_MAP_READ, 0, 0, 0) : nullptr;
    if (m) ::CloseHandle(m);
    ::CloseHandle(f);
    if (!p) {
        if (why) *why = "cannot map " + path;
        return nullptr;
    }
    size = size_t(sz.QuadPart);
    return static_cast<const uint8_t*>(p);
Model File Mapping

The Model class now uses map_file to map the model file. If map_file fails to map the file, it throws a std::runtime_error. This is a good approach, but it's important to ensure that the error message is informative enough to help diagnose issues with model loading on Windows.

Model::Model(const std::string& dir) {
    const std::string path = dir + "/model.q4nx";
    std::string why;
    const uint8_t* m = map_file(path, size_, &why);
    if (!m) throw std::runtime_error(why);
    map_ = const_cast<uint8_t*>(m);

@bong-water-water-bong
bong-water-water-bong merged commit 3c01ba3 into main Sep 25, 2026
5 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the win/core branch September 25, 2026 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant