Eval bug: K2-Horizon models fail to load

Name and Version

llama-cli --version
version: 0.3.0-dev (build 1, commit 95ef7fc)
built with GNU 15.2.0 for Linux x86_64

Operating systems

Linux

GGML backends

CUDA

Hardware

Intel(R) Core(TM) i9-14900HX
NVIDIA GeForce RTX 4060

Models

K2-Horizon-1B from https://huggingface.co/IFM/K2-Horizon-0.9B-GGUF

Problem description & steps to reproduce

The GGUF quantized model cannot run, error "unknown model architecture: 'k2-horizon'"

First Bad Commit

No response

Relevant log output

Logs

0.02.045.331 I srv    load_model: loading model 'K2-Horizon-1B-BF16.gguf'
0.02.084.496 E llama_model_load: error loading model: unknown model architecture: 'k2-horizon'
0.02.084.497 E llama_model_load_from_file_impl: failed to load model
0.02.086.338 E common_fit_params: encountered an error while trying to fit params to free device memory: failed to load model
0.02.105.311 E llama_model_load: error loading model: unknown model architecture: 'k2-horizon'
0.02.105.317 E llama_model_load_from_file_impl: failed to load model
0.02.105.318 E cmn  common_init_: failed to load model 'K2-Horizon-1B-BF16.gguf'
0.02.105.319 E srv    load_model: failed to load model, 'K2-Horizon-1B-BF16.gguf'
0.02.105.320 I srv    operator(): operator(): cleaning up before exit...
0.02.106.350 E srv  llama_server: exiting due to model loading error

According to the hf model card:

Compatibility: These models require a version of llama.cpp containing K2 Horizon architecture support. PR to llama.cpp is in progress. MBZUAI-IFM fork of llama.cpp is in https://github.com/MBZUAI-IFM/llama.cpp/tree/model/K2Horizon

~
❯ llama serve -m /mnt/D/Mymodels/K2-Horizon-7B-Q6_K.gguf  -ngl 99 \
        -c 20384 \
        --jinja -np 1
0.00.137.441 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.137.656 W srv  llama_server: -----------------
0.00.137.658 W srv  llama_server: CORS is set to allow all origins ('*') and no API key is set
0.00.137.658 W srv  llama_server: this can be a security risk (cross-origin attacks)
0.00.137.658 W srv  llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655
0.00.137.658 W srv  llama_server: -----------------
0.00.138.900 I srv    load_model: loading model '/mnt/D/Mymodels/K2-Horizon-7B-Q6_K.gguf'
0.00.497.953 E llama_model_load: error loading model: unknown model architecture: 'k2-horizon'
0.00.497.957 E llama_model_load_from_file_impl: failed to load model
0.00.497.974 E common_fit_params: encountered an error while trying to fit params to free device memory: failed to load model
0.00.587.002 E llama_model_load: error loading model: unknown model architecture: 'k2-horizon'
0.00.587.005 E llama_model_load_from_file_impl: failed to load model
0.00.587.007 E cmn  common_init_: failed to load model '/mnt/D/Mymodels/K2-Horizon-7B-Q6_K.gguf'
0.00.587.012 E srv    load_model: failed to load model, '/mnt/D/Mymodels/K2-Horizon-7B-Q6_K.gguf'
0.00.587.014 I srv    operator(): operator(): cleaning up before exit...
0.00.587.413 E srv  llama_server: exiting due to model loading error

~

Yup doesnt work

No upstream support yet for this model. This issue should be closed.

No upstream support yet for this model. This issue should be closed.

Why close it? The issue exists to track a lack of support, and the lack of support currently still exists regardless of if the issue is closed or not. If it's an issue with core GGML libraries, then someone could file a bug there and link it here - this issue tracks the lack of support, not "someone should make a PR for this here," necessarily.

See relevant discussion #28308

Any plans to enhance llama.cpp to support K2 Horizon?

It's currently the best MoE model under 40B according to the Artificial Analysis Intelligence Index v4.3.

fleet-patches: reference upstream tracking issue for K2-Horizon support

still not merge?

Navigation Menu

Repository navigation

Eval bug: K2-Horizon models fail to load #28361

Name and Version

Operating systems

GGML backends

Hardware

Models

Problem description & steps to reproduce

First Bad Commit

Relevant log output

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Eval bug: K2-Horizon models fail to load #28361

Description

Name and Version

Operating systems

GGML backends

Hardware

Models

Problem description & steps to reproduce

First Bad Commit

Relevant log output

Activity

N3RDIUM commented on Sep 4, 2026

AnuAnu22 commented on Sep 7, 2026

krenax commented on Sep 7, 2026

RetroDev256 commented on Sep 7, 2026

Terseus commented on Sep 7, 2026

BVEsun commented on Sep 15, 2026

toolyq commented on Sep 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions