Repository navigation

llama : refactor llama_kv_cache, llama_context and llm_build_context - #11213

Closed

ggerganov wants to merge 95 commits into

masterfrom

gg/llama-kv-cache

Closed

llama : refactor llama_kv_cache, llama_context and llm_build_context#11213
ggerganov wants to merge 95 commits into
masterfrom
gg/llama-kv-cache

Commits on Jan 26, 2025
- llama : add struct llama_kv_cache (wip) [no ci]
  ggerganov
  committed
  f78b396
- llama : cont
  ggerganov
  committed
  e4550fb
- kv_cache : functions -> members
  ggerganov
  committed
  4d7bd03
- kv_cache : fix
  ggerganov
  committed
  fef90cb
- kv_cache : minor
  ggerganov
  committed
  73a14ec
- context : prepare kv_cache_read/write to be moved to kv_cache
  ggerganov
  committed
  4cd1b6f
- kv_cache : move state read/write to llama_kv_cache
  ggerganov
  committed
  fd05ab8
- llama : update llama_kv_self API
  ggerganov
  committed
  17b363a
- context : minor
  ggerganov
  committed
  a19f671
- llama : fix names [no ci]
  ggerganov
  committed
  ae274f9
- llama : remove references to llama_kv_cache (wip)
  ggerganov
  committed
  f2524c0
- cont : move kv_self update to llama_context
  ggerganov
  committed
  b4ec1d4
- context : add get_ctx_padding()
  ggerganov
  committed
  f071349
- context : move adapter code in the implementation [no ci]
  ggerganov
  committed
  c75ba68
- context : initial need_reserve logic
  ggerganov
  committed
  133ad6a
- wip
  ggerganov
  committed
  cb8f209
- context : introduce llama_batch_manager
  ggerganov
  committed
  99422df
- context : prepare for abstraction
  ggerganov
  committed
  a0c500b
Commits on Jan 27, 2025
- Merge branch 'master' into gg/llama-kv-cache
  ggerganov
  committed
  e665b57
Commits on Jan 29, 2025
- llama : resolve rwkv conflict
  ggerganov
  committed
  9188856
- Merge branch 'master' into gg/llama-kv-cache
  ggerganov
  committed
  c30e34c
Commits on Jan 30, 2025
- Merge branch 'master' into gg/llama-kv-cache
  ggerganov
  committed
  a40ba49
Commits on Jan 31, 2025
- Merge branch 'master' into gg/llama-kv-cache
  ggerganov
  committed
  5d3491e
Commits on Feb 2, 2025
- context : store graph build function callback
  ggerganov
  committed
  3e23be7
- Merge branch 'master' into gg/llama-kv-cache
  ggerganov
  committed
  74b0807
Commits on Feb 3, 2025
- llama : fix rwkv inference (#11618)
  MollySophia
  authored
  1eca891
Commits on Feb 6, 2025
- llama : clear whitespaces
  ggerganov
  committed
  e0d913f
- Merge branch 'master' into gg/llama-kv-cache
  ggerganov
  committed
  0f1c1ca
- kv-cache : fix defrag condition
  ggerganov
  committed
  b15fede
Commits on Feb 10, 2025
- Merge branch 'master' into gg/llama-kv-cache
  ggerganov
  committed
  972f91c
- llama : dedup reserve code
  ggerganov
  committed
  f9971ef
- server : increase context size for the tests
  ggerganov
  committed
  879ba82
- context : add decode/encode
  ggerganov
  committed
  ef358ee
- bman : remove ubatch member
  ggerganov
  committed
  d1d8d53
- context : make output functions members
  ggerganov
  committed
  2cd8a90
Commits on Feb 11, 2025
- context : initial abstraction
  ggerganov
  committed
  02ef4be
Commits on Feb 12, 2025
- context : move encode/decode to llama-context.cpp
  ggerganov
  committed
  b52b79b
- context : improve llama_context encapsulation
  ggerganov
  committed
  8da7f61
- context : minor naming fix
  ggerganov
  committed
  d146a14
- context : move build_rope_factors to base class
  ggerganov
  committed
  5eae8e5
- context : introduce llama_graph_i
  ggerganov
  committed
  e633dc1
- context : prepare llama_model graph build
  ggerganov
  committed
  0ab50f1
- llama : models now build their graphs using llama_graph_i
  ggerganov
  committed
  f63aeec
- graph : restore ubatch in build_cb
  ggerganov
  committed
  6ee86e5
- context : rename to llama_context_kv_self
  ggerganov
  committed
  fbe6a07
Commits on Feb 13, 2025
- llama : introduce llama_io interfaces
  ggerganov
  committed
  3a504d9
- context : abstract state read/write
  ggerganov
  committed
  f7c7757
- context : minor cleanup
  ggerganov
  committed
  e08f38d
- context : move output functionality to base class
  ggerganov
  committed
  107d1e2
- context : abstract input
  ggerganov
  committed
  ed3cb55
- context : abstract constructor and init
  ggerganov
  committed
  131743f
Commits on Feb 14, 2025
- context : remove batch_manager
  ggerganov
  committed
  d5e8e1a
- context : move common inputs to base class
  ggerganov
  committed
  8280645
- graph : update attn/kv_self names
  ggerganov
  committed
  1d801d2
Commits on Feb 18, 2025
- Merge branch 'master' into gg/llama-kv-cache
  ggerganov
  committed
  f0d3ff2
- graph : add llama_graph_result
  ggerganov
  committed
  c235903
- cont : return important tensors
  ggerganov
  committed
  172f616
- cont : use returend tensors from the graph build
  ggerganov
  committed
  bc6f187
- llama : reorder encode/decode in sources
  ggerganov
  committed
  befe14f
- context : minor simplify
  ggerganov
  committed
  9e50456
- model : pass llama_graph_i as ptr
  ggerganov
  committed
  2bffc2d
- kv-cache : prepare for abstraction
  ggerganov
  committed
  f5cedbc
Commits on Feb 19, 2025
- kv-cache : remove llama_kv_cache_i
  ggerganov
  committed
  5f11a55
- context : add llama_context_recurrent
  ggerganov
  committed
  e17e4b7
- graph : simplify attention api
  ggerganov
  committed
  2eacb4c
- model : fix order kvq -> qkv
  ggerganov
  committed
  f95b04a
Commits on Feb 20, 2025
- Merge branch 'master' into gg/llama-kv-cache
  ggerganov
  committed
  072280e
- context : add cache-less llama_context
  ggerganov
  committed
  b1554be
- context : fix causal input for cache-less case
  ggerganov
  committed
  ad870c4
- context : add llama_kv_cache_recurrent prototype
  ggerganov
  committed
  08011c2
Commits on Feb 21, 2025
- context : add save/load for recurrent context
  ggerganov
  committed
  2645a7d
- graph : remove worst_case from the API
  ggerganov
  committed
  548c230
- context : add logs
  ggerganov
  committed
  ebf1bdf
- context : wrap input tensors in struct
  ggerganov
  committed
  f588a70
- context : fix n_outputs init
  ggerganov
  committed
  3753b30
- Merge branch 'master' into gg/llama-kv-cache
  ggerganov
  committed
  c4c0a4d
- wip enc-dec
  ggerganov
  committed
  f5e8020
Commits on Feb 23, 2025
- cont : enc should work now, next is dec
  ggerganov
  committed
  372fa3a
- graph : remove the build_kv_... API from llama_graph_i
  ggerganov
  committed
  6378112
- context : remove redundant virtual, protected -> private
  ggerganov
  committed
  0699a44
Commits on Feb 24, 2025
- context : fix recurrent reserve
  ggerganov
  committed
  a5a85a3
- context : reuse built_attn_mha
  ggerganov
  committed
  4a1054b
- context : explicit llama_context_i abstract interface
  ggerganov
  committed
  9cd78f1
- enc-dec : compose wip
  ggerganov
  committed
  be58e30
Commits on Feb 25, 2025
- context : enc-dec is now working
  ggerganov
  committed
  e5bc5f8
- context : fix enc-dec state save/load
  ggerganov
  committed
  e2b3294
- context : pass embeddings tensor from encoder to decoder
  ggerganov
  committed
  4efe989
Commits on Feb 27, 2025
- context : disable encoder embd tensor for now
  ggerganov
  committed
  952feed
- Merge branch 'master' into gg/llama-kv-cache
  ggerganov
  committed
  82675a0
- kv-cache : basic abstraction
  ggerganov
  committed
  828effd
Commits on Feb 28, 2025
- llama : introduce concept of llama_memory
  ggerganov
  committed
  38db8a5
- context : decouple inputs, llama_graph_i become const (WIP)
  ggerganov
  committed
  7f02ee5
- cont : migrate the rest of the inputs out of llama_context
  ggerganov
  committed
  9cab53c
- graph : move non-context related logic to llm_build_context
  ggerganov
  committed
  0f7daa9
- graph : add comments
  ggerganov
  committed
  624f7bd

Commits

Commits on Jan 26, 2025

Commits on Jan 27, 2025

Commits on Jan 29, 2025

Commits on Jan 30, 2025

Commits on Jan 31, 2025

Commits on Feb 2, 2025

Commits on Feb 3, 2025

Commits on Feb 6, 2025

Commits on Feb 10, 2025

Commits on Feb 11, 2025

Commits on Feb 12, 2025

Commits on Feb 13, 2025

Commits on Feb 14, 2025

Commits on Feb 18, 2025

Commits on Feb 19, 2025

Commits on Feb 20, 2025

Commits on Feb 21, 2025

Commits on Feb 23, 2025

Commits on Feb 24, 2025

Commits on Feb 25, 2025

Commits on Feb 27, 2025

Commits on Feb 28, 2025