Repository navigation

Fix per token atrributes bits - #7749

Merged

jaime-m-p merged 12 commits into

ggml-org:masterfrom

jaime-m-p:per-token-attribs

Jun 4, 2024

Merged

Fix per token atrributes bits#7749
jaime-m-p merged 12 commits into
ggml-org:masterfrom
jaime-m-p:per-token-attribs

Commits on Jun 1, 2024
- Add per token attrib enum
  jaime-m-p
  committed
  cec6a3b
- Using phi-3 for testing 'rstrip'
  jaime-m-p
  committed
  3ead1b9
- bugfix: assertions, wrong special token list
  jaime-m-p
  committed
  33de247
- Implement 'rstrip' properly
  jaime-m-p
  committed
  ada961c
- Refactor + add 'jina-v2' for testing 'lstrip'
  jaime-m-p
  committed
  01c9229
Commits on Jun 2, 2024
- Update phi-3 GGUF file (obsolete since 917dc8c)
  jaime-m-p
  committed
  8564c19
- Update brute force test: testing 'lstrip' and 'rstrip'
  jaime-m-p
  committed
  83cac0e
- Fix previous commit
  jaime-m-p
  committed
  54e9f23
- Replace llama_token_type with llama_token_attribs
  jaime-m-p
  committed
  ac40ff0
Commits on Jun 3, 2024
- Rename token attributes
  jaime-m-p
  committed
  18f5fc7
Commits on Jun 4, 2024
- Fix per token atrributes bits
  jaime-m-p
  committed
  f95c7dc
- Merge branch 'master' into per-token-attribs
  jaime-m-p
  authored
  6a0f3db