The prompt is not converted to tokens

./main -m ./models/7B/ggml-model-q4_0.bin -p "Building a website can be done in 10 simple steps:" -t 8 -n 512

llama.cpp Is the latest version
Can anyone help me? Thanks!

@ggerganov

@ggerganov Please，Thanks！

I'm seeing the same issue. Model works fine in f16 version, but doesn't work after quantization

Same issue here. Everything is fine on my mac laptop but after I deployed it on Linux (Centos 7), I found this issue.
Have tried to change the version of gcc, python, torch, and tried to modify the source code of 'llama_tokenize' to make the tokenizer working as expected. But none of these works. And also checked md5 sum for all files, all of the md5 sum are right.

Environment:
Mac (works fine): gcc 9.4.0, Python 3.9.6, Torch 1.9.0
Centos 7 (issue): gcc 9.3.1, Python 3.9.16, Torch 1.9.1

I saw in some other issues that some people tried making sure their project was up to date and then rebuilt it. I rebuilt mine, requantized, then ran it, and it started working. I have no idea why 😬

…

On Fri, Mar 17, 2023, 04:28 Casey Lai ***@***.***> wrote: Same issue here. Everything is fine on my mac laptop but after I deployed it on Linux (Centos 7), I found this issue. Have tried to change the version of gcc, python, torch, and tried to modify the source code of 'llama_tokenize' to make the tokenizer working as expected. But none of these works. And also checked md5 sum for all files, all of the md5 sum are right. Environment: Mac (works fine): gcc 9.4.0, Python 3.9.6, Torch 1.9.0 Centos 7 (issue): gcc 9.3.1, Python 3.9.16, Torch 1.9.1 — Reply to this email directly, view it on GitHub <#113 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AYMC3ACXD3YF5F2FAYIQWMTW4QOC7ANCNFSM6AAAAAAVZ5AVUY> . You are receiving this because you commented.Message ID: <ggerganov/llama. ***@***.***>

@zhoujian1028

./main -m ./models/7B/ggml-model-q4_0.bin -p "Building a website can be done in 10 simple steps:" -t 8 -n 512

llama.cpp Is the latest version Can anyone help me? Thanks!

@zhoujian1028 I tried to deploy on another machine and it worked smoothly, for your reference: Ubuntu 22.04 + gcc11.3.0 + python 3.10.9 + torch 1.13.1

./main -m ./models/7B/ggml-model-q4_0.bin -p "Building a website can be done in 10 simple steps:" -t 8 -n 512

llama.cpp Is the latest version Can anyone help me? Thanks!

Centos 7 even with sudo yum install devtoolset-11-gcc* can't work and output nothing meaningful, but just using an docker conatiner base on ubuntu jemmy with sudo apt-get install build-essential everything works fine in fp16.

Give it another try now that #252 landed.

No complaints after three weeks, let's assume this is fixed, possibly by #252.

CI Fix (temp-7248): Corrupted .gguf, incorrectly cached native CPU bu…

xdna: ggml-org#113 add ensure_compiled positive cache (skip disk I/O …

xdna: ggml-org#113 enable xrt::run cache by default in f3best LOOP

xdna: ggml-org#113 add paris_short_q4_0_f3best_loop correctness test

rdna-boosts: block 05: flash-attention band, native quantized K/V and…

Navigation Menu

Repository navigation

The prompt is not converted to tokens #113

44 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

The prompt is not converted to tokens #113

Description

Activity

zhoujian1028 commented on Mar 15, 2023

MillionthOdin16 commented on Mar 17, 2023

caseylai commented on Mar 17, 2023

MillionthOdin16 commented on Mar 17, 2023

caseylai commented on Mar 18, 2023

xlichao commented on Mar 18, 2023

ar-jan commented on Mar 20, 2023

sw commented on Apr 7, 2023

44 remaining items

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions