Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inference not accurate #35

Open
Hamad-exarta opened this issue Jul 18, 2023 · 0 comments
Open

Inference not accurate #35

Hamad-exarta opened this issue Jul 18, 2023 · 0 comments

Comments

@Hamad-exarta
Copy link

Hamad-exarta commented Jul 18, 2023

I used the same dataset provided to fine tune , but the inference seems to give general replies and not from the dataset used for fine tuning.

Any Ideas?

"instruction": "What are the three primary colors?",
"input": "",
"output": "The three primary colors are red, blue, and yellow. These colors are called primary because they cannot be created by mixing other colors and all other colors can be made by combining them in various proportions. In the additive color system, used for light, the primary colors are red, green, and blue (RGB)."

The above was part of the dataset , but when i run inference i get the following

_**Enter new instruction: Describe the structure of an atom.

An atom consists of a nucleus, made up of positively charged protons and neutral electrons, and a number of electrons arranged in shells or layers around the nucleus. The number of electrons in the outermost shell of the nucleus determines the atom's electronic**_

i used the following generate script

falcontune generate
--interactive
--model falcon-7b-instruct-4bit
--weights ./gptq_model-4bit-64g.safetensors
--lora_apply_dir falcon-7b-instruct-4bit-alpaca-selftrained/
--max_new_tokens 50
--use_cache
--do_sample
--instruction "What are the three primary colors?"
--backend triton

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant