Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Phi3.5-mini fails to execute using phi3-qa.py #1133

Closed
suyash-narain opened this issue Dec 9, 2024 · 1 comment
Closed

Phi3.5-mini fails to execute using phi3-qa.py #1133

suyash-narain opened this issue Dec 9, 2024 · 1 comment

Comments

@suyash-narain
Copy link

On executing phi3-qa.py using the latest commits on this script, execution of phi3.5-mini model fails.
I get the error:

$python3 phi3-qa.py -m cpu_and_mobile/cpu-int4-rtn=block-32-acc-level-4
Input: Hello, how are you?
Traceback (most recent call last):
File "/home/user/phi3-qa.py", line 92, in main(args)
File "/home/user/phi3-qa.py", line 42, in main
generator = og.Generator(model,params)
RuntimeError: input_ids not set in GeneratorParams

if i revert this change 'generator.append_tokens(input_tokens)' with params.input_ids = input_tokens and generator.compute_logits(), it works without any issues.

I am using Ubuntu 22.04 with ort v1.20.1 and gen-ai v0.5.2

@kunal-vaishnavi
Copy link
Contributor

You can see this announcement for more information on which example script to use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants