Skip to content

Llama 7B (4-bit) speed on Intel 12th or 13th generation  #1157

@Oxi84

Description

@Oxi84

Hello,

What is an average token generation speed on intel 12-13th generation CPUs?
I am sure somebody has it.

I only read here (#39), that speed for old intel with 4 cores is around 165 s/token and for AMD 5700G is around 100 ms/token.

So i thought that maybe Intel CPUs run faster than AMD ones.

Thanks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions