Always returning Cache'd Grammar Object

[UYousafzai]

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

So I am trying to generate a grammar Object from the string generated, however it appends an older grammar text to the end of it, despite confirming the text being passed to the LlamaGrammar object is newer.

Current Behavior

Grammar String passed:

root ::= (" "| "\n") grammar-models
grammar-models ::= more
more ::= "{" "\n" ws ""title"" ":" ws string "," "\n" ws ""author"" ":" ws string "\n" ws "}"
boolean ::= "true" | "false"
null ::= "null"
string ::= """ (
[^"\\] |
"\" (["\/bfnrt] | "u" [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F])
)* """ ws
ws ::= ([ \t\n] ws)?
float ::= ("-"? ([0-9] | [1-9] [0-9]*)) ("." [0-9]+)? ([eE] [-+]? [0-9]+)? ws
integer ::= [0-9]+

---

Generated Grammar Object (Coming from the LlamaGrammar since it is working in verbose mode

from_string grammar:
root ::= root_1 grammar-models
root_1 ::= [ ] | [<U+000A>]
grammar-models ::= more
more ::= [{] [<U+000A>] ws ["] [t] [i] [t] [l] [e] ["] [:] ws string [,] [<U+000A>] ws ["] [a] [u] [t] [h] [o] [r] ["] [:] ws string [<U+000A>] ws [}]
ws ::= ws_12
string ::= ["] string_10 ["] ws
boolean ::= [t] [r] [u] [e] | [f] [a] [l] [s] [e]
null ::= [n] [u] [l] [l]
string_8 ::= [^"] | [] string_9
string_9 ::= ["/bfnrt] | [u] [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F]
string_10 ::= string_8 string_10 |
ws_11 ::= [ <U+0009><U+000A>] ws
ws_12 ::= ws_11 |
float ::= float_14 float_20 float_24 ws
float_14 ::= float_15 float_16
float_15 ::= [-] |
float_16 ::= [0-9] | [1-9] float_17
float_17 ::= [0-9] float_17 |
float_18 ::= [.] float_19
float_19 ::= [0-9] float_19 | [0-9]
float_20 ::= float_18 |
float_21 ::= [eE] float_22 float_23
float_22 ::= [-+] |
float_23 ::= [0-9] float_23 | [0-9]
float_24 ::= float_21 |
integer ::= integer_26
integer_26 ::= [0-9] integer_26 | [0-9]

Environment and Context

Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.

• Physical (or virtual) hardware you are using, e.g. for Linux:

$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 32
On-line CPU(s) list: 0-31
Vendor ID: AuthenticAMD
Model name: AMD Ryzen 9 7950X 16-Core Processor
CPU family: 25
Model: 97
Thread(s) per core: 2
Core(s) per socket: 16
Socket(s): 1
Stepping: 2
CPU max MHz: 5881,0000
CPU min MHz: 400,0000
BogoMIPS: 9000.24
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good a
md_lbr_v2 nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp
legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw
pstate ssbd mba perfmon_v2 ibrs ibpb stibp ibrs_enhanced vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflusho
pt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local avx512_bf16 clzero irperf xsaveerptr rdpru wbn
oinvd cppc arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif x2avic v_spec_ctrl vnmi avx5
12vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid overflow_recov succor smca fsrm flush_l1d
Virtualization features:
Virtualization: AMD-V
Caches (sum of all):
L1d: 512 KiB (16 instances)
L1i: 512 KiB (16 instances)
L2: 16 MiB (16 instances)
L3: 64 MiB (2 instances)
NUMA:
NUMA node(s): 1
NUMA node0 CPU(s): 0-31
Vulnerabilities:
Gather data sampling: Not affected
Itlb multihit: Not affected
L1tf: Not affected
Mds: Not affected
Meltdown: Not affected
Mmio stale data: Not affected
Retbleed: Not affected
Spec rstack overflow: Mitigation; safe RET, no microcode
Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Spectre v2: Mitigation; Enhanced / Automatic IBRS, IBPB conditional, STIBP always-on, RSB filling, PBRSB-eIBRS Not affected
Srbds: Not affected
Tsx async abort: Not affected

• Operating System, e.g. for Linux:

$ uname -a
Linux dr-uu 6.5.0-14-generic #14~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Nov 20 18:15:30 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

• SDK version, e.g. for Linux:

$ python3 --version
Python 3.8.18
$ make --version
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ g++ --version
g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

Failure Information (for bugs)

Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.

Steps to Reproduce

1. take a pydantic method and declare something in float
2. generate the grammar
3. remove the float
4. generate a new grammar from updated pydantic method
5. still gets the entire grammar broken into these weird one tokens.

[UYousafzai]

Not concerning the CPP binding and instead created at the original llama.cpp repo

Issue id ggml-org/llama.cpp#5925