Returning control to vicuna causes garbage results

[CRD716]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

Pressing enter should make the model predict an instruction if nothing is entered, as it used to do a handful of commits ago.

Current Behavior

It begins acting randomly, as if it did not have any previous instruction. However, putting in Continue seems to let it have context just fine.

Environment and Context

Model: https://huggingface.co/eachadea/ggml-vicuna-13b-1.1/blob/main/ggml-vicuna-13b-1.1-q4_0.bin

• Physical (or virtual) hardware you are using:

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         39 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  4
  On-line CPU(s) list:   0-3
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
    CPU family:          6
    Model:               78
    Thread(s) per core:  2
    Core(s) per socket:  2
    Socket(s):           1
    Stepping:            3
    CPU(s) scaling MHz:  56%
    CPU max MHz:         3100.0000
    CPU min MHz:         400.0000
    BogoMIPS:            5199.98
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs 
                         bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadli
                         ne_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bm
                         i1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d a
                         rch_capabilities
Virtualization features: 
  Virtualization:        VT-x
Caches (sum of all):     
  L1d:                   64 KiB (2 instances)
  L1i:                   64 KiB (2 instances)
  L2:                    512 KiB (2 instances)
  L3:                    4 MiB (1 instance)
NUMA:                    
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-3
Vulnerabilities:         
  Itlb multihit:         KVM: Mitigation: VMX disabled
  L1tf:                  Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
  Mds:                   Mitigation; Clear CPU buffers; SMT vulnerable
  Meltdown:              Mitigation; PTI
  Mmio stale data:       Mitigation; Clear CPU buffers; SMT vulnerable
  Retbleed:              Mitigation; IBRS
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; IBRS, IBPB conditional, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
  Srbds:                 Mitigation; Microcode
  Tsx async abort:       Not affected

• Operating System:

Linux debianSpectre 6.1.0-7-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.20-2 (2023-04-08) x86_64 GNU/Linux

• SDK version:

Python 3.11.2
GNU Make 4.3
Built for x86_64-pc-linux-gnu
g++ (Debian 12.2.0-14) 12.2.0

Failure Information (for bugs)

> Explain some ways that humanity could solve pollution
1. Reduce, Reuse and Recycle is a well-known concept to reduce the amount of waste generated by humans. This can be achieved through segregation of waste at source, encouraging people not to litter, promoting composting etc.
> Continue
2. Encouragement of sustainable and eco-friendly practices in industries such as manufacturing, agriculture, construction can significantly reduce the amount of pollution generated by these sectors. This includes using renewable energy sources, implementing waste management techniques, and promoting environmentally responsible production methods.
3. Investment in public transportation systems to reduce reliance on personal vehicles is another way to decrease air pollution. Encouraging carpooling or use of electric/hybrid cars can also help mitigate the issue.
4. Implementation of strict regulations and laws regarding emissions from industrial plants, power generation facilities, transportation systems etc. to ensure compliance with environmental standards can significantly reduce air pollution levels in a region.
> 
 # Lijst van rijksmonumenten in Tilburg (plaats)

De stad Tilburg telt 194 rijksmonumenten, hieronder een overzicht van de meeste belangrijke en bekende monumenten:

Mediabestanden

Zie de categorie Rijksmonuments in Tilburg van Wikimedia Commons voor mediabestanden over dit onderwerp.^C

Steps to Reproduce

1. build llama.cpp from master branch
2. download ggml-vicuna-13b-1.1-q4_0.bin from https://huggingface.co/eachadea/ggml-vicuna-13b-1.1/blob/main/ggml-vicuna-13b-1.1-q4_0.bin
3. run main with -m models/ggml-vicuna-13b-1.1-q4_0.bin -n -1 -t 4 -c 2048 --temp 0.5 --repeat_penalty 1.24 --interactive-first --instruct --color
4. give an instruction
5. wait for it to end its response
6. tell it to continue
7. wait for it to end its response
8. press enter when prompted for input

Failure Logs

(Warning: Github will not format this, so there's an empty > line before the random text, and a # before Lisjst)

Complete Log

$ ./main -m models/ggml-vicuna-13b-1.1-q4_0.bin -n -1 -t 4 -c 2048 --temp 0.5 --repeat_penalty 1.24 --interactive-first --instruct --color main: seed = 1682101828 llama.cpp: loading model from models/ggml-vicuna-13b-1.1-q4_0.bin llama_model_load_internal: format = ggjt v1 (latest) llama_model_load_internal: n_vocab = 32000 llama_model_load_internal: n_ctx = 2048 llama_model_load_internal: n_embd = 5120 llama_model_load_internal: n_mult = 256 llama_model_load_internal: n_head = 40 llama_model_load_internal: n_layer = 40 llama_model_load_internal: n_rot = 128 llama_model_load_internal: ftype = 4 (mostly Q4_1, some F16) llama_model_load_internal: n_ff = 13824 llama_model_load_internal: n_parts = 1 llama_model_load_internal: model size = 13B llama_model_load_internal: ggml ctx size = 73.73 KB llama_model_load_internal: mem required = 9807.47 MB (+ 1608.00 MB per state) llama_init_from_file: kv self size = 1600.00 MB

system_info: n_threads = 4 / 4 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
main: interactive mode on.
Reverse prompt: '### Instruction:

'
sampling: temp = 0.500000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.240000
generate: n_ctx = 2048, n_batch = 8, n_predict = -1, n_keep = 2

== Running in interactive mode. ==

• Press Ctrl+C to interject at any time.
• Press Return to return control to LLaMa.
• If you want to submit another line, end your input in ''.

Explain some ways that humanity could solve pollution

1. Reduce, Reuse and Recycle is a well-known concept to reduce the amount of waste generated by humans. This can be achieved through segregation of waste at source, encouraging people not to litter, promoting composting etc.

Continue

2. Encouragement of sustainable and eco-friendly practices in industries such as manufacturing, agriculture, construction can significantly reduce the amount of pollution generated by these sectors. This includes using renewable energy sources, implementing waste management techniques, and promoting environmentally responsible production methods.
3. Investment in public transportation systems to reduce reliance on personal vehicles is another way to decrease air pollution. Encouraging carpooling or use of electric/hybrid cars can also help mitigate the issue.
4. Implementation of strict regulations and laws regarding emissions from industrial plants, power generation facilities, transportation systems etc. to ensure compliance with environmental standards can significantly reduce air pollution levels in a region.

Lijst van rijksmonumenten in Tilburg (plaats)

De stad Tilburg telt 194 rijksmonumenten, hieronder een overzicht van de meeste belangrijke en bekende monumenten:

Mediabestanden

Zie de categorie Rijksmonuments in Tilburg van Wikimedia Commons voor mediabestanden over dit onderwerp.^C

I have a suspicion this has something to do with #1104 or a commit around there.

[CRD716]

Needed to be run without --instruct, then have Human: prepended to the first message and reverse prompt set to Human: as well.