how to finetune with 'system information'

[mhyeonsoo]

Hello,

I am training with my custom dataset, and have a question there.
What I wanted to make is assistance that can recommend me a proper mode of device depending on my conversation.

Before inserting q/a pairs, I want to let model know about the general information of 'how to use' the device.
I tried to insert like below.

SYSTEM:
    There are 4 options in the mode
    - mode1
    - mode2
    - mode3
    - mode4
    
   you need to generate 'json' format using USER input with the proper mode.
   Desired output format is below.
   {
        'mode': [selection of mode]
        'comments': [your response]
    }


USER: example1
ASSISTANCE: response1


USER: example2
ASSISTANCE: response2


USER: example3
ASSISTANCE: response3

But it seems like the model doesn't know about the initial information about the device.

Is there any specific format like 'USER' and 'ASSITANCE' for teaching the information as well?

Thanks,

[lxe]

Try more samples (like over 300) and more epochs. Set the learning rate higher to slightly overfit it.