Training the neural network

Once you have your training set ready, training the neural network is pretty straightforward! I'll assume you're starting with a file called training_data.

In case your data is sorted, let's randomise the order:

shuf training_data > temp_training
mv temp_training training_data

Partition off a bit of your data to use a validation set. Here I use 5% (1/20th) as validation, but you can adjust that as you like:

total_count=$(wc -l < training_data)
validation_count=$((total_count / 20))
training_count=$((total_count - validation_count))
head -n "$validation_count" training_data > validation_set
tail -n "$training_count" training_data > training_set

Then train the model:

deepbinner train --train training_set --val validation_set --model_out model --epochs 1000