Error running vak prep

[athenasyarifa]

Hi Dave,

I am trying vak prep on my dataset and encountered an error when it tried to read the raven annotation files. The following is the error message:

2024-02-15 13:49:06,827 - vak.prep.frame_classification.frame_classification - INFO - vak version: 1.0.0a3
2024-02-15 13:49:06,827 - vak.prep.frame_classification.frame_classification - INFO - Will prepare dataset as directory: /mnt/c/Users/Lenovo/Documents/GitHub/willowtit-project/bioacoustic/vak_train/wav-vak-frame-classification-dataset-generated-240215_134906
2024-02-15 13:49:09,362 - vak.prep.spectrogram_dataset.prep - INFO - making array files containing spectrograms from audio files in: /mnt/c/Users/Lenovo/Documents/GitHub/willowtit-project/bioacoustic/vak_train/wav
2024-02-15 13:49:09,830 - vak.prep.spectrogram_dataset.audio_helper - INFO - creating array files with spectrograms
[########################################] | 100% Completed | 53.48 s
2024-02-15 13:50:07,598 - vak.prep.spectrogram_dataset.prep - INFO - creating dataset from spectrogram files in: /mnt/c/Users/Lenovo/Documents/GitHub/willowtit-project/bioacoustic/vak_train/wav-vak-frame-classification-dataset-generated-240215_134906/spectrograms_generated_240215_134906
2024-02-15 13:50:07,636 - vak.common.files.spect - INFO - validating set of spectrogram files
[########################################] | 100% Completed | 24.65 s
2024-02-15 13:50:35,896 - vak.prep.spectrogram_dataset.spect_helper - INFO - creating pandas.DataFrame representing dataset from spectrogram files
[########################################] | 100% Completed | 22.04 s
2024-02-15 13:51:02,126 - vak.prep.frame_classification.frame_classification - INFO - Will split dataset.
Traceback (most recent call last):
  File "/home/rifsyy/anaconda3/envs/vak_env/bin/vak", line 8, in <module>
    sys.exit(main())
  File "/home/rifsyy/anaconda3/envs/vak_env/lib/python3.10/site-packages/vak/__main__.py", line 48, in main
    cli.cli(command=args.command, config_file=args.configfile)
  File "/home/rifsyy/anaconda3/envs/vak_env/lib/python3.10/site-packages/vak/cli/cli.py", line 54, in cli
    COMMAND_FUNCTION_MAP[command](toml_path=config_file)
  File "/home/rifsyy/anaconda3/envs/vak_env/lib/python3.10/site-packages/vak/cli/cli.py", line 28, in prep
    prep(toml_path=toml_path)
  File "/home/rifsyy/anaconda3/envs/vak_env/lib/python3.10/site-packages/vak/cli/prep.py", line 122, in prep
    dataset_df, dataset_path = prep_module.prep(
  File "/home/rifsyy/anaconda3/envs/vak_env/lib/python3.10/site-packages/vak/prep/prep_.py", line 194, in prep
    dataset_df, dataset_path = prep_frame_classification_dataset(
  File "/home/rifsyy/anaconda3/envs/vak_env/lib/python3.10/site-packages/vak/prep/frame_classification/frame_classification.py", line 333, in prep_frame_classification_dataset
    dataset_df = split.frame_classification_dataframe(
  File "/home/rifsyy/anaconda3/envs/vak_env/lib/python3.10/site-packages/vak/prep/split/split.py", line 152, in frame_classification_dataframe
    labels = labels_from_df(dataset_df, dataset_path)
  File "/home/rifsyy/anaconda3/envs/vak_env/lib/python3.10/site-packages/vak/common/labels.py", line 111, in from_df
    return [annot.seq.labels for annot in annots]
  File "/home/rifsyy/anaconda3/envs/vak_env/lib/python3.10/site-packages/vak/common/labels.py", line 111, in <listcomp>
    return [annot.seq.labels for annot in annots]
AttributeError: 'Annotation' object has no attribute 'seq'

And my train.toml looks like this:

# PREP: options for preparing dataset
[PREP]
# dataset_type: corresponds to the model family such as "frame classification" or "parametric umap"
dataset_type = "frame classification"
# input_type: input to model, either audio ("audio") or spectrogram ("spect")
input_type = "spect"
# data_dir: directory with data to use when preparing dataset
data_dir = "/mnt/c/Users/Lenovo/Documents/GitHub/willowtit-project/bioacoustic/vak_train/wav"
# output_dir: directory where dataset will be created (as a sub-directory within output_dir)
output_dir = "/mnt/c/Users/Lenovo/Documents/GitHub/willowtit-project/bioacoustic/vak_train"
# audio_format: format of audio, either wav or cbin
audio_format = "wav"
# annot_format: format of annotations
annot_format = "raven"
# labelset: string or array with unique set of labels used in annotations
labelset = ""
# train_dur: duration of training split in dataset, in seconds
train_dur = 50
# val_dur: duration of validation split in dataset, in seconds
val_dur = 15
# test_dur: duration of test split in dataset, in seconds
test_dur = 30

# SPECT_PARAMS: parameters for computing spectrograms
[SPECT_PARAMS]
# fft_size: size of window used for Fast Fourier Transform, in number of samples
fft_size = 512
# step_size: size of step to take when computing spectra with FFT for spectrogram
# also known as hop size
step_size = 64

# TRAIN: options for training model
[TRAIN]
# model: the string name of the model. must be a name within `vak.models` or added e.g. with `vak.model.decorators.model`
model = "TweetyNet"
# root_results_dir: directory where results should be saved, as a sub-directory within `root_results_dir`
root_results_dir = "/mnt/c/Users/Lenovo/Documents/GitHub/willowtit-project/bioacoustic/vak_train"
# batch_size: number of samples from dataset per batch fed into network
batch_size = 8
# num_epochs: number of training epochs, where an epoch is one iteration through all samples in training split
num_epochs = 2
# normalize_spectrograms: if true, normalize spectrograms per frequency bin, so mean of each is 0.0 and std is 1.0
# across the entire training split
normalize_spectrograms = true
# val_step: step number on which to compute metrics with validation set, every time step % val_step == 0
# (a step is one batch fed through the network)
# saves a checkpoint if the monitored evaluation metric improves (which is model specific)
val_step = 400
# ckpt_step: step number on which to save a checkpoint (as a backup, regardless of validation metrics)
ckpt_step = 200
# patience: number of validation steps to wait before stopping training early
# if the monitored evaluation metrics does not improve after `patience` validation steps,
# then we stop training
patience = 4
# num_workers: number of workers to use when loading data with multiprocessing
num_workers = 4
# device: name of device to run model on, one of "cuda", "cpu"
device = "cuda"
# dataset_path : path to dataset created by prep. This will be added when you run `vak prep`, you don't have to add it

# train_dataset_params: parameters used when loading training dataset
# for a frame classification model, we use a WindowDataset with a specific `window_size`
[TRAIN.train_dataset_params]
window_size = 176

# val_transform_params: parameters used when transforming validation data
# for a frame classification model, we use FrameDataset with the eval_item_transform,
# that reshapes batches into consecutive adjacent windows with a specific `window_size`
[TRAIN.val_transform_params]
window_size = 176

# TweetyNet.optimizer: we specify options for the model's optimizer in this table
[TweetyNet.optimizer]
# lr: the learning rate
lr = 0.001

# TweetyNet.network: we specify options for the model's network in this table
[TweetyNet.network]
# hidden_size: the number of elements in the hidden state in the recurrent layer of the network
hidden_size = 256

Please let me know how I can fix this.

On an unrelated note, I would love to have access to vocalpy forum to ask further questions. I registered an account, but have not yet been approved by the administrators. Can you also help me in any way regarding this?

Many thanks in advance!

Best,
Rifa

[NickleDave]

Hi @athenasyarifa, glad to hear you're trying out vak!

I am trying vak prep on my dataset and encountered an error when it tried to read the raven annotation files.

What's going on here is that Raven annotations are boxes, but the TweetyNet model in vak expects to work on line segments. We don't have any models built into vak right now that train/predict bounding boxes, but you can convert the Raven annotations to segments by throwing away the frequency ranges.

Please see this issue on the TweetyNet repo from @sfcooke96 for more detail:
yardencsGitHub/tweetynet#223 (comment)
(edit: and this comment too: yardencsGitHub/tweetynet#223 (comment))

Here's an example script for converting Raven annotations to a format that you can use to train TweetyNet models:
vocalpy/crowsetta#261 (comment)

Please let me know if that helps!

It seems like we might want to add more to the docs and code for people working with Raven-style annotations, since you're not the first person to run into this issue. Probably the docs should better explain the geometry of annotations, both in crowsetta and in a vak page on TweetyNet (that doesn't exist currently 🙈 😇 ).
Maybe we should also consider adding functionality to convert bounding box annotations to sequences, like @sammlapp suggests here:
vocalpy/crowsetta#259

I would love to have access to vocalpy forum to ask further questions. I registered an account, but have not yet been approved by the administrators.

My fault! I approved the request. Definitely please ask questions and to introduce yourself there

[athenasyarifa]

Hi @NickleDave
Many thanks for your prompt response! That makes sense now, I will try to convert the Raven annotations as per your instructions.

The docs are really helpful and friendly for beginners like me, such that I understood that crowsetta can read box and seq-like annotation formats. I think it was just that I did not know that TweetyNet works only with seq-like annotations.

Thanks for your help, and looking forward to participate in the forum!

[NickleDave]

That makes sense now, I will try to convert the Raven annotations as per your instructions.

Excellent, I'll go ahead and close this

The docs are really helpful and friendly for beginners like me,

Very glad to hear it

I think it was just that I did not know that TweetyNet works only with seq-like annotations.

Good to know, we should definitely make this clearer

Thanks for your help, and looking forward to participate in the forum!

Happy to have you join, please feel free to introduce yourself! Couldn't help noticing you are working with related tools from several other projects I keep an eye on 😅 so would love to hear more about what you're working on! I'm happy to jump on a Zoom meeting if you need tech support too, just let me know!