AttributeError: 'deeptrack.deeplay' has no attribute 'Model' and RuntimeError with channel mismatch

[LiYuan-SJTU]

I downloaded the 03. distinguishing_particles_in_brightfield_tutorial.ipynb from the tutorial and ran it locally. However, I encountered the following error when executing the code:

model = dl.Model(
    net,
    train_data=data_pipeline,
    val_data=data_pipeline,
    loss=dl.torch.nn.CrossEntropyLoss(),
    optimizer=dl.Adam(lr=1e-3),
    metrics=[tm.F1Score(task="multiclass", num_classes=3)],
)

AttributeError: module 'deeptrack.deeplay' has no attribute 'Model'

To resolve this, I modified the code to use dl.Regressor instead of dl.Model:

model = dl.Regressor(
    net,
    loss=dl.torch.nn.CrossEntropyLoss(),
    optimizer=dl.Adam(lr=1e-3),
    metrics=[tm.F1Score(task="multiclass", num_classes=3)],
)

This works, but then I encounter another error when executing the following code:

input_image, target_image = data_pipeline.batch(4)

predicted_image = model.predict(input_image.astype(np.float32)).softmax(1)

RuntimeError: Given groups=1, weight of size [32, 1, 3, 3], expected input[4, 128, 128, 1] to have 1 channels, but got 128 channels instead

My pytorch version is 2.2.1 and python version is 3.11.5.

1. Is my modification to use dl.Regressor instead of dl.Model correct? If yes, how can I handle the RuntimeError related to channel mismatch in the following code?
2. In the deeplay tutorial, it's mentioned that the .fit() method handles training, validation, and logging, and also selects the best device (GPU if available). However, it seems to always train on the CPU. How can I force the training loop to run on the GPU?

Thanks in advance for your help!

[giovannivolpe]

It should work if you make the following additional changes:

import deeptrack.deeplay as dl
import torchmetrics as tm
import torch

unet_reg = dl.Regressor(
    model=net,
    loss=torch.nn.CrossEntropyLoss(),
    optimizer=dl.Adam(lr=1e-3),
    metrics=[tm.F1Score(task="multiclass", num_classes=3)],
).create()

data_pipeline = (((image_of_particles >> dt.MoveAxis(-1, 0)) & (image_of_particles >> get_target_image)) 
                  >> dt.pytorch.ToTensor(dtype=torch.float))  # This prepares the pipeline for PyTorch.

dataset = dt.pytorch.Dataset(data_pipeline, length=640, replace=0.01)
train_loader = torch.utils.data.DataLoader(dataset, batch_size=8, shuffle=True)
unet_trainer = dl.Trainer(max_epochs=200, accelerator="auto")  # auto here should select automatically CPU/GPU.
unet_trainer.fit(unet_reg, train_loader)

and for the image prediction:

predicted_image = unet_reg.predict(input_image).softmax(1)
...
    plt.imshow(input_image[i, 0, ...], cmap="gray")
...

Please, check it and let us know if you need more help.

Note: We're in the process to adapt all examples to DeepTarck2 2.0.0 to accommodate the shift to torch and deeplay from TensorFlow and Keras.
Some already updated notebooks that might be relevant for you are here:
https://github.com/DeepTrackAI/DeepLearningCrashCourse/blob/main/Ch05_UNet/ec05_1_unet/unet.ipynb
https://github.com/DeepTrackAI/DeepLearningCrashCourse/blob/main/Ch05_UNet/ec05_A_qdots_localization/qdots_localization.ipynb
https://github.com/DeepTrackAI/DeepLearningCrashCourse/blob/main/Ch05_UNet/ec05_B_cell_counting/cell_counting.ipynb

[giovannivolpe]

@LiYuan-SJTU We have now corrected the notebook and it's working. See: https://github.com/DeepTrackAI/DeepTrack2/blob/develop/examples/tutorials/03.%20distinguishing_particles_in_brightfield_tutorial.ipynb

[LiYuan-SJTU]

Thank you for providing the updated version of 03. distinguishing_particles_in_brightfield_tutorial.ipynb. I’ve tried running the new version, but I encountered another issue.

When executing the following line:

unet_trainer.fit(unet_reg, train_loader)

I got the error: RuntimeError: expected scalar type Long but found Float .
The error seems to originate from:
return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing) located in lighting\pytorch\trainer\trainer.py.
Could you help clarify how to resolve this issue? Is it related to the data type of the labels or inputs in the train_loader? Or is it a compatibility issue with the Lightning package? My lightning package version is 2.5.0.post0.

Thanks again for your assistance!