Convert all non-RGB images to RGB in CLIPImageTransform

[vancoykendall]

In the CLIPImageTransfrom, only RGBA images get converted to RGB. I don't see a reason to not convert all non-RGB images. I have a multimodal dataset with mostly RGB images, but a handful are in CMYK and L. I'm finetuning a Llama3.2 VLM which uses the CLIPImageTransform and my first training crashed in the middle because a CMYK image finally was loaded, didn't get converted, and the size didn't match.

torchtune/torchtune/models/clip/_transform.py

Lines 161 to 163 in 96dea61

	# Make image torch.tensor((3, H, W), dtype=dtype), 0<=values<=1
	if hasattr(image, "mode") and image.mode == "RGBA":
	image = image.convert("RGB")

I noticed the code in the eval recipe actually does convert all non-RGB images to RGB:

torchtune/recipes/eleuther_eval.py

Lines 168 to 170 in 96dea61

	for image in images:
	if image.mode != "RGB":
	image = image.convert("RGB")

[ebsmothers]

Thanks for creating the issue. Actually this was done as a bit of a one-off so we probably can consider changing to match the eval recipe. cc @pbontrager in case you have any strong objections to this, but otherwise @vancoykendall we'd love to have you open a PR with the change if you're interested.

[vancoykendall]

@ebsmothers sounds good. Also, I think like the eval code I can remove the check for the mode attribute as all PIL.Images have the attribute

[vancoykendall]

@ebsmothers Just created a pr #1976

[ebsmothers]

Closing now that the fix has landed. Thanks again!