-
Notifications
You must be signed in to change notification settings - Fork 406
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix forced int32 type conversion in RasterDataset #384
Conversation
This looks fine to me. I think the only thing we need to be careful of (and I've run into this in the past) is that pytorch doesn't support a uint16 data type and some geospatial imagery comes in this dtype. So the next line converting to a torch tensor would raise an error. Edit: the tests caught exactly this error. We should probably check if the numpy arrays are uint16 or uint32 and convert to int32. |
@isaaccorley was there also some kind of dtype issue with Kornia or was that only with the number of channels? |
Yeah but the kornia thing will have to happen at the datamodule level, if you're referring to the masks required to be float dtype. |
Yep, just want to make sure we update those if needed too. |
Good point. I think it should be fine. Once the updates to this PR get made, if it breaks any tests we can figure it out then. |
uint32 should rather be converted to int64 to prevent misinterpretation of high values, right? |
It looks like we don't have any existing RasterDatasets that use np.uint32. We'll have to add a fake dataset (or modify the dtype of an existing dataset) so that we can get 100% test coverage. |
Thanks for the contribution @tritolol! I just added a test case for when raster data is uint32 so we can keep 100% test coverage. I think the only pending thing is for you to sign the CLA above. |
Thank you for this useful repository. I hope that I find the time to contribute more in the future. |
I'm seeing this problem too, I'll find who to ask and @ you when its fixed |
@tritolol it was a recently introduced problem with the bot that was just rolled back -- do you mind trying again? (I'm seeing that the CLA page loads after ~10ish seconds) |
@calebrob6 now it worked! |
* fix forced int32 type conversion * add fix for numpy dtypes which are not supported by tensors * delete whitespace * Adding custom data to test the dtype transform * Fixed formatting Co-authored-by: Caleb Robinson <calebrob6@gmail.com>
* fix forced int32 type conversion * add fix for numpy dtypes which are not supported by tensors * delete whitespace * Adding custom data to test the dtype transform * Fixed formatting Co-authored-by: Caleb Robinson <calebrob6@gmail.com>
Fixes #379