fieldset time dimension

[dencan06]

Hello
I'm just a new user trying to read my model outputs from a netcdf file with following:
variables={'U': 'UVEL', 'V' :'VVEL', 'W':'WVEL'}
dimensions['U']={'time':'T','depth':'diag_levels','lat':'Y','lon':'Xp1'}
dimensions['V']={'time':'T','depth':'diag_levels','lat':'Yp1','lon':'X'}
dimensions['W']={'time':'T','depth':'diag_levels','lat':'Y','lon':'X'}
fset = FieldSet.from_netcdf(filenames, variables, dimensions,full_load=True)

however I'm getting this error:
field.pyc in from_netcdf(cls, filenames, variable, dimensions, indices, grid, mesh, allow_time_extrapolation, time_periodic, full_load, dimension_filename, **kwargs)
221 data[ti:ti+len(tslice), 0, :, :] = filebuffer.data[:, :, :]
222 else:
--> 223 data[ti:ti+len(tslice), :, :, :] = filebuffer.data[:, :, :, :]
224 ti += len(tslice)
225 else:

ValueError: could not broadcast input array from shape (583,50,200,176) into shape (1,50,200,176)

T is the unlimited dimension so it's a masked array. could this be causing the problem?

[erikvansebille]

I indeed expect that the problem is related to your time dimension. Could you copy the header of your netcdf file here, so that we can have a look?

By the way, is there a reason that you use full_load=True? Do you also get an error when you use full_load=False?

[dencan06]

Actually I tried first without full_load and then with, and also with specified indices but i got the same error.
"
<type 'netCDF4._netCDF4.Dataset'>
root group (NETCDF3_CLASSIC data model, file format NETCDF3):
MITgcm_version: checkpoint66g
...
MITgcm_mnc_ver: 0.9
sNx: 22
sNy: 25
OLx: 5
OLy: 5
nSx: 1
nSy: 1
nPx: 8
nPy: 8
Nx: 176
Ny: 200
Nr: 50
dimensions(sizes): T(583), Zmd000050(50), Xp1(177), Y(200), X(176), Yp1(201), Zld000050(50)
variables(dimensions): float64 diag_levels(Zmd000050), float64 Xp1(Xp1), float64 Y(Y), float64 X(X), float64 Yp1(Yp1), float64 T(T), int32 iter(T), float32 UVEL(T,Zmd000050,Y,Xp1), float32 VVEL(T,Zmd000050,Yp1,X), float32 WVEL(T,Zld000050,Y,X)
"
for T:
<type 'netCDF4._netCDF4.Variable'>
float64 T(T)
long_name: model_time
units: seconds since 2012-01-01, 00:00:00
unlimited dimensions: T
current shape = (583,)
filling on, default _FillValue of 9.96920996839e+36 used

[erikvansebille]

OK, thanks. Nothing immediately jumps out as an obvious problem.
Would you mind sending me a copy of the netcdf file, so I can try to debug? Could even be a subset if the full thing is too large. My email address is e.vansebille@uu.nl, perhaps you can use a transfer service like wetransfer or similar?

[erikvansebille]

After having explored the netcdf file you sent me, I think I have a fix. It's in a branch at https://github.com/OceanParcels/parcels/compare/fix-mitgcm-time-dimension

Could you try run with this branch, and see if that fixes the error for you?

[dencan06]

Hi, sorry for the late reply. Currently I'm out of city for a meeting, I'll look into it as soon as I get back.
Thanks!

[erikvansebille]

This branch has now become a PR (see #460) and I've also cleaned the code a bit. You will now have to explicitly add netcdf_engine='scipy' in your line

fset = FieldSet.from_netcdf(filenames, variables, dimensions, netcdf_engine='scipy')

Can you please check whether this now works for you?