Conda and GDAL 3 (changes to how TransformPoint works)

[wscullen]

Expected behaviour

Spatial queries to load data.

Actual behaviour

Spatial queries are always empty.

Steps to reproduce the behaviour

If GDAL 3 is installed, lat and longs are swapped when indexing datasets. If steps are taken to adjust preparation scripts to the output produced by GDAL 3, even if lat long are correct, no spatial queries will work because the lat and longs are swapped for internal queries in the datacube core (I suspect).

I confirmed this because downgrading to GDAL 2.4.3 with conda install gdal=2.4.3, re-indexing, and then spatial queries work fine.

I don't know what needs to be changed other than the install instructions need to be explicit and include a line about installing the right version of gdal (now that version 3 is available).

Environment information

• Which datacube --version are you using?
  Open Data Cube core, version 1.7

• What datacube deployment/enviornment are you running against?
  Using miniconda python3.7 docker image (https://hub.docker.com/r/frolvlad/alpine-miniconda3/) with the following setup in the docker file:

RUN apk update && apk upgrade && apk add bash bash-doc bash-completion

WORKDIR /code

RUN conda config --add channels conda-forge && conda config --set channel_priority strict

RUN conda create --name cubeenv python=3.6

# Add Tini. Tini operates as a process subreaper for jupyter. This prevents
# kernel crashes.
ENV TINI_VERSION v0.6.0
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /usr/bin/tini
RUN chmod +x /usr/bin/tini
ENTRYPOINT ["/usr/bin/tini", "--"]

RUN source activate cubeenv && conda install datacube && conda install jupyter matplotlib scipy boto3 ruamel.yaml shapely

About the change with GDAL 3:
OSGeo/gdal#1546

[Kirill888]

@sscullen thanks for the report, we haven't really tested GDAL 3 yet, primarily because rasterio only recently started supporting it. Although I believe @emmaai did try running with GDAL3 and contributed this 02b9167#diff-631065f019871752271f07ecc08d17cc, #799 to make it work for her case.

This issue is really two issues:

• Support for GDAL3 in prepare scripts (definitely broken, and hard to fix due to large number of almost identical scripts, which, by the way, are broken even for GDAL2 case because of the way lat-lon bounding box is computed Inaccurate computation of Lon/Lat bounds in geobox_info #537 )
• Support for GDAL3 in datacube-core itself (some people tried with some success, but we don't have CI tests that check that on the regular.

[SpacemanPaul]

Don't have a reference handy, but iirc GDAL3 finally fixed a LONG standing bug where the co-ordinate ordering specified by the CRS definition was ignored. Where it worked with GDAL2 there was a workaround in place to bypass this bug which will now break things in exactly the way you describe with GDAL3.

I had to switch to running in Docker to get around this issue.

[Kirill888]

This PR #821 also includes fixes for unit tests that were failing when running against GDAL3/PROJ6.

[Kirill888]

@sscullen as of a62214f all datacube-core tests pass when running with GDAL-3.0.2 and PROJ6.

Prepare scripts are still broken, but since these are in a separate repo, I'm going to close this issue. Any further discussion should go here:

opendatacube/datacube-dataset-config#10