NAD27 to NAD83 Conversions

[dmbaker]

In previous versions of pyproj, I was able to convert points between different datums. Specifically, NAD27 to NAD83 with both geographic and Cartesian coordinates. Is there a setting that needs to be set.? Is there a specific workflow to enable the conversions again? I have installed proj-data, but this had no effect.

import pyproj
from pyproj import Transformer

print(pyproj.__version__)

>>> 3.5.0

nad27 = 4267
nad83 = 4269
tx_central_nad27 = 32039

lat = 31.99762
lon = -104.065573

transformer = Transformer.from_crs(nad27, tx_central_nad27)
x_e, y_n = transformer.transform(lat, lon)
print('X/E:', x_e, 'Y/N:', y_n)

>>> X/E: 843069.7344854714 Y/N: 867191.381947474

transformer = Transformer.from_crs(nad83, tx_central_nad27)
x_e, y_n = transformer.transform(lat, lon)
print('X/E:', x_e, 'Y/N:', y_n)

>>> X/E: inf Y/N: inf

When going from NAD27 geographic coordinates to NAD27 Cartesian coordinates, the transformation works. When the setting the geographic coordinates to NAD83, inf values are returned. This code has worked in the past versions. I suspect it is a configuration issue.

[snowman2]

Please provide a minimal reproducible example of the code that previously worked: https://stackoverflow.com/help/minimal-reproducible-example

[dmbaker]

@snowman2, I have added working code... Thanks for the review. David

[snowman2]

Have you reviewed this? https://pyproj4.github.io/pyproj/stable/gotchas.html#axis-order-changes-in-proj-6

[dmbaker]

I have and in particular the section Proj (Not a generic latitude/longitude to projection converter). The code above is using Transformer as recommended.

[snowman2]

Great - I was thinking this was an axis order issue as that commonly causes the inf responses. However, that is not the case for your scenario as your set up seems correct.

[snowman2]

I tried it out and I was able to get a valid non-inf response with:

>>> import pyproj
>>> pyproj.show_versions()
pyproj info:
    pyproj: 3.6.1
      PROJ: 9.3.0
  data dir: ~/tmp/lib/python3.11/site-packages/pyproj/proj_dir/share/proj
user_data_dir: ~.local/share/proj
PROJ DATA (recommended version): 1.15
PROJ Database: 1.2
EPSG Database: v10.094 [2023-08-08]
ESRI Database: ArcGIS Pro 3.1 [2023-19-01]
IGNF Database: 3.1.0 [2019-05-24]

System:
    python: 3.11.9 (main, Apr  6 2024, 17:59:24) [GCC 11.4.0]
executable: ~/tmp/bin/python
   machine: Linux-6.5.0-41-generic-x86_64-with-glibc2.35

Python deps:
   certifi: 2024.6.2
    Cython: None
setuptools: 65.5.0
       pip: 24.0

I also succeeded with conda and it returned a valid non-inf response as well:

>>> import pyproj
>>> pyproj.show_versions()
pyproj info:
    pyproj: 3.6.1
      PROJ: 9.4.0
  data dir: ~/mambaforge/envs/tmp/share/proj
user_data_dir: ~/.local/share/proj
PROJ DATA (recommended version): 1.17
PROJ Database: 1.3
EPSG Database: v11.004 [2024-02-24]
ESRI Database: ArcGIS Pro 3.2 [2023-11-02]
IGNF Database: 3.1.0 [2019-05-24]

System:
    python: 3.12.3 | packaged by conda-forge | (main, Apr 15 2024, 18:38:13) [GCC 12.3.0]
executable: ~/mambaforge/envs/tmp/bin/python
   machine: Linux-6.5.0-41-generic-x86_64-with-glibc2.35

Python deps:
   certifi: 2024.6.2
    Cython: None
setuptools: 69.5.1
       pip: 24.0

[dmbaker]

Adding to the test...

Create a conda environment from pyproj_test.yml:

name: pyproj_test
channels:
  - conda-forge
  - anaconda
  - defaults
dependencies:
  - pyproj
  - proj-data
  - python=3.9

> conda env create -f pyproj_test.yml -y
> activate pyproj_test

Create the test script pyproj_test.py:

import pyproj
from pyproj import Transformer

pyproj.show_versions()
print('-------')

nad27 = 4267
nad83 = 4269
tx_central_nad27 = 32039
tx_central_nad83 = 2277

lat = 31.99762
lon = -104.065573

transformer = Transformer.from_crs(nad27, tx_central_nad27)
x_e, y_n = transformer.transform(lat, lon)
print('From:', transformer.source_crs.name, '-', transformer.source_crs.type_name)
print('To:', transformer.target_crs.name, '-', transformer.target_crs.type_name)
print('X/E:', x_e, 'Y/N:', y_n)
print('-------')

transformer = Transformer.from_crs(nad27, tx_central_nad83)
print('From:', transformer.source_crs.name, '-', transformer.source_crs.type_name)
print('To:', transformer.target_crs.name, '-', transformer.target_crs.type_name)
x_e, y_n = transformer.transform(lat, lon)
print('X/E:', x_e, 'Y/N:', y_n)
print('-------')

transformer = Transformer.from_crs(nad83, tx_central_nad27)
print('From:', transformer.source_crs.name, '-', transformer.source_crs.type_name)
print('To:', transformer.target_crs.name, '-', transformer.target_crs.type_name)
x_e, y_n = transformer.transform(lat, lon)
print('X/E:', x_e, 'Y/N:', y_n)
print('-------')

transformer = Transformer.from_crs(nad83, tx_central_nad83)
print('From:', transformer.source_crs.name, '-', transformer.source_crs.type_name)
print('To:', transformer.target_crs.name, '-', transformer.target_crs.type_name)
x_e, y_n = transformer.transform(lat, lon)
print('X/E:', x_e, 'Y/N:', y_n)

Run the script:

> python pyproj_test.py

Output:

pyproj info:
    pyproj: 3.6.1
      PROJ: 9.4.0
  data dir: C:\Users\USERNAME\AppData\Local\miniconda3\envs\pyproj_test\Library\share\proj
user_data_dir: C:\Users\USERNAME\AppData\Local/proj
PROJ DATA (recommended version): 1.18
PROJ Database: 1.3
EPSG Database: v11.006 [2024-03-13]
ESRI Database: ArcGIS Pro 3.3 [2024-05-07]
IGNF Database: 3.1.0 [2019-05-24]

System:
    python: 3.9.19 | packaged by conda-forge | (main, Mar 20 2024, 12:38:46) [MSC v.1929 64 bit (AMD64)]
executable: C:\Users\USERNAME\AppData\Local\miniconda3\envs\pyproj_test\python.exe
   machine: Windows-10-10.0.19045-SP0

Python deps:
   certifi: 2024.6.2
    Cython: None
setuptools: 70.1.0
       pip: 24.0
-------
From: NAD27 - Geographic 2D CRS
To: NAD27 / Texas Central - Projected CRS
X/E: 843069.7344854714 Y/N: 867191.381947474
-------
From: NAD27 - Geographic 2D CRS
To: NAD83 / Texas Central (ftUS) - Projected CRS
X/E: inf Y/N: inf
-------
From: NAD83 - Geographic 2D CRS
To: NAD27 / Texas Central - Projected CRS
X/E: inf Y/N: inf
-------
From: NAD83 - Geographic 2D CRS
To: NAD83 / Texas Central (ftUS) - Projected CRS
X/E: 1139677.7406187858 Y/N: 10709720.277436513

Conversion between datums is still not working. Any ideas?

[snowman2]

Some suggestions:

• Python 3.10+ (DEP: Minimum supported Python version 3.10 #1357)
• Only use conda-forge channel or use strict priority (https://pyproj4.github.io/pyproj/stable/installation.html)

name: pyproj_test
channels:
  - conda-forge
dependencies:
  - pyproj
  - proj-data
  - python=3.10

[dmbaker]

@snowman2, have set strict channel priority with conda-forge at the top and then this install command with no luck:

conda create -n pyproj_test python=3.10 conda-forge::pyproj conda-forge::proj-data --strict-channel-priority

I am not sure what is going on. It is as if pyproj cannot find the gird shift files.

[jjimenezshaw]

Would PROJ_DEBUG=3 environment variable give some useful information?
And later PROJ_DATA=... to see if it does fix the problem.
(I used those variables directly in proj, I do not know what happens in pyproj).

[dmbaker]

Though via conda I ask to install the grid files, when I look at the PROJ_DATA directory I see no grid files. I tried:

projsync --all

But got:

Cannot open https://cdn.proj.org/files.geojson: schannel: CertGetCertificateChain trust error CERT_TRUST_IS_UNTRUSTED_ROOT
Cannot download files.geojson

[dmbaker]

Though not the most ideal environment for setup, what did work is putting the pyproj install in the pip dependency section of the conda environment file:

name: pyproj_test
channels:
  - conda-forge
dependencies:
  - pip
  - python=3.10
  - pip:
    - pyproj

I have tried every which way with conda, and could not get it to work. I am not sure if this is something to do with my local environment or a more general problem with a conda install on Windows.

[snowman2]

May be worth raising an issue here: https://github.com/conda-forge/pyproj-feedstock/

[dmbaker]

After additional testing, I found running

> pyproj sync --area-of-use "USA - Conterminous" --verbose

in the environment created from first yaml corrected the issue. That is, when pyproj was installed in the conda dependencies section.

Please note, though the environment created with the second conda yaml, the one with pyproj installed in the pip section of the dependencies section, produced results, they are different from those of the former environment by several ftUS going from both NAD27 to NAD83 and from NAD83 to NAD27. Subsequently, I ran the pyproj sync command in the latter environment and this corrected the difference in results except for the NAD83 to NAD27 transform. In both coordinates, they differ by no more the 0.008 and 0.002 ftUS X and Y respectively, a difference greater than the 1 in 10,000 for the NAD27 standard but close. This seems a bit more than might be expected for a numeric precision issue though.

[snowman2]

This may be a helpful reference: https://pyproj4.github.io/pyproj/stable/transformation_grids.html

[dmbaker]

I did... That put me onto the result above. @snowman2 thanks for your help.

[snowman2]

Have you looked into the TransformerGroup?

[dmbaker]

I have and saw the references to the grid files, but those files did not exist.

[snowman2]

Just wanted to point it out as you can try different transformations available with it and compare results.