Better handle package data for conda build

[jhkennedy]

The current conda-forge build is missing a critical "package data" file. This PR explicitly calls out *.yml and *.yaml files inside src/RAiDER as package data so they are included in the archive.

NOTE: while a MANIFEST.in works for pip and setuptools everywhere outside of conda, conda-build would still not include the package data in the conda build. This is the only method that I could get to work.

---

Explicitely calling out the package data, or including a MANIFEST.in, isn't supposed to be needed because everything is included via setuptools_scm. Importantly, when building the RAiDER distribution setuptools_scm uses the git history (tags, tracked files) to define the package version and included files and produces a built src or binary distribution, which pip uses to install the package.

when building the conda package, we're not actually cloning the repo, but instead downloading a git archive with does not include the git history. For the package version, we explicitly tell setuptools_scm which version to use:
https://github.com/conda-forge/raider-feedstock/blob/main/recipe/meta.yaml#L14
But we also need to explicitly tell it what files to include, which is what this PR does.

---

Alternatively we could switch to building the conda package from distribution published to PyPI, which will include metadata describing the version package files to include, but pip installs from PyPI won't work (easily at least) as we have dependencies that are conda-forge only (e.g., ISCE3) and compiled extensions (we'd need to build binary distributions for a variety of architectures). This is not a good user experience so we shouldn't do this unless we do the work to make installing from PyPI relatively easy.

[jlmaurer]

@jhkennedy @dbekaert this seems fine to me. Will this manifest catch all package files? I didn't see a "*.yaml" or anything like so wasn't sure.

This is not a good user experience so we shouldn't do this unless we do the work to make installing from PyPI relatively easy

It is a bit annoying that it would be so much work to make a PyPI package; probably I would vote against it for the moment, seems like a lot of work to be added when we are trying to get up to a stable version.

[jhkennedy]

Will this manifest catch all package files? I didn't see a "*.yaml" or anything like so wasn't sure.

@jlmaurer I'm still working out if this actually fixes it or not, but yes, the intent is to include all of it, which is what the graft keyword is supposed to do: https://packaging.python.org/en/latest/guides/using-manifest-in/

[jhkennedy]

To check the conda package locally, I'm doing:

git clone -b conda-manifest git@github.com:jhkennedy/raider-feedstock.git
cd raider-feedstock

./build_locally.py linux_64_numpy1.21python3.10.____cpython

cd build_artifacts/linux-64
unzip raider-0.4.0-py310h9b08913_1.conda
tar --use-compress-program=unzstd -tf pkg-raider-0.4.0-py310h9b08913_1.tar.zst | sort

Note: I'm applying this change as a patch because I'm attempting to patch the current release on conda-forge as well; see conda-forge/raider-feedstock#7

This unfortunately still doesn't show a lib/python3.10/site-packages/RAiDER/cli/raider.yaml file being added to the package 😭

[jhkennedy]

Okay, after entirely too many iterations, I've got a setup that works for:

• pip installed (editable and not)
• sdist and .whl builds of the package
• conda-build tested locally.

So the raider.yaml package data file should now be included everywhere.

[jhkennedy]

The miniconda GitHub action has been having a ton of intermittent environment creation errors, causing a lot of the checks to fail. I've also:

• dropped minconda for micromamba
• bumped the reusable actions versions

which should resolve those issues. More discussion here: ASFHyP3/actions#49

[jlmaurer]

@jhkennedy that's some great work!

One of our unit tests is pinging a server that is evidently down a lot, which is why circleCI is failing. pinging @dbekaert and @bbuzz31 just to let them know. The problem test is in test_datelist and I believe calls the GMAO server. We don't need to worry about it for this PR.