3.0 component upgrades must use specified versions

[sashakames]

Apparently there are versions in esg_init.py that aren't considered in an upgrade. Also they may not be considered in installs from scratch but really should.

apache_frontend_version, apache_frontend_tag (needed for getting the esgf-httpd.conf template), but open to scrapping for 3.0

cdat_version # conda
esgprep_version, # pip
cmor_version, # conda
esgf_dashboard_version, # dashboard_ip tarball or could be git repo
esgf_stats_api_version, # java webapp - on mirror

The following are no longer deployed by the installer: esgf_desktop_version, openssl*, ant_version

[nathanlcarlson]

Regarding cdat, what exactly should be installed relating to cdat? This has been a very gray matter to me and it could use some specific definition.

[sashakames]

CDAT has a version check using "cdat_info", and upgrades have benefitted from uninstalling and reinstalling the following modules as follows:

conda remove -y cdms2 cdtime cdutil cdat_info libnetcdf
conda install -y -c conda-forge cdms2 cdtime libnetcdf cdat_info

So far it hasn't been necessary to pin the cdat modules to specific versions, with the version number used for detection against the currently installed version, in order to suggest a needed upgrade

[nathanlcarlson]

Okay, but in the case of first time install what is the conda install command exactly for cdat module(s)? Or conda create as they may be included at creation time.

[sashakames]

first time it installs the modules within the "conda create": conda create -y -n esgf-pub -c conda-forge cdms2 cdtime cdat_info python=2.7
libnetcdf is left out because its a dependency. There have been issues with it not properly upgraded if not uninstalled and reinstalled, hence the need for that procedure in a upgrade. We are working around wonky/glitchy limitations of Conda.

[nathanlcarlson]

Okay I asked because those modules are currently not being specifically installed. Right now it looks like:

conda create -y -n esgf-pub "python<3" cdutil cmor "hdf5==1.10.3" -c pcmdi/label/nightly -c conda-forge

I am not sure if those modules are a subset of the dependencies here or what, but I would like to have a clear definition that I can safely follow. @sashakames Should it look like what I put above, or like what you put above

first time it installs the modules within the "conda create": conda create -y -n esgf-pub -c conda-forge cdms2 cdtime cdat_info python=2.7

[sashakames]

That looks somewhat outdated, due to divergence between the 2.x and 3.0 ongoing work. I assume this create step is in the bootstrap. cmor should be pinned to a version - we've had issues of new versions breaking.

[nathanlcarlson]

Okay, so not the nightly for cmor as previously discussed?

That looks somewhat outdated, due to divergence between the 2.x and 3.0 ongoing work.

Okay, is there a file maintained somewhere that can easily be referenced for what specifically should be installed?

[sashakames]

Nightly is a workaround. Whether we keep that depends on if it works vs just using the version in condo-forge. The code in "esg-node" is the reference for packages used in 2.x for conda/cdat.

[nathanlcarlson]

Looking more closely at cdat_version in the master branch I see that it actually is not used to specify the version of any module or package, it is only used to check the version installed. It appears that the latest versions of cdat related packages are being installed no matter what, which makes that check sort of artificial. Unless there is code some where external from the repository that uses it.