Inventory CKAN 2.8+ upgrade

[adborden]

As a Data.gov developer, I want to rebuild Inventory from a clean CKAN 2.8+ so that I can take advantage of the latest security updates and bug fixes without worrying about the tech debt from Data.gov customizations.

Wiki Documentation

Must

• [research: 1d] Identify CKAN commits that are still necessary once on 2.8+
  • Pay particular attention to schema changes!!
• [test: 1d] Test integration of required CKAN extensions
• [insert issues identified above here, making room in sprint backlog if necessary]
• Inventory-app deployed in the AWS sandbox (Deploy CKAN 2.8-based Inventory in AWS sandbox #1481)
• [research: 1d] Test the inventory upgrade/migration with production data ([research 1d] Test the inventory upgrade/migration with production data #1675)
• [insert issues identified above here]

Should:

• Identify extension commits that are still necessary once on newer versions
• Inventory-app has unit tests or integration tests for extensions

[adborden]

Notes from our discussion on Catalog and Inventory.

[adborden]

After chatting with @pjsharpe07 , I've added a column to the CKAN changes to be clear that we're not porting any custom changes to a new CKAN 2.8 fork and that it looks like a CKAN fork won't be necessary for 2.8.

[adborden]

@jbrown-xentity I'm not seeing the saml2 extensions in the inventory_ckan_2.8 branch anymore but no issue to add them back. Were they removed for a reason, and are there any other extensions that were removed that we might need to add back?

[adborden]

Spoke too soon, looks like we may need a CKAN patch for datapusher as per @avdata99's datapusher PR

[jbrown-xentity]

Yes, this branch started by removing all extensions and adding them back in one at a time. The ckanext saml hasn't been added back in as there is no way to attempt local testing at this time (to my knowledge). I believe the CKAN multi team was working on that though, so maybe soon? Otherwise we'll need to test on staging.

[adborden]

Just to recap the conversation I had with @jbrown-xentity:

We'll want to capture the work of adding extensions back in an issue. The CKAN multi team is working on the saml2 forks, but that probably is not a blocker for adding the extensions back unless there is a specific issue with CKAN 2.8 (and repoze.who might already be a blocker). Right, we're not planning on doing any local testing with a saml2 IdP, but since requirements-freeze.txt is still the artifact we use for deploying the application, so it still must include the saml2 extensions. For local development, our only test is "does it install, and can we run CKAN" which is what the existing integration tests are validating.

Once we have the 2.8 branch working in the AWS sandbox, we should move ahead to testing on staging with MAX.gov as the SAML IdP, and the missing extensions are a blocker, so we want to make sure we're tracking that on our board.

[mogul]

Everything that remained in the Icebox is confirmed still an issue after the 2.8 rollout. So I've unparented them from this Epic; they remain in the Icebox, but this Epic is Done.