Collecting all existing (intake) data catalogues

[poplarShift]

As raised over on the gitter, having one master data catalogue would be helpful in getting new users started. Nothing says "come try me out" like when your favourite dataset is already good to go. Of course, as @rabernat pointed out, we'd like the data to be supplied in cloud-ready format from the beginning, but that may take a while.

So far, there are (list to grow as I find more or you add "yours"):

• https://github.com/pangeo-data/pangeo/blob/master/gce/catalog.yaml
• https://raw.githubusercontent.com/NicWayand/polar.pangeo.io-deploy/staging/deployments/polar.pangeo.io/image/catalog.yaml'
• https://github.com/pangeo-data/pangeo_ocean_examples/blob/master/catalog.yaml

Then, should these be merged somehow? Indexed?

For reference: #39

[rabernat]

Thanks again for another good issue.

We are having a broader discussion about how to deal with data catalogs over in the https://github.com/pangeo-data/pangeo-datastore repo.

We need to have an "authoritative" catalog. The problem is that intake is rather python specific, it is great for a catalog of convenience, but it is not based on any widely accepted specification uses in geosciences (of which there are many).

In the interim, I agree we need to put all of these catalogs in one place. I suggest that place is the pangeo-datastore repo. If you would like to help with this, you could start by making a PR on that repo.

[poplarShift]

Thanks, those are good points. I wasn't sure whether pangeo-datastore was not too specifically about the long-term solution, so I opened the discussion here.

I'll see about a PR over there to start such an authoritative catalogue; in the meantime, we can keep collecting files/discussing in this thread?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it had not seen recent activity. The issue can always be reopened at a later date.