Deprecate sources in favour of origins

[Marigold]

Problem

Maintaining both sources and origins is cumbersome. Migrating all sources to origins would allow us to deprecate a significant amount of code in both ETL and Grapher, potentially speeding up the process. We've already migrated the most critical datasets to origins and are currently migrating the rest as needed. However, a one-time full migration might be more efficient.

Solution

Full migration to origins in snapshots

This would involve migrating all sources in snapshots to origins. We’d need to devise an automatic source-to-origin transformation and modify all steps to use Table instead of DataFrame to ensure proper origin propagation.

Adding origins to YAML files

For all garden and grapher datasets, we could extract sources, automatically migrate them to origins, and add them to the metadata YAML files (potentially as *.meta.override.yml).

TODO

• How many (unarchived) datasets still use sources?

[larsyencken]

It would be good to understand the size of the migration needed to work out if it's worthwhile now.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.