Task/worker system

[davenquinn]

Some processing for maps, etc. will need a system for effectively running long-running tasks to, e.g. set topology, import/export maps, and calculate derived parameters. Right now some of this is happening in Kubernetes cron jobs or external listeners, but this has several disadvantages:

• Processing steps are defined in Kubernetes configuration instead of in application repositories or in external watchers that have to stay running
• There is little introspection on tasks, ability to cancel stalled tasks, etc.

The first step to get to a better organization is to set up a task/worker infrastructure in the Kubernetes environment. Right now, this can be a bit of an empty shell in anticipation of building out more functionality. Architectural considerations:

• Celery is probably fine unless there is a compelling reason to use another orchestrator
• For now, can have a simple "hello world" task, or tasks to calculate rgeoms for imported maps
• Examples of how to kick off tasks from e.g., the API or Macrostrat's CLI