Support pipeline execution in docker container

[michelvocks]

Currently, pipelines are directly executed on the host system. It would be helpful if pipelines could be executed in their own docker container which can be removed after execution (like a sandbox).

[fubarhouse]

Will this feature be obligatory, opt-out or optional?

[michelvocks]

Hello @fubarhouse 🤗
My idea is to make it optional. It will be disabled by default and can be activated manually.
Any opinions? 😄

[fubarhouse]

@michelvocks,

Thanks!

I guess, how it should be set up - it would be important to understand how it's being used currently by people. It could impact everybody using Gaia...

Obviously the best migration path would be to opt-into this functionality, but for best practice I would recommend opting out.

It's quite important to me that this is actually available rather than being forced. I personally don't mind either way but it's important that it's available in both forms. I love that you could support both - it would make Gaia once again a shining example in a crowd of highly opinionated systems.

[michelvocks]

This issue is now blocked until #46 is implemented because of the following reasons:

• After working on several prototypes and POCs I have found out that implementing this issue without [Feature Request] Distributed execution #46 makes no sense.
• The whole pipeline build process and pipeline execution is tightly coupled to local filesystem actions (which makes sense in my opinion).
• Building and running pipelines inside a Docker container is a remote action.
• As soon as [Feature Request] Distributed execution #46 has been implemented, we can simply start an agent in a Docker container, build the pipeline, execute the pipeline and finally kill the container.
• This leaves the build process clean and one-purpose driven.

Sorry for delaying this feature once again but in the end, I think this is the best decision for Gaia and it's architecture.

[Skarlso]

Doonnee by #201 🎉