Scope of App Server and Workflow Manager

[JHancox]

An interesting discussion point arose at a recent working group meeting. Here is the simplest scenario that encapsulates the question:

In a clinical workflow, an algorithm detects some image feature, which then requires some data to be retrieved from a clinical system in order to provide the desired output/prediction.

The question this raises is:
Should the App that detects the feature query the relevant clinical system and/or should the Workflow Manager facilitate this sort of conditional routing?

Why does it matter?

1. Firstly, one way or another, the MONAI Deploy system needs to be flexible enough to support this sort of requirement
2. It is not 100% clear how such a requirement would be met in a way that is consistent with some of the current assumptions
3. Each solution to this has implications that need to be properly considered

Background
In the currently proposed architecture MONAI Deploy allows algorithm developers to deploy their AI/algos as ‘Apps’. These Apps can be composed into a DAG that can be executed by the Workflow Manager. This architecture was conceived for the following reasons (non-exhaustive):
• To map the different needs of App Developers and Workflow Creators to different APIs
• To provide App Developers with abstractions that maximise portability of their Apps
• To provide App Developers with abstractions that simplify deployment of their Apps
• To provide App Developers with tools that maximise performance of their Apps
• To limit the exposure of clinical systems to externally developed Apps
• To allow local IT to use Apps within their local Clinical Workflows
• To provide abstractions that allow local IT to easily and securely connect these workflows into local clinical systems

Solutions

1. The Workflow Manager already has the ability to make calls to external clinical systems using the Task API, so rather than the App containing the conditional logic it could be decomposed into two or more apps that form part of a DAG that includes this external system call.
2. The App SDK could include an operator that is similar to that already proposed in the Workflow Manager that can facilitate clinical system interrogation. Apps could be classified according to their inherent needs/risks (e.g. passive, active, privileged..)
3. The Workflow Manager could call the same App multiple times in a DAG with different inputs to affect the conditional functionality needed

Considerations
• We would like to maximise simplicity of the solution, which means minimising duplication of functionality between MWM/App Server and may also mean not catering for every corner case
• App developers should not have to understand or integrate with the complexities and idiosyncrasies of specific local hospital IT systems. This is something that the MONAI Deploy platform should provide and that is configured by local IT to communicate appropriately with local systems.
• Ideally the App SDK should provide a secure sandboxed environment within which each App can operate on incoming data as a black box.
• Sandboxing an App would impose some restrictions on its capabilities – App developers would need to understand this. If they can’t then this can always be deployed as a non MAP App
• MONAI Apps should be portable. The more passive and self-contained they are, the more feasible this is.
• The vast majority all of today’s Medical AI ISVs are providing cloud-based solutions, which operate very much in the ‘data in, answer out’ mode and are certainly not querying IT systems at any point. These app developers know that their deployment has to be able to scale, and to do this it needs to be really simple to consume.
• There is another breed of AI App that has been developed in-house by the academic/researcher community (probably the Clara Deploy users today), which has much more direct access to IT systems because the creators are local to the institution. These are the people who are mostly engaged with the MONAI Deploy WG today and are most likely to have engineered more complex workflows into their Apps

Clarifications
• How common is this requirement likely to be?
• What exactly is the scope of a MONAI App and does this fit with the requirements of the community we are trying to serve?
• What exactly is the scope of the Workflow Manager and does this fit with the requirements of the community we are trying to serve?
• Should we encourage more ISVs (AI Start-Ups etc) to join the MONAI Deploy meetings to make sure we capture their likely needs?
• What are the security considerations here and how can thy best be managed?

[jonanswer]

fully agree with all of the above, particularly in encouraging more input from end users of the system, this should definitely be prioritised to ensure any decisions made are relevant to the community. In my opinion, deployed apps should have no external connectivity - any data required for their inference should be provided by the Workflow Manager, potentially retrieved by an operator earlier in the DAG.