feat(pipeline): implement pause and resume functionality for prompt exec control #499

eliteprox · 2025-11-10T16:06:31Z

This pull request introduces the Frame Overlay feature from pytrickle to assist with hot-reloading comfystream for prompt updates.

This change also refactors the ComfyStream pipeline and client interface to provide control over prompt execution. New control message handlers were added for pausing, resuming, and stopping prompts.

Warmup logic has also been improved to utilize the on_stream_start handler from pytrickle with initial stream parameters.

BYOC overlay and processor configuration:

Integrated OverlayConfig with a progress bar overlay into the BYOC stream processor, providing visual feedback during loading and warmup.

ComfyStream Server API Changes (app.py)

Added new control message handlers for pause_prompts, resume_prompts, and stop_prompts on the data channel, allowing runtime control over prompt execution. These handlers send structured JSON responses for success and error cases, improving reliability and client feedback.
Refactored prompt application logic in offer and related endpoints to use apply_prompts and start_streaming, ensuring prompts are properly applied and pipeline execution is started in a unified way.
Changed pipeline cleanup calls to use stop_prompts(cleanup=True) for both frame and audio collection, ensuring prompt execution is properly stopped and resources are cleaned up.

Pipeline initialization and warmup:

Added explicit pipeline initialization during server startup to ensure pipeline resources are ready before handling requests.
Refactored BYOC startup to remove redundant warmup endpoint and clarified that warmup runs automatically as part of on_stream_start, simplifying the startup flow.

Logging and configuration improvements:

Added timeout filters to both comfy.cmd.execution and comfystream loggers to suppress verbose execution logs, and set the comfy.model_detection logger to warning level for less noise. This was necessary to suppress transient errors during workflow prompt switching (affected mostly controlnet tensorrt workflows)
Improved logging messages to clarify server startup and workflow parameter expectations for BYOC.

Testing completed:

In comfystream server (app.py):
- Tested new stream with sd 15 and audio transcription workflows
- Stopped and started new stream with no issues, warmup transitions better than before
- Prompt updates
In BYOC.py
- /health shows LOADING until default prompt initializes comfyui fully, then shows IDLE
- false starts/stops during loading phase are rejected and do not affect pipeline state or cause double loading
- Cold startups with any workflow are successful, sd15, audio transcription, etc.
- Prompt updates for active streams with any workflow, even modality changes from video processing to audio processing update frame routing correctly
- Frame overlay auto-loading/auto-hide is consistent

…ecution - Updated `pyproject.toml` and `requirements.txt` to use the latest version of `pytrickle`. - Enhanced `VideoStreamTrack` to support stopping and resuming prompts. - Added new endpoints for pausing and resuming prompts in the server. - Implemented pause and resume methods in `ComfyStreamClient` and `Pipeline` classes. - Improved error handling and logging for prompt control actions.

pschroedl · 2025-11-10T21:50:49Z

server/app.py

            logger.error(f"Unexpected error in frame collection: {str(e)}")
        finally:
-            await self.pipeline.cleanup()
+            await self.pipeline.stop_prompts(cleanup=True)


Any particular reason not to default to True in the stop_prompts method?

The await self.pipeline.stop_prompts(cleanup=True) option shuts down ComfyUI entirely, which has an impact on startup time. I think for this interface pattern pause_prompts, resume_prompts, stop_prompts, the default is best to not cleanup. stop_prompts is only for stopping (and clearing) the running workflow prompts.

There is a similar method pipeline.cleanup that overlaps with the stop_prompts(cleanup=True) option, I can at least tidy this up to avoid duplicate methods

pschroedl · 2025-11-10T21:55:40Z

server/app.py

        await pipeline.set_prompts(prompts)
-        logger.info("[Offer] Set workflow prompts")
+        await pipeline.resume_prompts()
+        logger.info("[Offer] Set workflow prompts and resumed execution")


nit: not sure I like this pattern of explicitly adding context such as [Offer]. I would prefer to see the logger have that as a parameter
ala:
function = "offer"
logger.info("{} Set workflow prompts and resumed execution").format(function)

I think this would go along with a larger refactor of loggers and we should track as a separate enhancement

pschroedl · 2025-11-10T21:56:23Z

server/app.py

+                            "success": True
+                        }
+                        channel.send(json.dumps(response))
+                    elif params.get("type") == "stop_prompts":


this seems busy - maybe a case statement with function calls would be more readable?

Yeah, this block has become very long. I'll refactor

Refactored in 754929f and c9abfee

pschroedl · 2025-11-10T21:56:49Z

server/byoc.py

+    # Add pause endpoint
+    async def pause_handler(request):
+        try:
+            # Fire-and-forget: do not await pause


a little elaboration in these comments could be helpful

Removed these handlers, they are not actually useful for byoc pipelines and would break the encoder. Pause/resume is only needed as an internal function of prompt execution within the pipeline 8715fb6

pschroedl · 2025-11-10T21:57:02Z

server/frame_processor.py

            logger.info("Stopping ComfyStream client prompt execution")
            try:
-                await self.pipeline.client.cleanup()
+                await self.pipeline.stop_prompts(cleanup=True)


see previous

See response above #499 (comment)

…cated async functions for each control action (get_nodes, update_prompts, update_resolution, pause_prompts, resume_prompts, stop_prompts) - Added success response handling for prompt updates and resolution changes

…age functions - Added a dedicated `send_error_response` function to streamline error handling across various control actions. - Implemented error responses for missing parameters and exceptions in `update_prompts`, `update_resolution`, `pause_prompts`, `resume_prompts`, and `stop_prompts` handlers.

- Updated `AudioStreamTrack` to stop prompts with cleanup option. - Improved prompt cancellation logic in `cancel_collect_frames`. - Added immediate stopping of prompts in `ComfyStreamFrameProcessor` to prevent overlap during updates. - Refactored `ComfyStreamClient` to manage prompt execution state more effectively, including new methods for immediate stopping and fallback to passthrough workflow. - Enhanced `Pipeline` class with immediate stop functionality for prompt execution. - Improved error handling and logging across prompt management functions.

…agement - Updated `pyproject.toml` and `requirements.txt` to use the latest version of `pytrickle`. - Refactored prompt management in `ComfyStreamFrameProcessor` and `Pipeline` classes to improve loading overlay functionality. - Introduced new methods for applying prompts with loading messages and warmup handling. - Enhanced error handling during prompt application and streaming state transitions. - Added support for managing pipeline states with `PipelineState` and `PipelineStateManager` to streamline lifecycle management.

… execution immediately

- Updated `pyproject.toml` to use the latest version of `pytrickle`. - Modified `.devcontainer/devcontainer.json` to include additional port forwarding and bind mounts. - Refactored loading overlay management in `ComfyStreamFrameProcessor` and `Pipeline` classes for improved user experience. - Removed deprecated loading methods and streamlined loading state handling. - Enhanced error handling and logging during prompt processing and streaming state transitions.

…l methods - Removed the `_stop_event` from `ComfyStreamClient` as it was no longer needed. - Updated `pause_prompts` and `resume_prompts` methods to eliminate unnecessary async calls. - Adjusted `PipelineStateManager` to call `pause_prompts` directly without awaiting, streamlining state management.

… endpoint in byoc.py

- Updated `pyproject.toml` and `requirements.txt` to use the latest commit of `pytrickle`. - Improved logging and handling of stream start parameters in `ComfyStreamFrameProcessor`. - Refactored warmup overlay management to streamline user experience during streaming. - Added support for processing prompt payloads and managing warmup states more effectively.

… configuration for better control. - Refactored `ComfyStreamFrameProcessor` to simplify initialization and remove unused warmup overlay logic.

…rlays - Introduced a constant for default timeout in `byoc.py` to improve maintainability. - Updated `frame_processor.py` to use the new timeout constant for overlay processing.

…ual track classes. - Added centralized cleanup handling in the `offer` function for connection failures and closures. - Improved logging for pipeline cleanup operations to enhance error tracking.

eliteprox changed the title ~~feat(pipeline): implement pause and resume functionality for prompt ex…~~ feat(pipeline): implement pause and resume functionality for prompt exec control Nov 10, 2025

eliteprox mentioned this pull request Nov 10, 2025

ComfyStream Pipeline Pause/Resume/Stop #500

Closed

pschroedl reviewed Nov 10, 2025

View reviewed changes

eliteprox added 4 commits November 11, 2025 05:09

Merge branch 'main' into feat/prompt-exec-ctrl

6b0c46e

byoc: remove pause/resume handlers

8715fb6

eliteprox requested a review from pschroedl November 11, 2025 19:35

eliteprox added 12 commits November 11, 2025 22:51

feat(app): add resolution handling in offer function

ce06c48

refactor(client): update cancel_running_prompts method to stop prompt…

ea98c7f

… execution immediately

revert devcontainer changes

8d7f841

bump pytrickle

44d9b96

update pytrickle dependency to latest commit and remove unused warmup…

86c3889

… endpoint in byoc.py

byoc(stream_processor): add LoadingConfig with overlay mode enabled

66ba1a7

bump pytrickle for cv2 check

d1e73ef

eliteprox mentioned this pull request Nov 24, 2025

ComfyStream warmup loading screen #468

Closed

This was linked to issues Nov 24, 2025

ComfyStream warmup loading screen #468

Closed

ComfyStream Pipeline Pause/Resume/Stop #500

Closed

eliteprox added 3 commits November 25, 2025 19:03

refactor(byoc): add frame_count_to_disable parameter in the loading…

a2a4d64

… configuration for better control. - Refactored `ComfyStreamFrameProcessor` to simplify initialization and remove unused warmup overlay logic.

bump pytrickle

7a89091

chore(deps): pin pytrickle v0.1.5 in pyproject.toml and requirements.txt

3b6708f

eliteprox mentioned this pull request Nov 25, 2025

Feat/warmup overlay refactor entrypoint #487

Closed

eliteprox requested a review from ad-astra-video November 25, 2025 21:53

revert changes to devcontainer

77161f4

ad-astra-video approved these changes Nov 25, 2025

View reviewed changes

eliteprox added 3 commits November 25, 2025 22:41

fix requirements.txt

c68d7dd

refactor(byoc, frame_processor): standardize timeout handling for ove…

805b586

…rlays - Introduced a constant for default timeout in `byoc.py` to improve maintainability. - Updated `frame_processor.py` to use the new timeout constant for overlay processing.

fix(server.app): remove redundant pipeline cleanup calls from individ…

4b8c7ca

…ual track classes. - Added centralized cleanup handling in the `offer` function for connection failures and closures. - Improved logging for pipeline cleanup operations to enhance error tracking.

eliteprox merged commit 48fcca4 into main Nov 25, 2025
5 checks passed

eliteprox deleted the feat/prompt-exec-ctrl branch November 25, 2025 22:57

eliteprox linked an issue Dec 8, 2025 that may be closed by this pull request

Develop interface for comfystream.pipeline to start/stop workers #270

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(pipeline): implement pause and resume functionality for prompt exec control #499

feat(pipeline): implement pause and resume functionality for prompt exec control #499

Uh oh!

eliteprox commented Nov 10, 2025 •

edited

Loading

Uh oh!

pschroedl Nov 10, 2025

Uh oh!

eliteprox Nov 10, 2025 •

edited

Loading

Uh oh!

pschroedl Nov 10, 2025

Uh oh!

eliteprox Nov 11, 2025

Uh oh!

pschroedl Nov 10, 2025

Uh oh!

eliteprox Nov 10, 2025

Uh oh!

eliteprox Nov 11, 2025

Uh oh!

pschroedl Nov 10, 2025

Uh oh!

eliteprox Nov 11, 2025 •

edited

Loading

Uh oh!

pschroedl Nov 10, 2025

Uh oh!

eliteprox Nov 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

feat(pipeline): implement pause and resume functionality for prompt exec control #499

feat(pipeline): implement pause and resume functionality for prompt exec control #499

Uh oh!

Conversation

eliteprox commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eliteprox Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eliteprox Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

eliteprox commented Nov 10, 2025 •

edited

Loading

eliteprox Nov 10, 2025 •

edited

Loading

eliteprox Nov 11, 2025 •

edited

Loading