-
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Crawlee does not include native support for "daemonization" or task distribution, but you can integrate it with other libraries that provide these features. For instance, to run a Crawlee project as a daemon, you can embed it within a framework like FastAPI and communicate with it using HTTP. If you need distributed task management, Celery could be an option, which can handle task scheduling, queuing, and scaling. Both FastAPI and Celery are free, open-source tools, that you can integrate with Crawlee. If you’re interested in hosting your crawlers on a managed platform, you can use Apify. Check out our Deploy to Apify guide for more details. |
Beta Was this translation helpful? Give feedback.
-
|
ok, thanks for your replay. |
Beta Was this translation helpful? Give feedback.
Crawlee does not include native support for "daemonization" or task distribution, but you can integrate it with other libraries that provide these features. For instance, to run a Crawlee project as a daemon, you can embed it within a framework like FastAPI and communicate with it using HTTP. If you need distributed task management, Celery could be an option, which can handle task scheduling, queuing, and scaling. Both FastAPI and Celery are free, open-source tools, that you can integrate with Crawlee.
If you’re interested in hosting your crawlers on a managed platform, you can use Apify. Check out our Deploy to Apify guide for more details.