Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not crash stream connectors when Live Streams are not started #3175

Open
gileri opened this issue Dec 20, 2024 · 0 comments
Open

Do not crash stream connectors when Live Streams are not started #3175

gileri opened this issue Dec 20, 2024 · 0 comments
Labels
feature use for describing a new feature to develop needs triage use to identify issue needing triage from Filigran Product team

Comments

@gileri
Copy link
Member

gileri commented Dec 20, 2024

Use case

Currently stream connectors crash at startup when the associated stream is stopped :

{"timestamp": "2024-12-20T09:21:31.489309Z", "level": "ERROR", "name": "Microsoft Sentinel", "message": "Error in ListenStream loop, exit.", "exc_info": "Traceback (most recent call last):\n  File \"/usr/local/lib/python3.11/site-packages/pycti/connector/opencti_connector_helper.py\", line 627, in run\n    messages = SSEClient(\n               ^^^^^^^^^^\n  File \"/usr/local/lib/python3.11/site-packages/filigran_sseclient/sseclient.py\", line 63, in __init__\n    self._connect()\n  File \"/usr/local/lib/python3.11/site-packages/filigran_sseclient/sseclient.py\", line 82, in _connect\n    self.resp.raise_for_status()\n  File \"/usr/local/lib/python3.11/site-packages/requests/models.py\", line 1024, in raise_for_status\n    raise HTTPError(http_error_msg, response=self)\nrequests.exceptions.HTTPError: 410 Client Error: This live stream is stopped for url: http://127.0.0.1:80/stream/675e6270-9e92-4189-8515-7972916a7e56?recover=2024-12-20T09:21:31Z", "attributes": {"reason": "410 Client Error: This live stream is stopped for url: http://127.0.0.1:80/stream/675e6270-9e92-4189-8515-7972916a7e56?recover=2024-12-20T09:21:31Z"}}
Terminated

Current Workaround

The connector crashes, and then gets restarted by the orchestration (docker/podman, kubernetes etc.)

Proposed Solution

Add an infinite retry loop that waits for the stream to be started, with an info/debug log explaining the situation.

Additional Information

Would you be willing to submit a PR?

As this span between pytcti, and multiple connectors, I thinks it's best to let someone more knowledgeable tackle this, if it's something we want to do.

We strongly encourage you to submit a PR if you want and whenever you want. If your issue concern a "Community-support" connector, your PR will probably be accepted after some review. If the connector is "Partner-support" or "Filigran-support", a dev team make take over but will base its work on your PR, speeding the process. It will be much appreciated.

@gileri gileri added feature use for describing a new feature to develop needs triage use to identify issue needing triage from Filigran Product team labels Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature use for describing a new feature to develop needs triage use to identify issue needing triage from Filigran Product team
Projects
None yet
Development

No branches or pull requests

1 participant