Twitter fetch failure info & configurable timeout #65
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If the twitter get fails, provide a little more info in the error message. This adds the response message and the error code. My local instance gets polled every 20 mins from a rss2imap cron job. Lately I've been getting sporadic 404 failures, but clicking through the same link works fine. I added this to try to track down what's going wrong. Turns out they've all been timeout errors.
From the previous commit, I discovered my sporadic failures were all timeouts. Hardcoding this to 10 seconds made them go away. This environment variable allows the default 2 seconds to be overridden. I don't know if this needs a corresponding change in the Apache config. I notice the `-appConnTimeout 3` in the apache config.
If the TWITRSSME_TIMEOUT_SEC value gets increased, the apache config timeouts need to increase as well. Rather than use a hardcoded apache.conf, create one on the fly using computed environment variables.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If the twitter get fails, add the response message to the existing "can't screenscrape twitter" message and return the response error code.
My local instance gets polled every 20 mins from a rss2imap cron job. Lately I've been getting sporadic 404 failures, but clicking through the same link then works fine.
I added this to try to track down what's going wrong. Turns out they've all been timeout errors.
To that end, this PR also adds a
TWITRSSME_TIMEOUT_SECenvironment variable, which allows this to be customised. The original value of 2 seconds is left as a default.In the Apache config, the
-idle-timeoutand-appConnTimeoutwould need to be increased to handle the longer app timeout. For the docker startup, the apache config file is now dynamically created, and these values get set as appropriate.It may of course just be simpler to increase the hardcoded
2, but I assume your live version had the low timeout value for a reason.