[carbonlink] Python 3 Pickle Compatible Metric Request Parser #340
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
When graphite-web is installed on Python 3.0+ environments
graphite-web
will send pickle request tocarbonlink
using the highest pickle protocol (link to code in graphite-web) .In python 3.0 until 3.7 highest pickle protocol is
4
, while in python 3.8 highest is5
.Since
go-carbon's
carbonlink
was written for python 2.x, when highest protocol available was2
, it is unable to parsecarbonlink
pickle messages coming from pythongraphite-web
. Even if the protocol is set to2
when using python 3.x it won't be able to parse it due encoding change from ASCII in python 2.x to Unicode in python 3.x.Steps to recreate
Run
go-carbon
normally, but rungraphite-web
in Python 3.0+ and request a metric that will utilizecarbonlink
. You will see protocol 4 message request coming *protocol 5 in Python 3.8+). Consequentlygo-carbon
will WARN regardingbad pickle message
error and wont serve metric from cache.PR Fixes
This PR should allow
carbonlink
to handle pickle protocols from 2 up to 5 by expanding thecarbonlink
pickle message parser to Python <= 3.8 pickle messages. More on python pickle protocols can be found hereNote: Will make PR later in graphite-web to make pickle protocol configurable for
carbonlink
requests, so that if newer python version with higher pickle protocol comes out it will not breakcarbonlink
communication (one can set desired protocol) ifgo-carbon
will not be able to parse it.