Bad Request: Request Line is too large (5465 > 4094)

[ivorbosloper]

I tried the current master (1.0-beta11) to be able to use setUser(). However, events can not be logged anymore because the request line is too long (my sentry installation is deployed behind Apache, which limits requestline lengh to 4096).

The makerequest function has been changed from a POST to a GET:

function makeRequest(data) {
    new Image().src = globalServer + getAuthQueryString() + '&sentry_data=' + encodeURIComponent(JSON.stringify(data));
}

With GET requests the data will always be limited (IE8 and IE9 limit it up to 2083 chars), and I think POST should always be used (or at least be an option).

[rassie]

👍, concerns me too even though I haven't reached the limit yet. Have to support IE7 and higher.

[mattrobenolt]

I would highly highly encourage modifying these values in Apache:

• http://httpd.apache.org/docs/2.2/mod/core.html#LimitRequestFieldSize
• http://httpd.apache.org/docs/2.2/mod/core.html#LimitRequestLine

I'm aware that there are limitations across browsers, and I try my best to keep payloads as small as possible, and will be doing things in the future to help it even more.

For now though, a POST request provides it's own set of incompatibilities, especially in regards to IE8 and 9, so it's a lose lose situation, honestly. This is all terrible, and it has been a battle over what a browser will even allow us to do.

[mattrobenolt]

@rassie, at this point, raven-js wouldn't even work in IE7 without an external JSON library like json2.js or the like. Just FYI.

[rassie]

@mattrobenolt json2.js is part of my assets anyway, so no problem there :)

[rassie]

So not using POST is CORS-related and -limited, while GET is limited by the URL length (either client-side or server-side)?

[mattrobenolt]

Awesome. I'll be honest, I've done 0 testing at this point for IE7, so I can't guarantee that it works. I've tested with IE8+.

[rassie]

I haven't upgraded my main Sentry instance to >=5.1.5 yet, but yeah, if it won't work, you'll know. I'm putting great hopes in TraceKit for IE7 ;-)

[mattrobenolt]

Correct. CORS is a much bigger problem, in my opinion, and more tricky to set up/configure/deploy correctly. It's easier to attempt to keep the GET request smaller, and more browsers handle longer querystrings. I will be documenting all of these tips and tricks.

We've added some new features to Sentry as well that allows Sentry to fetch the actual lines of context for the error for you, so the client doesn't need to send them. Of course, there's no way for raven-js to automatically know if your assets are at a publicly accessible URL or not without telling it, so we'll probably expose that as an option in the future. This would drastically shorten the payload needed to be sent. Hence why I'd prefer to move forward with a GET instead of dealing with a POST.

Bottom line: a GET request is an issue with potential solutions. POST is just a road block that we can't get around.

[mattrobenolt]

@rassie, totally! Please let me know. It's not our #1 priority, obviously, but IE7 it is a concern. We just have to make it actually usable in at least the modern browsers before we begin to make a crappy one work. :)

[rassie]

@mattrobenolt If Sentry accepts POST payload, could you still introduce an option for those of us who could get their CORS to work, e.g. running Sentry on the same domain? Maybe even with GET as a fallback, but I still consider POSTing a bit cleaner. Thanks.

[mattrobenolt]

It requires a bit of extra bloat and testing to add to the library to support that. If anything, I'd provide two separate builds of raven.js. One that uses POST and one that uses a GET.

I'll definitely look into it. At the moment, GET is the most sane way of handling everything. It's limitations aren't that severe. Like I said, Sentry has the features now to fetch js sources on it's own, so the client technically doesn't even need to send as large of a payload if the sources are accessible. It already opts out of sending if the Javascript source is minified, so this problem probably isn't as large as it seems.

[ivorbosloper]

Thanks for the responses. I understand both POST and GET have problems. I tested the current source with a simple error (small stacktrace) and noticed it already didn't work. I'll watch the repo and will test new solutions as they come along.

@mattrobenolt I had already tried to change the Apache LimitRequestLine in the first vhost of the bound port, but I still ran into the error. Maybe the wsgi-server has limits too?

[mattrobenolt]

Do you mind sending me an example of a payload generated? I'm curious why
it's so large in the first place. I've had really good luck keeping them
~1000 characters.

Sent from my iPhone

On Jan 23, 2013, at 4:42, Ivor notifications@github.com wrote:

Thanks for the responses. I understand both POST and GET have problems. I
tested the current source with a simple error (small stacktrace) and
noticed it already didn't work. I'll watch the repo and will test new
solutions as they come along.

@mattrobenolt https://github.com/mattrobenolt I had already tried to
change the Apache LimitRequestLine in the first vhost of the bound port,
but I still ran into the error. Maybe the wsgi-server has limits too?

—
Reply to this email directly or view it on
GitHubhttps://github.com//issues/58#issuecomment-12593828.

[ivorbosloper]

@mattrobenolt This failing image:

http://sentry.example.com/api/3/store/?sentry_version=2.0&sentry_client=raven-js/1.0-beta11&sentry_key=XXXXXXXXXXXXX&sentry_data=%7B%22project%22%3A3%2C%22logger%22%3A%22javascript%22%2C%22platform%22%3A%22javascript%22%2C%22sentry.interfaces.Http%22%3A%7B%22url%22%3A%22http%3A%2F%2Flocalhost.example.com%3A8000%2Fexample%2Fdashboard%2F%22%2C%22querystring%22%3A%22%22%7D%2C%22sentry.interfaces.Exception%22%3A%7B%22type%22%3A%22ReferenceError%22%2C%22value%22%3A%22testjavascripterror%20is%20not%20defined%22%7D%2C%22sentry.interfaces.Stacktrace%22%3A%7B%22frames%22%3A%5B%7B%22abs_path%22%3A%22http%3A%2F%2Flocalhost.example.com%3A8000%2Fmedia%2Fmain.js%257C0%2F1%2Fravenjs%2Fraven-1.0-beta11.js%3Fversion%3D5c38ee8f0eb13dfa5e7d4a54e675a15ceef9ffb5%22%2C%22filename%22%3A%22raven-1.0-beta11.js%3Fversion%3D5c38ee8f0eb13dfa5e7d4a54e675a15ceef9ffb5%22%2C%22lineno%22%3A182%2C%22function%22%3A%22%22%2C%22post_context%22%3A%5B%22%20%20%20%20%7D%22%2C%22%22%2C%22%20%20%20%20report.subscribe%20%3D%20subscribe%3B%22%2C%22%20%20%20%20report.unsubscribe%20%3D%20unsubscribe%3B%22%2C%22%20%20%20%20return%20report%3B%22%5D%2C%22context_line%22%3A%22%20%20%20%20%20%20%20%20throw%20ex%3B%20%2F%2F%20re-throw%20to%20propagate%20to%20the%20top%20level%20(and%20cause%20window.onerror)%22%2C%22pre_context%22%3A%5B%22%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20lastException%20%3D%20null%3B%22%2C%22%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20notifyHandlers.apply(null%2C%20%5Bstack%2C%20null%5D.concat(args))%3B%22%2C%22%20%20%20%20%20%20%20%20%20%20%20%20%7D%22%2C%22%20%20%20%20%20%20%20%20%7D%2C%20(stack.incomplete%20%3F%202000%20%3A%200))%3B%22%2C%22%22%5D%2C%22in_app%22%3Atrue%7D%2C%7B%22abs_path%22%3A%22http%3A%2F%2Flocalhost.example.com%3A8000%2Fexample%2Fdashboard%2F%22%2C%22filename%22%3A%22http%3A%2F%2Flocalhost.example.com%3A8000%2Fexample%2Fdashboard%2F%22%2C%22lineno%22%3A79%2C%22colno%22%3A3%2C%22function%22%3A%22HTMLDocument.%3Canonymous%3E%22%2C%22post_context%22%3A%5B%22%5Ct%7D)%3B%22%2C%22%3C%2Fscript%3E%22%2C%22%22%2C%22%20%20%20%20%22%2C%22%20%20%20%20%3C!--%20End%20Javascripts%20--%3E%22%5D%2C%22context_line%22%3A%22%5Ct%5Cttestjavascripterror.test()%3B%22%2C%22pre_context%22%3A%5B%22%3Cscript%20type%3D%5C%22text%2Fjavascript%5C%22%20src%3D%5C%22%2Fmedia%2Fdashboard_templates.js--en%257C0%2F0%2F2%2Fexample%2Fdashboard%2FfieldTemplates.html%3Fversion%3D7fbfdf17eea44c847117669bf80c0c90701a694e%5C%22%3E%3C%2Fscript%3E%22%2C%22%3Cscript%20type%3D%5C%22text%2Fjavascript%5C%22%20src%3D%5C%22%2Fmedia%2Fdashboard_templates.js--en%257C0%2F0%2F3%2Fexample%2Fdashboard%2FstockTemplates.html%3Fversion%3D184fb0aca156d483e58f25d74dacfdafd7a5c25f%5C%22%3E%3C%2Fscript%3E%22%2C%22%3Cscript%20type%3D%5C%22text%2Fjavascript%5C%22%20src%3D%5C%22%2Fmedia%2Fdashboard_templates.js--en%257C0%2F0%2F4%2Fexample%2Fdashboard%2FyieldTemplates.html%3Fversion%3Ddb197fcf83bb9efc9f765074055645069d78b0b0%5C%22%3E%3C%2Fscript%3E%22%2C%22%3Cscript%3E%22%2C%22%5Ct%24(document).ready(function()%20%7B%22%5D%2C%22in_app%22%3Atrue%7D%2C%7B%22abs_path%22%3A%22http%3A%2F%2Flocalhost.example.com%3A8000%2Fmedia%2Fmain.js%257C0%2F1%2Fravenjs%2Fraven-1.0-beta11.js%3Fversion%3D5c38ee8f0eb13dfa5e7d4a54e675a15ceef9ffb5%22%2C%22filename%22%3A%22raven-1.0-beta11.js%3Fversion%3D5c38ee8f0eb13dfa5e7d4a54e675a15ceef9ffb5%22%2C%22lineno%22%3A1121%2C%22colno%22%3A27%2C%22function%22%3A%22HTMLDocument._fn%22%2C%22post_context%22%3A%5B%22%20%20%20%20%20%20%20%20%20%20%20%20%7D%20catch%20(e)%20%7B%22%2C%22%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20TraceKit.report(e)%3B%22%2C%22%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20throw%20e%3B%22%2C%22%20%20%20%20%20%20%20%20%20%20%20%20%7D%22%2C%22%20%20%20%20%20%20%20%20%7D%3B%22%5D%2C%22context_line%22%3A%22%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20return%20fn.apply(this%2C%20arguments)%3B%22%2C%22pre_context%22%3A%5B%22%22%2C%22%20%20%20%20var%20_oldReady%20%3D%20%24.fn.ready%3B%22%2C%22%20%20%20%20%24.fn.ready%20%3D%20function%20traceKitjQueryReadyWrapper(fn)%20%7B%22%2C%22%20%20%20%20%20%20%20%20var%20_fn%20%3D%20function%20()%20%7B%22%2C%22%20%20%20%20%20%20%20%20%20%20%20%20try%20%7B%22%5D%2C%22in_app%22%3Atrue%7D%2C%7B%22abs_path%22%3A%22http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fjquery%2F1.7.1%2Fjquery.min.js%22%2C%22filename%22%3A%22jquery.min.js%22%2C%22lineno%22%3A2%2C%22colno%22%3A14784%2C%22function%22%3A%22n%22%2C%22in_app%22%3Atrue%7D%2C%7B%22abs_path%22%3A%22http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fjquery%2F1.7.1%2Fjquery.min.js%22%2C%22filename%22%3A%22jquery.min.js%22%2C%22lineno%22%3A2%2C%22colno%22%3A15553%2C%22function%22%3A%22Object.o.fireWith%22%2C%22in_app%22%3Atrue%7D%2C%7B%22abs_path%22%3A%22http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fjquery%2F1.7.1%2Fjquery.min.js%22%2C%22filename%22%3A%22jquery.min.js%22%2C%22lineno%22%3A2%2C%22colno%22%3A9773%2C%22function%22%3A%22Function.e.extend.ready%22%2C%22in_app%22%3Atrue%7D%2C%7B%22abs_path%22%3A%22http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fjquery%2F1.7.1%2Fjquery.min.js%22%2C%22filename%22%3A%22jquery.min.js%22%2C%22lineno%22%3A2%2C%22colno%22%3A14348%2C%22function%22%3A%22HTMLDocument.c.addEventListener.B%22%2C%22in_app%22%3Atrue%7D%5D%7D%2C%22culprit%22%3A%22http%3A%2F%2Flocalhost.example.com%3A8000%2Fexample%2Fdashboard%2F%22%2C%22message%22%3A%22testjavascripterror%20is%20not%20defined%22%2C%22sentry.interfaces.User%22%3A%7B%22user%22%3A%22ivor%40example.com%22%2C%22id%22%3A%221%22%7D%7D

This decodes into:

"project":3,
"logger":"javascript",
"platform":"javascript",
"sentry.interfaces.Http":{"url":"http://localhost.example.com:8000/example/dashboard/","querystring":""},
"sentry.interfaces.Exception":{"type":"ReferenceError","value":"testjavascripterror is not defined"},
"sentry.interfaces.Stacktrace":{"frames":[{"abs_path":"http://localhost.example.com:8000/media/main.js%7C0/1/ravenjs/raven-1.0-beta11.js?version=5c38ee8f0eb13dfa5e7d4a54e675a15ceef9ffb5","filename":"raven-1.0-beta11.js?version=5c38ee8f0eb13dfa5e7d4a54e675a15ceef9ffb5","lineno":182,"function":"","post_context":["    }","","    report.subscribe = subscribe;","    report.unsubscribe = unsubscribe;","    return report;"],"context_line":"        throw ex; // re-throw to propagate to the top level (and cause window.onerror)","pre_context":["                lastException = null;","                notifyHandlers.apply(null, [stack, null].concat(args));","            }","        }, (stack.incomplete ? 2000 : 0));",""],"in_app":true},{"abs_path":"http://localhost.example.com:8000/example/dashboard/","filename":"http://localhost.example.com:8000/example/dashboard/","lineno":79,"colno":3,"function":"HTMLDocument.<anonymous>","post_context":["\\t});","</script>","","    ","    <!-- End Javascripts -->"],"context_line":"\\t\\ttestjavascripterror.test();","pre_context":["<script type=\\"text/javascript\\" src=\\"/media/dashboard_templates.js--en%7C0/0/2/example/dashboard/fieldTemplates.html?version=7fbfdf17eea44c847117669bf80c0c90701a694e\\"></script>","<script type=\\"text/javascript\\" src=\\"/media/dashboard_templates.js--en%7C0/0/3/example/dashboard/stockTemplates.html?version=184fb0aca156d483e58f25d74dacfdafd7a5c25f\\"></script>","<script type=\\"text/javascript\\" src=\\"/media/dashboard_templates.js--en%7C0/0/4/example/dashboard/yieldTemplates.html?version=db197fcf83bb9efc9f765074055645069d78b0b0\\"></script>","<script>","\\t$(document).ready(function() {"],"in_app":true},{"abs_path":"http://localhost.example.com:8000/media/main.js%7C0/1/ravenjs/raven-1.0-beta11.js?version=5c38ee8f0eb13dfa5e7d4a54e675a15ceef9ffb5","filename":"raven-1.0-beta11.js?version=5c38ee8f0eb13dfa5e7d4a54e675a15ceef9ffb5","lineno":1121,"colno":27,"function":"HTMLDocument._fn","post_context":["            } catch (e) {","                TraceKit.report(e);","                throw e;","            }","        };"],"context_line":"                return fn.apply(this, arguments);","pre_context":["","    var _oldReady = $.fn.ready;","    $.fn.ready = function traceKitjQueryReadyWrapper(fn) {","        var _fn = function () {","            try {"],"in_app":true},{"abs_path":"http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js","filename":"jquery.min.js","lineno":2,"colno":14784,"function":"n","in_app":true},{"abs_path":"http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js","filename":"jquery.min.js","lineno":2,"colno":15553,"function":"Object.o.fireWith","in_app":true},{"abs_path":"http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js","filename":"jquery.min.js","lineno":2,"colno":9773,"function":"Function.e.extend.ready","in_app":true},{"abs_path":"http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js","filename":"jquery.min.js","lineno":2,"colno":14348,"function":"HTMLDocument.c.addEventListener.B","in_app":true}]},
"culprit":"http://localhost.example.com:8000/example/dashboard/",
"message":"testjavascripterror is not defined",
"sentry.interfaces.User":{"user":"ivor@example.com","id":"1"}}

[mattrobenolt]

Thanks @ivorbosloper. I'll use this as a good reference to see where we can cut corners and reduce the amount of characters being sent over the wire.

[mattrobenolt]

On good news, it captured the error correctly! 👍

[mattrobenolt]

@ivorbosloper, we're going to, by default, not send the lines of context, which is the majority of the bloat. Since Sentry now supports fetching the source itself, we can avoid all of that and leave it behind a configuration option to enable specifically for scenarios where your app is running behind a firewall or in some other non-publicly accessible location.

[ivorbosloper]

@mattrobenolt I could test a new version and close the issue. Great project btw :)

[mattrobenolt]

Feel free to try out: http://d3nslu0hdya83q.cloudfront.net/build/master/raven.min.js

That's our build from master. It's updated as a post-commit hook. This build by default doesn't make an attempt to send the lines of context, but let's Sentry try and fetch them for us. You can disable this behavior, which will make the payload large again, by using fetchContext: true as an option to config, like:

Raven.config(..., {fetchContext: true}).install();

[riklaunim]

I have an app using emberjs and some jQuery widgets (like jQuery File Upload widget). The hierarchy is short and quite often I run out of limit:

http://wklej.org/id/955778/
http://wklej.org/id/955780/

And that's even with that raven.min.js.

[mattrobenolt]

@riklaunim, so it looks like everything is correct on our end, just the generated payload is too large. There's nothing we can do about this, except suggest fixing your web server to handle it. The alternatives are not worth the amount of effort needed to support them right now. Sorry. :(

[riklaunim]

Yes, that what we are going to do - configure nginx and whatever is between it and sentry to let it pass.

At least tracebacks will be meaningful 👍

[mattrobenolt]

nginx by default is a pretty high value, I'm pretty sure larger than 4096 characters. If not, let me know and I'll add info to the docs.

[coderanger]

This specific message is from gunicorn, not nginx/apache. Actual fix discussed at getsentry/sentry#874

[joar]

Correct me if I'm mistaken: I tthink all CORS setups (fill in allowed domains, send HTTP headers) are managed by sentry, and thus it's not at all very hard to get a working CORS setup on modern browsers.

On the other hand, the Image.src method is an ugly hack, and it disguises itself as a sane solution by working fine most of the times, especially in small projects and projects created by the people that decided on the Image.src hack, since i assume they know to avoid it.

It's brilliant software, and I think it would be more brilliant if it was using CORS with an Image.src hack as fallback for older browsers.

[wejendorp]

I agree, it seems like an overly optimistic solution.
I wanted to test out sentry, but can't use an error tracer that fails because it attempts to get a bit of context.

[mattrobenolt]

@wejendorp Not sure I understand your point. This is not typically an issue. Sentry fetches context on the server side, and the client doesn't normally fetch it unless you're operating in a scenario where Sentry won't be able to fetch. For example, in an application behind a firewall, or on a mobile phone, behind HTTP basic auth, etc.

Or is this what you're referring to?

[wejendorp]

I wasn't aware of that, and I'm sorry for not taking the time to understand what was going on.
All I know is that I was attempting to swap errorception for sentry, and only managed to get a "Hello world" error through to the server, never a stacktrace from my (angular) app.
So I thought it quite ironic that the error handling script was failing.