Infinite Stack Recursion with HTTP #1999
Comments
|
@saulshanabrook Thanks for reporting this, but can you simplify the reproduction steps down to just a Ruby/Rails application? The amount of dependencies and manual operations required to run it seems to be quite high:
I gave up on step 3 because I don't feel comfortable changing system configs (even harmless) for reproducing an issue that doesn't require it. |
|
@saulshanabrook also is this a regression introduced by 5.8.0 (since that's the version in your issue) or does it also happen on older versions? |
|
ok so what is happening is this @saulshanabrook can you print |
|
@st0012 Yeah, that totally makes sense about the complicated reproduction steps. I have tested this to reproduce the bug, which doesn't require messing with hosts, getting a sentry config, or having a domain: $ git clone https://github.com/saulshanabrook/discourse-hosting
$ cd discourse-hosting
$ echo 'HOSTNAME=any-domain.com' > .env
$ docker compose run --rm upload_assetsLet me know if that doesn't work to reproduce it. |
|
@sl0thentr0py I was just starting to use sentry with this project. I tried it with 5.7.0 a few weeks ago when that was the latest, was hitting the same error, upgraded to 5.8.0 to see if it would fix it but to no avail. Do you recommend I try downgrading to an earlier version? |
|
@sl0thentr0py Sure, is this what you mean? I believe it's $ docker compose run --rm upload_assets rdbg -c -- bundle exec rake --trace s3:upload_assets
WARN[0000] The "SEND_GRID_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "SECRET_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "DSN" variable is not set. Defaulting to a blank string.
WARN[0000] The "DSN" variable is not set. Defaulting to a blank string.
WARN[0000] The "EMAIL" variable is not set. Defaulting to a blank string.
WARN[0000] The "SECRET_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "SEND_GRID_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "DSN" variable is not set. Defaulting to a blank string.
WARN[0000] The "DSN" variable is not set. Defaulting to a blank string.
WARN[0000] The "SECRET_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "SEND_GRID_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "DSN" variable is not set. Defaulting to a blank string.
WARN[0000] The "SEND_GRID_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "SECRET_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "EMAIL" variable is not set. Defaulting to a blank string.
WARN[0000] The "SEND_GRID_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "EMAIL" variable is not set. Defaulting to a blank string.
WARN[0000] The "SEND_GRID_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "EMAIL" variable is not set. Defaulting to a blank string.
WARN[0000] The "SEND_GRID_API_KEY" variable is not set. Defaulting to a blank string.
WARN[0000] The "EMAIL" variable is not set. Defaulting to a blank string.
WARN[0000] The "SEND_GRID_API_KEY" variable is not set. Defaulting to a blank string.
[+] Running 4/0
⠿ Container discourse-hosting-minio-1 Running 0.0s
⠿ Container discourse-hosting-redis-1 Running 0.0s
⠿ Container discourse-hosting-db-1 Running 0.0s
⠿ Container discourse-hosting-glitchtip-web-1 Running 0.0s
[4, 13] in /usr/local/bin/rake
4| #
5| # The application 'rake' is installed as part of a gem, and
6| # this file is here to facilitate running it.
7| #
8|
=> 9| require 'rubygems'
10|
11| Gem.use_gemdeps
12|
13| version = ">= 0.a"
=>#0 <top (required)> at /usr/local/bin/rake:9
#1 [C] Kernel.load at /usr/local/bundle/gems/bundler-2.3.22/lib/bundler/cli/exec.rb:58
# and 14 frames (use `bt' command for all frames)
(rdbg) b /usr/local/bundle/gems/sentry-ruby-5.8.0/lib/sentry/net/http.rb:35 # break command
#0 BP - Line (pending) /usr/local/bundle/gems/sentry-ruby-5.8.0/lib/sentry/net/http.rb:35
(rdbg) c # continue command
** Invoke s3:upload_assets (first_time)
** Invoke environment (first_time)
** Execute environment
** Invoke s3:ensure_cors_rules (first_time)
** Invoke environment
** Execute s3:ensure_cors_rules
Installing CORS rules...
skipping
** Execute s3:upload_assets
[30, 39] in /usr/local/bundle/gems/sentry-ruby-5.8.0/lib/sentry/net/http.rb
30| return super if from_sentry_sdk?
31|
32| Sentry.with_child_span(op: OP_NAME, start_timestamp: Sentry.utc_now.to_f) do |sentry_span|
33| set_sentry_trace_header(req, sentry_span)
34|
=> 35| super.tap do |res|
36| record_sentry_breadcrumb(req, res)
37|
38| if sentry_span
39| request_info = extract_request_info(req)
=>#0 block {|sentry_span=nil|} in request at /usr/local/bundle/gems/sentry-ruby-5.8.0/lib/sentry/net/http.rb:35
#1 Sentry::Hub#with_child_span(instrumenter=:sentry, attributes={:op=>"http.client", :start_timestamp=>16..., block=#<Proc:0x00007f9058c78a40 /usr/local/bund...) at /usr/local/bundle/gems/sentry-ruby-5.8.0/lib/sentry/hub.rb:100
# and 102 frames (use `bt' command for all frames)
Stop by #0 BP - Line /usr/local/bundle/gems/sentry-ruby-5.8.0/lib/sentry/net/http.rb:35 (line)
(ruby) Net::HTTP.ancestors
[Sentry::Net::HTTP] |
|
no i just wanted to know if this was a regression or not. In that case downgrading won't help, we'll need to find the problem in this setup and fix it. |
|
I am also getting the same problem in the main server, I believe in a sidekiq worker process: Discourse also captures some of the stacktrace:
Let me know if you have any ideas on ways to troubleshoot this! |
|
ok some investigation, still not sure what the issue is. discourse has the following in their gemfile (faraday with explicit net-http gem) and then you're adding sentry manually in the Dockerfile with @st0012 do you see any problems with the patching in this scenario? Normally the ancestors should look like this [1] pry(main)> Net::HTTP.ancestors
=> [Sentry::Net::HTTP, Net::HTTP, Net::Protocol, Object, PP::ObjectMixin, JSON::Ext::Generator::GeneratorMethods::Object, Kernel, BasicObject]while you have just |
|
@sl0thentr0py I tried adding |
|
It's because discourse injects method_profiler, which uses We had several cases like this:
Given that more and more tools use Debugging note: As a major contributor of the I based my investigation on the command you provided:
|
|
wow great debugging @st0012 ! |
|
@saulshanabrook does that answer your question? You'll need to patch discourse upstream to make it work with sentry, I don't think we can do much here. If it resolves your question, please close this issue. |
|
Wow @st0012 that is excellent detective work! I had seen the Thank you for your work on the debug gem! I have actually never used ruby before trying to get Discourse up and running and the debugging experience was surprisingly pleasant :) I posted a topic on the Discourse meta forum about this issue, so I will reply with your suggested fix (replacing use of Thank you all for helping me debug this, I really thought I would just have to give up on using Sentry for this project ❤️. |
Issue Description
Hello!
I am coming across an error trying to use the sentry ruby bindings on with a rails app, during a rake task. The task uploads the precompiled assets to an S3 server. It exits cleanly without the sentry integration, but when I add it, it enters into an infinitely recursive loop with the patched HTTP module. I am not sure how to debug this further or what to do.
Thank you!
Reproduction Steps
edited with simpler steps:
Following the steps of https://github.com/saulshanabrook/discourse-hosting#local-development-with-docker-compose the error will be raised when running
docker compose run --rm upload_assets:Expected Behavior
The command exits cleanly without an error
Actual Behavior
It raises a
SystemStackError:Ruby Version
3.1
SDK Version
sentry-ruby-5.8.0
Integration and Its Version
No response
Sentry Config
The text was updated successfully, but these errors were encountered: