Occur Encoding::UndefinedConversionError when include escaped string in request

[keisuke-koshikawa]

Describe the bug

I try to catch exception with Sentry.capture_exception but occured Encoding::UndefinedConversionError.
After investigating, the error occurred because it contained an escaped string such as \ xE3 in event_hash.

https://github.com/getsentry/sentry-ruby/blob/master/sentry-ruby/lib/sentry/transport.rb#L58

error log

Unable to record event with remote Sentry server (Encoding::UndefinedConversionError - "\xE5" from ASCII-8BIT to UTF-8)

To Reproduce
request string include an escaped string such as \ xE3, \xE5.

Expected behavior
I`d like to avoid the error even if it contains escaped strings.

Actual behavior
occur Encoding::UndefinedConversionError - "\xE5" from ASCII-8BIT to UTF-8 if event_hash contain escaped string such as \ xE3, \xE5.

Environment

• Ruby Version: upper 2.6
• SDK Version: latest
• Integration Versions (if any):
• e.g. Rails 5.2

Raven Config

This is not necessary but could be helpful.

Raven.configure do |config|
  # the config you're using, without DSN and other sensitive data
end

[st0012]

@keisuke-koshikawa are you using sentry-ruby only or also sentry-rails? and by latest do you mean the master version?

[keisuke-koshikawa]

@st0012
yes, I use sentry-ruby and sentry-rails.

[st0012]

@keisuke-koshikawa I can reproduce errors with escaped characters, but with a different error: JSON::GeneratorError: partial character in source. can you provide me minimum reproduction steps that can produce the error you see?

[keisuke-koshikawa]

@st0012
sorry, I forgot how to reproduce...

but I found a request when I was testing. I use graphql-ruby.

:request=>
  {:data=>
    "{\"query\":\"mutation upsertSampleRecord($SampleRecordInput: SampleRecordInputForm!)
      {\\n  upsertSampleRecord(SampleRecordInput: $SampleRecordInput)
        {\\n
          success\\n
        }\\n
      }\",
      \"variables\": {
        \"SampleRecordInput\" : {
          \"SampleRecord\": {
            \"date\":\"2020-11-30\",
            \"content\":\"\xE5\x86\x85\",
            \"title\":\"\xE3\x82\xBF\xE3\x82\xA4\",
            \"isPublished\":\"publish\"
          }
        }
      },
      \"operationName\":\"upsertSampleRecord\"
    }"

[matheusbsilva]

Any updates on this issue? I'm facing the same problem using sentry-ruby 4.1.6 and sentry-rails 4.1.7. In my case the problem occurs when there are special characters like: í and ° in the request:

sentry: Unable to record event with remote Sentry server (Encoding::UndefinedConversionError - "\xC3" from ASCII-8BIT to UTF-8)

[matheusbsilva]

Looking the traceback the problem occurs in a JSON.generate call, I tried to parse the request content using JSON.generate and worked fine. The problem only happens when the request content is in an Event object.

W, [2021-05-11T08:21:08.250685 #1]  WARN -- sentry: Unable to record event with remote Sentry server (Encoding::UndefinedConversionError - "\xC3" from ASCII-8BIT to UTF-8):
--
/usr/local/lib/ruby/2.7.0/json/common.rb:224:in `generate'
/usr/local/lib/ruby/2.7.0/json/common.rb:224:in `generate'
/home/fretadao/app/vendor/bundle/ruby/2.7.0/gems/sentry-ruby-core-4.1.6/lib/sentry/transport.rb:58:in `encode'
/home/fretadao/app/vendor/bundle/ruby/2.7.0/gems/sentry-ruby-core-4.1.6/lib/sentry/transport.rb:73:in `prepare_encoded_event'
/home/fretadao/app/vendor/bundle/ruby/2.7.0/gems/sentry-ruby-core-4.1.6/lib/sentry/transport.rb:27:in `send_event'
/home/fretadao/app/vendor/bundle/ruby/2.7.0/gems/sentry-ruby-core-4.1.6/lib/sentry/client.rb:92:in `send_event'
/home/fretadao/app/vendor/bundle/ruby/2.7.0/gems/sentry-ruby-core-4.1.6/lib/sentry/client.rb:47:in `block in capture_event'
/home/fretadao/app/vendor/bundle/ruby/2.7.0/gems/sentry-ruby-core-4.1.6/lib/sentry/background_worker.rb:33:in `block in perform'
/home/fretadao/app/vendor/bundle/ruby/2.7.0/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:363:in `run_task'
/home/fretadao/app/vendor/bundle/ruby/2.7.0/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:352:in `block (3 levels) in create_worker'
/home/fretadao/app/vendor/bundle/ruby/2.7.0/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:335:in `loop'
W, [2021-05-11T08:21:08.251822 #1]  WARN -- sentry: Failed to submit event. Unreported Event: <no message value>
 ```

[st0012]

@keisuke-koshikawa sorry for the delay. what version are you using now? I tried setting the data you gave to a request and it got processed successfully:

this is the event json:

"request":{"url":"http://localhost:3000/","method":"GET","data":"{\"query\":\"mutation 
upsertSampleRecord($SampleRecordInput: SampleRecordInputForm!)\n      {\\n  upsertSampleRecord(SampleRecordInput: 
$SampleRecordInput)\n        {\\n\n          success\\n\n        }\\n\n      }\",\n      \"variables\": {\n        \"SampleRecordInput\" : {\n      
    \"SampleRecord\": {\n            \"date\":\"2020-11-30\",\n            \"content\":\"\u5185\",\n            \"title\":\"\u30bf\u30a4\",\n            
\"isPublished\":\"publish\"\n          }\n        }\n      },\n      \"operationName\":\"upsertSampleRecord\"\n    }"

[st0012]

@maheskett thanks for reporting this. can you also create a sample app that can reproduce the issue? I also can't reproduce your case

I think it has something to do with the request pipeline or encoding setting in the application, like something escaped the request string first and made it invalid. so it'd be very helpful if I can have an app to reproduce the issue 🙂

[st0012]

Closing because I'm unable to reproduce this issue. If you still have this issue please leave a comment. I'll reopen it and work with you to reproduce it 🙂