-
Notifications
You must be signed in to change notification settings - Fork 609
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
mysql2 related crashes when running specs #2662
Comments
Do you have a way for us to reproduce the problem? Just the stacktrace gives very little information and doesn't give enough to either solve of further investigate the problem. It might be a bug in Rubinius or Mysql2 or perhaps something completely different, but it's impossible to say what is going with only this information. |
I will see if I can come up with a reproducible environment. By the way, I did also encounter GC related crashes, such as:
#2655 (comment) seems to have fixed such crashes. 50 rspec runs and no GC related crashes yet 👍 (mysql2 crashes still happen though...) |
@sayap Getting a repro would be great and would really help with debugging this, since otherwise it gets really tricky and ends up being guess work about what's going on :). Seeing |
Do you think you can extract the issue for the mysql2 crash so we can investigate that? |
In mysql2, we recently fought hard with Ruby GC and hoped that we'd won... @sayap Can you provide a repro script so that I can try to see the errors on my own? |
@sodabrew What kind of issues were those? |
Some internal refcounting and RB_GC_GUARD usage to prevent the GC from freeing C structures out from under us. The mysql2 0.3.13 release has this work merged: |
@dbussink I also just merged a fix for mysql connections that were left dangling after a GC run. However, the specs for it require the ability to call @dbussink is there a way I can pass |
@sodabrew You can use the RBXOPT environment variable for options like that. |
@sodabrew I checked that spec also btw, and it seem pretty problematic to me to be honest. Nothing prevents a GC from happening at other points than where you do GC.start, so it could perhaps sporadically fail even when it's not a problem. Might be more theoretical problem, but things like this are always very tricky with specs like that. |
Got it, spec is passing now. Yes, other GC triggers are certainly a risk for that test, but it works reliably enough to catch and demonstrate the fix to the resource problem that was there before. All that said, I re-read the original backtrace gist, and @sayap could you recompile mysql2 with debugging symbols (CFLAGS="-g") and then repro the crash once more? There's only a memory address within mysql2 and that's not enough to narrow down the problem (or even implicate mysql2 necessarily). |
I don't know if @sayap also uses nokogiri, but I also sent a pull request there to fix a crash bug. |
Still happen with latest rbx, latest mysql2, and nokogiri-1.5.10: https://gist.github.com/sayap/7623735 Will try recompiling mysql2 with debugging symbols, along with https://github.com/dbussink/nokogiri/commit/3fc79c0744d7a00da2d961eebcc99141a3fdf99c Unfortunately, no reproducible environment yet... |
@sayap Did you manage to figure out a way to reproduce this in the mean time? |
Closing this one due to the lack of feedback. Feel free to re-open if the issue still persists when using Rubinius 2.5.3. |
I'm starting to see this pretty often on Travis with rbx-2.5.7: https://gist.github.com/sodabrew/656e0f44d33d6b46dc1b I have a bunch of these, coming from different places in the Ruby trace, but the C traces are all similar. Memory pressure or GC thresholds would be triggering a GC run, at which point the mark function is called on all object, and mysql2 calls |
I guess, basically, "what would cause rb_gc_mark to segfault, and should I be testing against that condition before calling it?" |
My guess is that either our GC is messed up or somehow |
I searched for other Rubinius bugs with the text "saw_object" and traced my way to sparklemotion/nokogiri#1047 - since mysql2 is also a C extension, the issues might be related? (Edit: oh derp, you even linked to nokogiri above). |
@sodabrew didn't you recently fix another GC-related issue in mysql2? Could this be related? Did you run this under Valgrind? Is this consistently reproducible when running the tests? Have you built Rubinius in debug mode ( |
I built rbx in debug and reproduced |
@tamird When this occurs could you please provide GDB backtraces as explained at http://rubini.us/doc/en/how-to/obtaining-gdb-backtraces/? Also if you could print the various variables in the offending call frame (using |
I haven't seen any failures on Travis today - Is it possible that Rubinius 2.5.8 fixes this? |
Running Rubocop in the same VM that ran the mysql2 tests failed here: https://travis-ci.org/brianmario/mysql2/jobs/71173451 |
Yep, bummer. Here's a gist of that crash for future reference: https://gist.github.com/sodabrew/e7de223e019a390a3926 |
In general, the MRI C-API for Rubinius is deprecated, but that doesn't mean it will be going anywhere soon. Compatibility for C-extensions will continue to be evaluated on a case-by-case basis. However, C-extensions that depend on functions that examine or manipulate built-in MRI data structures will never be supported. The focus for Rubinius in the near term is on the following capabilities:
Contributions in the form of PRs for any of the areas of focus above are appreciated. |
When running the specs of my application, the process would crash intermittently. Seems like a similar issue to #2655, but specific to mysql2.
https://gist.github.com/sayap/6861102
rubinius 2.0.0 (2.1.0 2013-10-04 JI) [x86_64-linux-gnu]
Linux henry 3.11.1-ck1 #1 SMP PREEMPT Sun Sep 22 17:47:30 MYT 2013 x86_64 Intel(R) Core(TM) i5-2410M CPU @ 2.30GHz GenuineIntel GNU/Linux
The text was updated successfully, but these errors were encountered: