[Epic] Pure C API

[amomchilov]

Shopify is actively working on decoupling the RBS parser from its Ruby bindings, in order to expose a new pure C API.

This API will be more compatible (e.g. it can integrate into JRuby, Truffle, Sorbet, etc.), and will allow for multi-threaded RBS parsing (since it won't be blocked by Ruby's GVL).

PR list

This is still a work-in-progress, but this list will be updated to list all PRs that are ready to review, in the order that they should be merged.

1. Add templating mechanism to automatically generate .c/.h files #2098
2. Clean up temporary workarounds from templated code #2102
3. Un-hardcode repo name in test suite #2128
4. Designated initializers #2131
5. Remove forward declarations of local variables #2132
6. Extract out usages of Ruby IDs #2175
7. Reimplement the RBS Constant Pool in pure C #2221

CC @Morriar, @soutaro

[headius]

It looks like all the linked PRs have been merged. Is this done? Is there a way to get a prerelease gem pushed for testing, or JRuby added to CI?

[Morriar]

The main missing part would be #2398.

[headius]

@Morriar Ok thank you for that update!

In #2433 I noticed there are still a number of places that the ruby.h file and friends are required. I'm not sure if #2398 addresses all of those but any dependency on the header files from CRuby will likely need to be addressed in order for the library to build and run with JRuby.

I am standing by to help. The current dependencies prevent doing much on my end, though.

[Morriar]

With #2398, ruby.h isn't required anymore.

Here's how we use it from Sorbet (C++ implementation): https://github.com/sorbet/sorbet/blob/master/rbs/rbs_common.h#L5 and we don't need the Ruby VM.