Move manageiq and plugins to use zeitwerk autoloader instead of classic

[jrafanie]

https://guides.rubyonrails.org/autoloading_and_reloading_constants.html#rails-autoloaders

Rails 7 requires zeitwerk for autoloading code and we have numerous code loading problems that may be resolved by moving our plugins and core code to using zeitwerk. We just have to decide how we can split it up and do it.

https://github.com/ManageIQ/manageiq/issues/zeitwerk

% bin/rails zeitwerk:check

https://github.com/rails/rails/blob/cbfe735c69a37446d0c5a9d448ad6d181abd1736/guides/source/classic_to_zeitwerk_howto.md

TODO list:

Phase 1: make our dependencies compatible with zeitwerk

Make core code zeitwerk compatible by enabling eager loading and fixing any problems or marking code for later fix or excluded from zeitwerk. We won't enable eager loading in the app but it helps us find problems. Note, rails engines shouldn't use zeitwerk by themselves as the rails app should handle autoload but they need to be zeitwerk friendly and pass zeitwerk:check.

High level changes:

• Move lib/manageiq-providers-amazon.rb to lib/manageiq/providers/amazon.rb to follow the rubygems suggestions: https://guides.rubygems.org/name-your-gem/. Note, bundler will try the gem-name.rb first, then try the other anyway: https://github.com/rubygems/rubygems/blob/d0d0f4eb6f34ecc34d8b0f6c26a8e1a70faf9207/bundler/lib/bundler/runtime.rb#L70-L77. Related: Clarification about usage with nested subgems fxn/zeitwerk#138
• Fix plugin generator to generate lib/manageiq/providers/amazon.rb NOT lib/manageiq-providers-amazon.rb. See: https://github.com/ManageIQ/manageiq/blob/master/lib/generators/manageiq/plugin/templates/lib/%25plugin_name%25.rb Change plugin generator to create zeitwerk compliant files #22725
• Fix invalid constants... files not defining the constants expected based on path
• etc.

Individual changes:

• Use namespaced gem require instead of hyphenated name amazon_ssa_support#96
• Normalize screaming case html yaml for zeitwerk #22489
• Zeitwerk #22488
• Use AS::Dependencies interlock with classic autoloader #22539
• Use namespaced gem require instead of hyphenated name #22548
• Pre-zeitwerk - Add inflections and acronyms #22620
• Use zeitwerk/bundler friendly namespaced gem 'require' manageiq-api#1219
• Move gem name loader to proper namespaced location. manageiq-appliance_console#208
• Move gem name loader to proper namespaced location. manageiq-automation_engine#525
• Make rails engines zeitwerk autoloader friendly  manageiq-consumption#205
• Move gem name loader to proper namespaced location. manageiq-content#729
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-decorators#88
• Add zeitwerk loader and teach it about this gem manageiq-gems-pending#551
• Zeitwerk friendly changes manageiq-gems-pending#559
• Add missing dependency on net-scp, consolidate gem loader to the namespaced location manageiq-providers-amazon#811
• Drop lib/gem-name.rb, use lib/gem/name.rb instead manageiq-providers-ansible_tower#292
• Move gem name loader to proper namespaced location. manageiq-providers-autosde#229
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-providers-awx#13
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-providers-azure#534
• Make backward compatible zeitwerk friendly changes manageiq-providers-azure_stack#93
• Move gem name loader to proper namespaced location. manageiq-providers-cisco_intersight#90
• Fix some bad constants and incorrectly named files found by zeitwerk manageiq-providers-cisco_intersight#91
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-providers-foreman#114
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-providers-google#247
• Move gem name loader to proper namespaced location. manageiq-providers-ibm_cic#32
• Move gem name loader to proper namespaced location. manageiq-providers-ibm_cloud#450
• Move gem name loader to proper namespaced location. manageiq-providers-ibm_power_hmc#135
• Move gem name loader to proper namespaced location. manageiq-providers-ibm_power_vc#84
• Move gem name loader to proper namespaced location. manageiq-providers-ibm_terraform#92
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-providers-kubernetes#483
• Move gem name loader to proper namespaced location. manageiq-providers-kubevirt#218
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-providers-lenovo#386
• Make engine zeitwerk friendly manageiq-providers-nsxt#85
• Remove lib/gem-name.rb loader in favor of namespaced file manageiq-providers-nuage#290
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-providers-openshift#245
• Fix incorrect constant name reference by referencing HostAggregate manageiq-providers-openstack#853
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-providers-openstack#848
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-providers-oracle_cloud#86
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-providers-ovirt#641
• Move gem name loader to proper namespaced location. manageiq-providers-red_hat_virtualization#27
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-providers-redfish#174
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-providers-vmware#868
• Drop lib/gem-name.rb in favor of lib/gem/name.rb manageiq-providers-workflows#15
• Remove empty, duplicated, or incorrectly split up files. manageiq-ui-classic#8730
• Add support for zeitwerk autoloader/reloader manageiq-ui-classic#8771

Phase 2: enable zeitwerk in application with eager load turned off

Make application code zeitwerk compatible by enabling eager loading and fixing any problems or marking code for later fix or excluded from zeitwerk. We won't enable eager loading in the app but it helps us find problems.

• Verify include_concern not needed - may be verified in phase one for plugins that use it.
• Verify require_nested not needed - may be verified in phase one for plugins that use it.
• Confirm that DescendantLoader is still needed. This is for STI. We need to eager load the STI descendants, otherwise queries will not work correctly.
• Verify autoload works correctly for main application and plugins
• Test code reloading in dev mode
• Verify test suite

[Fryguy]

Hoping this can kill DescendantLoader. 🙏

[jrafanie]

I was considering if it would be easier to enable zeitwerk loader in the plugins first... since they might be easier to get in "compliance" and could guide the direction for these big repositories.