Reduce memory impact

[pmackay]

I notice from derailed-benchmarks that this takes a chunk of memory - is there any way that the gem could be configured to only load the runtime required parts in production?

  packs-rails: 9.875 MiB
    packs: 9.8438 MiB
      packs-specification: 2.8125 MiB (Also required by: code_ownership)
        sorbet-runtime: 2.1875 MiB (Also required by: parse_packwerk, code_teams, and 3 others)
          /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/sorbet-runtime-0.5.11114/lib/types/props/pretty_printable: 0.3906 MiB
            pp: 0.3594 MiB (Also required by: faraday/logging/formatter, pry/color_printer)
          /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/sorbet-runtime-0.5.11114/lib/types/props/decorator: 0.3594 MiB
        packs/pack: 0.4844 MiB
      code_ownership: 2.6719 MiB
        code_ownership/private: 1.1719 MiB
        code_ownership/cli: 0.6563 MiB
          optparse: 0.6094 MiB (Also required by: i18n-js, /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/yard-0.9.36/lib/yard/cli/yardopts_command)
        code_ownership/configuration: 0.625 MiB
      packs/private: 2.375 MiB
        packs/private/interactive_cli: 2.0938 MiB
          tty-prompt: 1.5313 MiB
            /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/tty-prompt-0.23.1/lib/tty/prompt: 1.5156 MiB
              tty-reader: 0.6406 MiB
                /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/tty-reader-0.9.0/lib/tty/reader: 0.6406 MiB
      parse_packwerk: 0.7344 MiB
        parse_packwerk/package: 0.3281 MiB
      code_teams: 0.6406 MiB (Also required by: code_ownership)
        code_teams/plugins/identity: 0.3906 MiB
      packs/cli: 0.3594 MiB

[technicalpickles]

Thanks for sharing! I haven't tried running derailed-benchmarks in awhile, but I'm curious what the packs-rails memory used is relative to the rest of what the benchmark is returning? Just wanted to get a sense of scale.

Could you elaborate on what you mean by this?

only load the runtime required parts in production

As in, don't require (load) it in development, but do load it in production? If that is the case, you might be able to do that via the Gemfile, ie require: false, and then require it yourself in the environments you want.

[pmackay]

@technicalpickles here's a larger snippet of some of the largest memory footprint gems:

TOP: 86.0 MiB
  rails/all: 24.6563 MiB
    rails: 15.6875 MiB (Also required by: active_record/railtie, active_model/railtie, and 10 others)
      rails/application: 14.1563 MiB (Also required by: packs-rails)
        active_support/encrypted_configuration: 7.125 MiB
          active_support/encrypted_file: 7.0625 MiB
            active_support/message_encryptor: 6.25 MiB (Also required by: rails/secrets)
              active_support/messages/codec: 6.0625 MiB (Also required by: active_support/message_verifier)
                /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.3.4/lib/active_support/messages/metadata: 6.0156 MiB
                  active_support/json: 5.9688 MiB (Also required by: action_controller/responder, active_model/serializers/json)
                    active_support/json/encoding: 5.7188 MiB
                      active_support/core_ext/object/json: 5.6875 MiB (Also required by: active_support/core_ext/object, active_record/token_for)
                        active_support/core_ext/time/conversions: 5.0 MiB (Also required by: active_support/core_ext/date_time/conversions, active_support/core_ext/time/calculations)
                          active_support/values/time_zone: 5.0 MiB (Also required by: active_support/core_ext/date_time/conversions, active_support/time_with_zone)
                            tzinfo: 4.9063 MiB
                              /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/tzinfo-2.0.6/lib/tzinfo/string_deduper: 2.9219 MiB
                                concurrent: 2.9063 MiB (Also required by: /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/tzinfo-2.0.6/lib/tzinfo/data_source, sprockets/manifest, and 2 others)
                                  concurrent/configuration: 0.6406 MiB (Also required by: concurrent/scheduled_task, concurrent/options, and 3 others)
                                    concurrent/delay: 0.5469 MiB (Also required by: concurrent/utility/processor_counter, concurrent)
                                      concurrent/concern/obligation: 0.3594 MiB (Also required by: concurrent/ivar)
                                  concurrent/promises: 0.4688 MiB
                                  concurrent/synchronization: 0.4688 MiB
                                    concurrent/synchronization/condition: 0.4531 MiB
                                  concurrent/executors: 0.375 MiB
                                    concurrent/executor/timer_set: 0.375 MiB
                                      concurrent/scheduled_task: 0.3438 MiB (Also required by: concurrent, concurrent/timer_task)
                                        concurrent/ivar: 0.3438 MiB (Also required by: concurrent/async, concurrent/future, and 2 others)
                                          concurrent/concern/observable: 0.3125 MiB (Also required by: concurrent/agent, concurrent/atom)
                                            concurrent/collection/copy_on_notify_observer_set: 0.3125 MiB (Also required by: concurrent/scheduled_task, concurrent/atom)
                              /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/tzinfo-2.0.6/lib/tzinfo/data_sources/zoneinfo_data_source: 0.3125 MiB
                        uri/generic: 0.375 MiB (Also required by: global_id/uri/gid)
            tempfile: 0.7344 MiB (Also required by: rails/secrets, rack/utils, and 3 others)
              tmpdir: 0.6563 MiB (Also required by: packs/private, appsignal/config)
                fileutils: 0.5469 MiB (Also required by: rack/utils, /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rack-test-2.1.0/lib/rack/test/uploaded_file, and 9 others)
        active_support/key_generator: 4.4844 MiB
          openssl: 4.4844 MiB (Also required by: active_support/message_encryptor, active_support/message_verifier, and 8 others)
            openssl.so: 2.6094 MiB
            /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/3.3.0/openssl/ssl: 1.7344 MiB
              ipaddr: 0.3438 MiB (Also required by: active_support/core_ext/object/json, rails/application/configuration)
        rails/engine: 0.9063 MiB (Also required by: turbo/engine, /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rails_event_store-2.12.1/lib/rails_event_store/browser)
          rails/railtie: 0.3281 MiB (Also required by: global_id/railtie, sprockets/railtie, and 9 others)
        yaml: 0.8594 MiB (Also required by: active_support/encrypted_configuration, rails/secrets, and 19 others)
          psych: 0.8594 MiB
        active_support/deprecation: 0.4375 MiB (Also required by: active_support/rails, active_model/serializers/xml)
      active_support: 1.3594 MiB (Also required by: active_support/railtie, active_support/i18n_railtie, and 20 others)
        active_support/dependencies/autoload: 1.125 MiB (Also required by: view_component, pundit)
          active_support/inflector/methods: 1.1094 MiB (Also required by: active_support/core_ext/time/conversions, active_support/core_ext/date_time/conversions, and 6 others)
            active_support/inflections: 1.0625 MiB (Also required by: active_support/inflector)
              active_support/inflector/inflections: 0.9063 MiB (Also required by: active_support/inflector)
                active_support/i18n: 0.6875 MiB (Also required by: active_support/inflector/transliterate, abstract_controller, and 2 others)
                  i18n: 0.5313 MiB (Also required by: ransack/translate, i18n-js, and 2 others)
    active_record/railtie: 4.1094 MiB (Also required by: active_storage/engine, /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/avo-3.8.1/lib/avo/engine)
      active_record: 2.9688 MiB (Also required by: active_storage, orm_adapter/adapters/active_record, and 7 others)
        arel: 1.0 MiB
          arel/visitors: 0.4844 MiB
            arel/visitors/to_sql: 0.3438 MiB
          arel/nodes: 0.4375 MiB
        active_record/migration: 0.7031 MiB (Also required by: /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/mobility-1.2.9/lib/rails/generators/mobility/active_record_migration_compatibility)
        active_record/errors: 0.6094 MiB
          active_model/errors: 0.4063 MiB
      action_controller/railtie: 1.0313 MiB (Also required by: active_storage/engine, rails/all)
        action_view/railtie: 0.4844 MiB (Also required by: rails/all, /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/avo-3.8.1/lib/avo/engine)
          action_view: 0.3906 MiB (Also required by: sprockets/rails/helper, draper, and 4 others)
            active_support/core_ext/string/output_safety: 0.3438 MiB (Also required by: action_view/helpers/capture_helper, action_view/helpers/output_safety_helper, and 7 others)
        action_controller: 0.4531 MiB (Also required by: responders, gon)
          action_controller/metal/strong_parameters: 0.3906 MiB
    active_storage/engine: 3.5781 MiB
      active_storage: 2.1094 MiB
        marcel: 2.0469 MiB
          marcel/magic: 1.9688 MiB
            marcel/tables: 1.9531 MiB
      action_dispatch/routing/route_set: 0.5313 MiB
        action_dispatch/journey: 0.3906 MiB
          action_dispatch/journey/router: 0.3125 MiB
      active_job/railtie: 0.4844 MiB (Also required by: action_mailer/railtie, rails/all)
        global_id/railtie: 0.4531 MiB
          active_support/core_ext/integer/time: 0.3125 MiB (Also required by: active_support/time, /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.3.4/lib/active_support/core_ext/integer.rb)
    rails/test_unit/railtie: 0.9063 MiB
      rails/test_unit/line_filtering: 0.8906 MiB
        rails/test_unit/runner: 0.875 MiB
          rails/test_unit/test_parser: 0.4531 MiB
            ripper: 0.4531 MiB (Also required by: rails/source_annotation_extractor)
          rake/file_list: 0.3906 MiB
    action_cable/engine: 0.3281 MiB
  packs-rails: 7.4844 MiB
    packs: 7.4844 MiB
      packs/private: 2.4688 MiB
        packs/private/interactive_cli: 2.0 MiB
          tty-prompt: 1.6406 MiB
            /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/tty-prompt-0.23.1/lib/tty/prompt: 1.6406 MiB
              tty-reader: 0.5781 MiB
                /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/tty-reader-0.9.0/lib/tty/reader: 0.5781 MiB
      packs-specification: 2.2031 MiB (Also required by: code_ownership)
        sorbet-runtime: 1.9531 MiB (Also required by: parse_packwerk, code_teams, and 3 others)
      code_ownership: 1.4531 MiB
        code_ownership/cli: 0.7813 MiB
          optparse: 0.7656 MiB (Also required by: i18n-js)
        code_ownership/configuration: 0.3906 MiB
      code_teams: 0.4375 MiB (Also required by: code_ownership)
        code_teams/plugins/identity: 0.3281 MiB
      packs/cli: 0.4063 MiB
  rails_event_store: 7.4219 MiB
    rails_event_store_active_record: 6.5313 MiB
      ruby_event_store/active_record: 6.5156 MiB
        /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/ruby_event_store-active_record-2.12.1/lib/ruby_event_store/active_record/event: 5.9063 MiB
          active_record/base: 4.7656 MiB
            active_record/type_caster: 1.2813 MiB
            active_record/enum: 0.4844 MiB
              active_record/type: 0.4063 MiB
            active_record/querying: 0.4219 MiB
            active_record/validations: 0.3281 MiB
          active_record/relation: 0.6094 MiB (Also required by: active_record_extended)
            active_record/relation/query_methods: 0.3281 MiB (Also required by: active_record_extended)
          active_record/associations/collection_proxy: 0.4219 MiB
        /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/ruby_event_store-active_record-2.12.1/lib/ruby_event_store/active_record/generators/rails_migration_generator: 0.3438 MiB
    /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rails_event_store-2.12.1/lib/rails_event_store/all: 0.8906 MiB
      ruby_event_store: 0.625 MiB
  sprockets/rails: 6.6875 MiB
    sprockets/railtie: 6.6875 MiB
      sprockets/rails/context: 4.75 MiB
        action_view/helpers: 4.7344 MiB (Also required by: inline_svg/action_view/helpers)
          action_view/helpers/sanitize_helper: 5.2188 MiB (Also required by: action_view/helpers/text_helper)
            rails-html-sanitizer: 5.2188 MiB
              loofah: 4.9844 MiB
                nokogiri: 3.7969 MiB (Also required by: inline_svg/transform_pipeline, inline_svg)
                  /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/nokogiri-1.16.5-arm64-darwin/lib/nokogiri/xml: 1.2813 MiB
                    /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/nokogiri-1.16.5-arm64-darwin/lib/nokogiri/xml/node: 0.3281 MiB (Also required by: /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/nokogiri-1.16.5-arm64-darwin/lib/nokogiri/html5/node)
                  /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/nokogiri-1.16.5-arm64-darwin/lib/nokogiri/html4: 0.9844 MiB (Also required by: /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/nokogiri-1.16.5-arm64-darwin/lib/nokogiri/html)
                    /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/nokogiri-1.16.5-arm64-darwin/lib/nokogiri/html4/element_description_defaults: 0.8281 MiB
                  /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/nokogiri-1.16.5-arm64-darwin/lib/nokogiri/css: 0.5313 MiB
                  /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/nokogiri-1.16.5-arm64-darwin/lib/nokogiri/extension: 0.4688 MiB (Also required by: /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/nokogiri-1.16.5-arm64-darwin/lib/nokogiri/version/info)
                    /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/nokogiri-1.16.5-arm64-darwin/lib/nokogiri/3.3/nokogiri: 0.4688 MiB
                /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/loofah-2.22.0/lib/loofah/html5/scrub: 0.4375 MiB
                  crass: 0.375 MiB
                    /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/crass-1.0.6/lib/crass/parser: 0.375 MiB
                /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/loofah-2.22.0/lib/loofah/html5/safelist: 0.3906 MiB
      sprockets: 1.5 MiB (Also required by: sprockets/rails/context, sprockets/rails/helper)
        sprockets/environment: 0.5 MiB
          sprockets/base: 0.5 MiB (Also required by: sprockets/cached_environment)
            sprockets/configuration: 0.3281 MiB (Also required by: sprockets)
  avo: 6.4219 MiB (Also required by: avo/advanced, avo/pro, and 3 others)
    /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/avo-3.8.1/lib/avo/engine: 2.75 MiB
      httparty: 0.7813 MiB
      active_link_to: 0.7656 MiB
        addressable/uri: 0.7656 MiB (Also required by: addressable, addressable/template)
          addressable/idna: 0.3281 MiB
            addressable/idna/pure: 0.3281 MiB
      pagy: 0.3906 MiB
        /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/pagy-8.4.4/lib/pagy/frontend: 0.3594 MiB
          /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/pagy-8.4.4/lib/pagy/i18n: 0.3594 MiB
    /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/avo-3.8.1/lib/avo/base_resource.rb: 0.3906 MiB
  avo/advanced: 4.125 MiB
    avo/pro: 2.9375 MiB
      avo/menu: 1.5938 MiB
      avo/dashboards: 0.9531 MiB
    avo/dynamic_filters: 0.7031 MiB
  dry-struct: 3.8906 MiB
    dry/struct: 3.8906 MiB
      dry/types: 3.2344 MiB
        dry/types/core: 2.1094 MiB
          /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/dry-types-1.7.1/lib/dry/types/nominal.rb: 0.3125 MiB
        dry/types/printer: 0.6094 MiB (Also required by: dry/struct/printer)
  dry-rails: 3.5156 MiB
    dry/rails: 3.5 MiB
      dry/rails/container: 3.1094 MiB
        /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/dry-system-1.0.1/lib/dry/system/container.rb: 2.9688 MiB
          /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/dry-core-1.0.1/lib/dry/core/container.rb: 1.75 MiB
            /Users/paulmackay/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/dry-configurable-1.1.0/lib/dry/configurable/config.rb: 1.3906 MiB

To expand on this:

only load the runtime required parts in production

what i mean is, what is the essential functionality that must be loaded in production, to make packs work (loading files from directories, etc) and what could be loaded only when needed in development, e.g. cli, code_ownership, sorbet (if we're not using that), etc.