Issues getting tests running

[ThisIsMissEm]

Steps to reproduce (local)

1. Clone the repo
2. bundle install
3. rake spec

Fails with a huge number of errors:

$ rake spec
[Coveralls] Set up the SimpleCov formatter.
[Coveralls] Using SimpleCov's 'rails' settings.

An error occurred while loading ./spec/controllers/application_metal_controller_spec.rb. - Did you mean?
                    rspec ./spec/controllers/applications_controller_spec.rb
                    rspec ./spec/controllers/authorizations_controller_spec.rb
                    rspec ./spec/controllers/tokens_controller_spec.rb

Failure/Error:
  class AccessGrant < ::ActiveRecord::Base
    include Doorkeeper::Orm::ActiveRecord::Mixins::AccessGrant

LoadError:
  Error loading the 'sqlite3' Active Record adapter. Missing a gem it depends on? can't activate sqlite3 (~> 1.4), already activated sqlite3-2.0.2-arm64-darwin. Make sure all dependencies are added to Gemfile.
# ./lib/doorkeeper/orm/active_record/access_grant.rb:6:in `<module:Doorkeeper>'
# ./lib/doorkeeper/orm/active_record/access_grant.rb:5:in `<top (required)>'
# ./lib/doorkeeper/config.rb:455:in `access_grant_model'
# ./lib/doorkeeper/orm/active_record.rb:40:in `initialize_configured_associations'
# ./lib/doorkeeper/orm/active_record.rb:32:in `run_hooks'
# ./lib/doorkeeper.rb:164:in `run_orm_hooks'
# ./lib/doorkeeper/engine.rb:24:in `block in <class:Engine>'
# ./spec/dummy/config/environment.rb:5:in `<top (required)>'
# ./spec/spec_helper.rb:18:in `<top (required)>'
# ./spec/spec_helper_integration.rb:4:in `<top (required)>'
# ./spec/controllers/application_metal_controller_spec.rb:3:in `<top (required)>'
# ------------------
# --- Caused by: ---
# Gem::LoadError:
#   can't activate sqlite3 (~> 1.4), already activated sqlite3-2.0.2-arm64-darwin. Make sure all dependencies are added to Gemfile.
#   ./lib/doorkeeper/orm/active_record/access_grant.rb:6:in `<module:Doorkeeper>'

[Coveralls] Set up the SimpleCov formatter.
[Coveralls] Using SimpleCov's 'rails' settings.
[omitted]/doorkeeper-gem/doorkeeper/spec/spec_helper.rb:16: warning: already initialized constant DOORKEEPER_ORM
[omitted]/doorkeeper-gem/doorkeeper/spec/spec_helper.rb:16: warning: previous definition of DOORKEEPER_ORM was here

An error occurred while loading ./spec/routing/scoped_routes_spec.rb.
Failure/Error: Rails.application.initialize!

FrozenError:
  can't modify frozen Array: ["[omitted]/doorkeeper-gem/doorkeeper/spec/dummy/app/controllers", "[omitted]/doorkeeper-gem/doorkeeper/spec/dummy/app/helpers", "[omitted]/doorkeeper-gem/doorkeeper/spec/dummy/app/models", "[omitted]/doorkeeper-gem/doorkeeper/app/controllers", "[omitted]/doorkeeper-gem/doorkeeper/app/helpers"]
# ./spec/dummy/config/environment.rb:5:in `<top (required)>'
# ./spec/spec_helper.rb:18:in `<top (required)>'
# ./spec/routing/scoped_routes_spec.rb:3:in `<top (required)>'
No examples found.

Steps to reproduce (docker)

1. clone the repo
2. run docker build --pull -t doorkeeper:test .

Fails with:

[+] Building 5.6s (12/13)                                                                                                                  docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                       0.0s
 => => transferring dockerfile: 556B                                                                                                                       0.0s
 => [internal] load metadata for docker.io/library/ruby:2.6.5-alpine                                                                                       2.3s
 => [auth] library/ruby:pull token for registry-1.docker.io                                                                                                0.0s
 => [internal] load .dockerignore                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                            0.0s
 => [1/8] FROM docker.io/library/ruby:2.6.5-alpine@sha256:a5b974e2ebb2b72642f4de4e5562597ec0883c3bfd93e9553cee6bd395dfbf00                                 0.0s
 => => resolve docker.io/library/ruby:2.6.5-alpine@sha256:a5b974e2ebb2b72642f4de4e5562597ec0883c3bfd93e9553cee6bd395dfbf00                                 0.0s
 => [internal] load build context                                                                                                                          0.0s
 => => transferring context: 2.39MB                                                                                                                        0.0s
 => CACHED [2/8] RUN apk add --no-cache   ca-certificates   wget   openssl   bash   build-base   git   sqlite-dev   tzdata                                 0.0s
 => CACHED [3/8] RUN gem install bundler -v 2.1.4 -i /usr/local/lib/ruby/gems/$(ls /usr/local/lib/ruby/gems) --force                                       0.0s
 => CACHED [4/8] WORKDIR /srv                                                                                                                              0.0s
 => CACHED [5/8] COPY Gemfile doorkeeper.gemspec /srv/                                                                                                     0.0s
 => CACHED [6/8] COPY lib/doorkeeper/version.rb /srv/lib/doorkeeper/version.rb                                                                             0.0s
 => ERROR [7/8] RUN bundle install                                                                                                                         3.2s
------                                                                                                                                                          
 > [7/8] RUN bundle install:                                                                                                                                    
0.227 Your Gemfile lists the gem timecop (>= 0) more than once.                                                                                                 
0.227 You should probably keep only one of them.                                                                                                                
0.227 Remove any duplicate entries and specify the gem only once.                                                                                               
0.227 While it's not a problem now, it could cause errors if you change the version of one of them later.                                                       
0.247 The dependency activerecord-jdbcsqlite3-adapter (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for ruby but the dependency is only for java. To add those platforms to the bundle, run `bundle lock --add-platform java`.
0.247 The dependency tzinfo-data (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for ruby but the dependency is only for x86-mingw32, x86-mswin32, x64-mingw32. To add those platforms to the bundle, run `bundle lock --add-platform x86-mingw32 x86-mswin32 x64-mingw32`.
1.407 Fetching gem metadata from https://rubygems.org/..........
2.985 Fetching gem metadata from https://rubygems.org/.
3.010 Resolving dependencies...
3.021 Bundler found conflicting requirements for the Ruby version:
3.021   In Gemfile:
3.021     Ruby
3.021 
3.021     doorkeeper was resolved to 5.7.1, which depends on
3.021       Ruby (>= 2.7)
3.021 
3.021 Ruby (>= 2.7), which is required by gem 'doorkeeper', is not available in the
3.021 local ruby installation
------

 4 warnings found (use --debug to expand):
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 13)
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 14)
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 15)
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 17)
Dockerfile:25
--------------------
  23 |     COPY lib/doorkeeper/version.rb /srv/lib/doorkeeper/version.rb
  24 |     
  25 | >>> RUN bundle install
  26 |     
  27 |     COPY . /srv/
--------------------
ERROR: failed to solve: process "/bin/sh -c bundle install" did not complete successfully: exit code: 6

It seems the Dockerfile is still using ruby 2.6, when the project dropped 2.6 support 19 months ago in #1622

I managed to "fix" the dockerfile by upgrading ruby and bundle to 3.3.4 and 2.5.11 respectively, and fixed the ENV warnings:

ENV LANG=en_US.UTF-8
ENV LANGUAGE=en_US:en
ENV LC_ALL=en_US.UTF-8

ENV BUNDLER_VERSION=2.5.11

However, upon running docker run -it --rm doorkeeper:test the tests once again failed:

[Coveralls] Set up the SimpleCov formatter.
[Coveralls] Using SimpleCov's 'rails' settings.
Error loading 'sprockets/railtie' (cannot load such file -- nokogiri/nokogiri)

An error occurred while loading ./spec/controllers/application_metal_controller_spec.rb. - Did you mean?
                    rspec ./spec/controllers/applications_controller_spec.rb
                    rspec ./spec/controllers/authorizations_controller_spec.rb
                    rspec ./spec/controllers/tokens_controller_spec.rb

Failure/Error: Bundler.require(*Rails.groups)

LoadError:
  cannot load such file -- nokogiri/nokogiri
# /usr/local/bundle/gems/zeitwerk-2.6.16/lib/zeitwerk/kernel.rb:34:in `require'
# /usr/local/bundle/gems/nokogiri-1.16.6-aarch64-linux/lib/nokogiri/extension.rb:31:in `rescue in <top (required)>'
# /usr/local/bundle/gems/nokogiri-1.16.6-aarch64-linux/lib/nokogiri/extension.rb:4:in `<top (required)>'
# /usr/local/bundle/gems/nokogiri-1.16.6-aarch64-linux/lib/nokogiri.rb:8:in `require_relative'
# /usr/local/bundle/gems/nokogiri-1.16.6-aarch64-linux/lib/nokogiri.rb:8:in `<top (required)>'
# /usr/local/bundle/gems/zeitwerk-2.6.16/lib/zeitwerk/kernel.rb:34:in `require'
# /usr/local/bundle/gems/loofah-2.22.0/lib/loofah.rb:3:in `<top (required)>'
# /usr/local/bundle/gems/zeitwerk-2.6.16/lib/zeitwerk/kernel.rb:34:in `require'
# /usr/local/bundle/gems/rails-html-sanitizer-1.6.0/lib/rails-html-sanitizer.rb:5:in `<top (required)>'
# /usr/local/bundle/gems/zeitwerk-2.6.16/lib/zeitwerk/kernel.rb:34:in `require'
# /usr/local/bundle/gems/actionview-7.1.3.4/lib/action_view/helpers/sanitize_helper.rb:3:in `<top (required)>'
# /usr/local/bundle/gems/zeitwerk-2.6.16/lib/zeitwerk/kernel.rb:34:in `require'
# /usr/local/bundle/gems/actionview-7.1.3.4/lib/action_view/helpers.rb:8:in `<top (required)>'
# /usr/local/bundle/gems/zeitwerk-2.6.16/lib/zeitwerk/kernel.rb:34:in `require'
# /usr/local/bundle/gems/sprockets-rails-3.5.1/lib/sprockets/rails/context.rb:1:in `<top (required)>'
# /usr/local/bundle/gems/zeitwerk-2.6.16/lib/zeitwerk/kernel.rb:34:in `require'
# /usr/local/bundle/gems/sprockets-rails-3.5.1/lib/sprockets/railtie.rb:11:in `<top (required)>'
# /usr/local/bundle/gems/zeitwerk-2.6.16/lib/zeitwerk/kernel.rb:34:in `require'
# /usr/local/bundle/gems/sprockets-rails-3.5.1/lib/sprockets/rails.rb:3:in `<top (required)>'
# /usr/local/bundle/gems/zeitwerk-2.6.16/lib/zeitwerk/kernel.rb:34:in `require'
# ./spec/dummy/config/application.rb:18:in `<top (required)>'
# ./spec/dummy/config/environment.rb:2:in `<top (required)>'
# ./spec/spec_helper.rb:18:in `<top (required)>'
# ./spec/spec_helper_integration.rb:4:in `<top (required)>'
# ./spec/controllers/application_metal_controller_spec.rb:3:in `<top (required)>'
# ------------------
# --- Caused by: ---
# LoadError:
#   cannot load such file -- sprockets-rails
#   /usr/local/bundle/gems/zeitwerk-2.6.16/lib/zeitwerk/kernel.rb:34:in `require'
# [...]
[Coveralls] Set up the SimpleCov formatter.
[Coveralls] Using SimpleCov's 'rails' settings.
/srv/spec/spec_helper.rb:16: warning: already initialized constant DOORKEEPER_ORM
/srv/spec/spec_helper.rb:16: warning: previous definition of DOORKEEPER_ORM was here
Error loading 'sprockets/railtie' (cannot load such file -- nokogiri/nokogiri)

An error occurred while loading ./spec/controllers/authorizations_controller_spec.rb. - Did you mean?

Other Notes

I also tried running the specs like they are run from GitHub Actions with:

$ rbenv local 3.2.3
$ CI=true BUNDLE_GEMFILE=gemfiles/rails_7_1.gemfile bundle install 
A gemspec development dependency (rspec-rails, >= 0) is being overridden by a Gemfile dependency (rspec-rails, ~> 5.0).
This behaviour may change in the future. Please remove either of them, or make sure they both have the same requirement
Bundle complete! 25 Gemfile dependencies, 114 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
$ CI=true BUNDLE_GEMFILE=gemfiles/rails_7_1.gemfile bundle exec rake spec

This still failed with (wrapped for readability):

Randomized with seed 8188
.................FF.....................................................................................................
.............................................................................................F.FFFFFF..FFF..............
...........FFFFFFFFFFFFFF...............................................................................................
..............................................................................FF........................................
...............................FFFFFFF...............F..................................................................
.................F........................FFFFFFF................................................F......................
.......................................................................FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF........FFFFF
FFFF..................................FFFFFFFFFFF.......................................................................
..................FFFFFFFFFFFF....................FF...FFFFF..................FFFFFFFFFFFF..............................
..FF.FFFFFFFFFFFFFFF.FFF....F...........FFF.........................................................................

The errors all seem to be related to Failure/Error: JSON.parse(request_response.body) where request_response.body is a HTML string:

[ThisIsMissEm]

Looks like the failures for JSON.parse are due to: https://guides.rubyonrails.org/configuring.html#actiondispatch-hostauthorization

So we likely need something like Rails.application.config.hosts = [ 'www.example.com' ], as all the tests are using that, and the HTML response reads: Blocked hosts: www.example.com

[ThisIsMissEm]

With this diff I was able to significantly reduce the failures down to 1196 examples, 6 failures:

diff --git a/spec/dummy/config/environments/development.rb b/spec/dummy/config/environments/development.rb
index 0702a8b1..87d65102 100644
--- a/spec/dummy/config/environments/development.rb
+++ b/spec/dummy/config/environments/development.rb
@@ -28,4 +28,6 @@ Dummy::Application.configure do
   config.assets.debug = true
 
   config.eager_load = false
+
+  config.hosts << 'www.example.com'
 end
diff --git a/spec/dummy/config/environments/production.rb b/spec/dummy/config/environments/production.rb
index 5365afb9..eef5028b 100644
--- a/spec/dummy/config/environments/production.rb
+++ b/spec/dummy/config/environments/production.rb
@@ -61,4 +61,6 @@ Dummy::Application.configure do
   config.active_support.deprecation = :notify
 
   config.eager_load = true
+
+  config.hosts << 'www.example.com'
 end
diff --git a/spec/dummy/config/environments/test.rb b/spec/dummy/config/environments/test.rb
index b184dff9..915f5f3a 100644
--- a/spec/dummy/config/environments/test.rb
+++ b/spec/dummy/config/environments/test.rb
@@ -42,4 +42,6 @@ Dummy::Application.configure do
   config.active_support.deprecation = :stderr
 
   config.eager_load = true
+
+  config.hosts << 'www.example.com'
 end

Failures:

  1) Authorization endpoint when forgery protection enabled raises exception on forged requests
     Failure/Error:
       expect do
         page.driver.post authorization_endpoint_url(
           client_id: @client.uid,
           redirect_uri: @client.redirect_uri,
           response_type: "code",
         )
       end.to raise_error(ActionController::InvalidAuthenticityToken)
     
       expected ActionController::InvalidAuthenticityToken but nothing was raised
     # ./spec/requests/endpoints/authorization_spec.rb:87:in `block (4 levels) in <top (required)>'
     # ./spec/support/helpers/authorization_request_helper.rb:37:in `allowing_forgery_protection'
     # ./spec/requests/endpoints/authorization_spec.rb:86:in `block (3 levels) in <top (required)>'

  2) Doorkeeper::RedirectUriValidator when force secured uri configured invalidates the uri when the uri does not use a secure protocol
     Failure/Error: expect(client).not_to be_valid
       expected #<Doorkeeper::Application id: 1, name: "Application 511", uid: "GDXul7I7lBLhOJ5fsegc8RcECRt2-sjudYcA8i4EOgU", secret: "RCxVU8dlrVU9CRSN24qLyaw2GtNf4du6Rtehp9jc4vg", redirect_uri: "http://example.com/callback", scopes: "", created_at: "2024-07-27 17:01:37.366199000 +0000", updated_at: "2024-07-27 17:01:37.366199000 +0000", owner_id: nil, owner_type: nil, confidential: true> not to be valid
     # ./spec/doorkeeper/redirect_uri_validator_spec.rb:150:in `block (3 levels) in <top (required)>'

  3) Implicit Grant Flow (request spec) when reuse_access_token enabled returns a new token each request
     Failure/Error: expect(response.location).not_to include(token.token)
       expected nil not to include "UfikQJ3aYJSB_nucFqaf_LyfxE66BrhCO8dh-FdYHrU", but it does not respond to `include?`
     # ./spec/requests/flows/implicit_grant_spec.rb:71:in `block (3 levels) in <top (required)>'

  4) Implicit Grant Flow (request spec) when reuse_access_token enabled returns the same token if it is still accessible
     Failure/Error: expect(response.location).to include(token.token)
       expected nil to include "y2bw-a0KT5YfInpdjekBZtbZbHKLycDZkaNBNe3ob8Y", but it does not respond to `include?`
     # ./spec/requests/flows/implicit_grant_spec.rb:88:in `block (3 levels) in <top (required)>'

  5) Doorkeeper::Config force_ssl_in_redirect_uri is true by default in non-development environments
     Failure/Error: expect(config.force_ssl_in_redirect_uri).to eq(true)
     
       expected: true
            got: false
     
       (compared using ==)
     
       Diff:
       @@ -1 +1 @@
       -true
       +false
       
     # ./spec/lib/config_spec.rb:300:in `block (3 levels) in <top (required)>'

  6) Authorization Code Flow silently authorizes if active matching token exists
     Failure/Error: expect(request_response.status.to_i).to eq(status)
     
       expected: 200
            got: 403
     
       (compared using ==)
     # ./spec/support/helpers/request_spec_helper.rb:90:in `response_status_should_be'
     # ./spec/requests/flows/authorization_code_spec.rb:202:in `block (2 levels) in <top (required)>'

But I'm not sure what these failures are

[ransombriggs]

I workaround the sqlite issue by tweaking the following when I run specific specs locally fwiw

diff --git a/Gemfile b/Gemfile
index f2cdb76b..bf59f529 100644
--- a/Gemfile
+++ b/Gemfile
@@ -23,7 +23,7 @@ gem "rubocop-rspec", require: false
 gem "bcrypt", "~> 3.1", require: false
 
 gem "activerecord-jdbcsqlite3-adapter", platform: :jruby
-gem "sqlite3", "~> 2.0", platform: %i[ruby mswin mingw x64_mingw]
+gem "sqlite3", "~> 1.7", platform: %i[ruby mswin mingw x64_mingw]
 
 gem "tzinfo-data", platforms: %i[mingw mswin x64_mingw]
 gem "timecop"

[ThisIsMissEm]

Whilst docker-based development experience isn't fantastic (rebuilds are quiet slow) the changes in #1732 do at least unblock me, and if those were merged, then we could consider this closed.

[nbulaj]

I believe we fixed this, right? Or do we still have a problem?