Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use .knapsack_pro directory for temporary files instead of the tmp directory in the user's project directory #155

Merged
merged 49 commits into from
Aug 17, 2021
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
a709b09
Raise error when prerun tests report does not exist on the disk. This…
ArturT Aug 6, 2021
4f43966
Create KnapsackPro::Config::TempFiles and add spec
ArturT Aug 6, 2021
e2b4f1d
remove tmp dir only if it exists
ArturT Aug 6, 2021
1daee2c
use temp directory path in tracker
ArturT Aug 6, 2021
0965a3d
use new temp directory in report
ArturT Aug 6, 2021
988365f
use new temp directory in slow test file determiner
ArturT Aug 6, 2021
65ce2c0
use temp directory in base adapter
ArturT Aug 6, 2021
fc8ccb8
use new temp directory in RSpecTestExampleDetector
ArturT Aug 6, 2021
5f111a9
remove old tmp dir const
ArturT Aug 6, 2021
255cbad
use new temp dir path in message
ArturT Aug 6, 2021
fcbdeb9
update message
ArturT Aug 6, 2021
2f350ce
update warning message
ArturT Aug 6, 2021
98ef84a
clean only .knapsack_pro temp directory
ArturT Aug 6, 2021
2ef9649
Update spec_helper.rb
ArturT Aug 6, 2021
6217728
Update CHANGELOG.md
ArturT Aug 6, 2021
97af123
use .knapsack_pro gem relative path
ArturT Aug 6, 2021
8aeb0d1
Update report_spec.rb
ArturT Aug 6, 2021
2864bc1
Update base_adapter_spec.rb
ArturT Aug 6, 2021
3997b94
updater warning message. Remove outdate message.
ArturT Aug 6, 2021
46c65be
When you use knapsack_pro Regular Mode, the reason for no tests execu…
ArturT Aug 6, 2021
4a060b0
remove new line
ArturT Aug 6, 2021
2449713
Fix randomly failing spec
ArturT Aug 9, 2021
28b1c62
mock temp_directory_path method
ArturT Aug 9, 2021
1636994
Update base_adapter_spec.rb
ArturT Aug 9, 2021
eb9ab70
simplify spec
ArturT Aug 9, 2021
5b993f0
extract KNAPSACK_PRO_TMP_DIR
ArturT Aug 9, 2021
4ecff24
Update CHANGELOG.md
ArturT Aug 13, 2021
880f263
typo: update text
ArturT Aug 13, 2021
427abda
add storing word
ArturT Aug 13, 2021
a7073c2
Use File.exist? instead of deprecated File.exists?
ArturT Aug 13, 2021
56df05c
update message
ArturT Aug 13, 2021
5f12f1b
use more general name .knapsack_pro and fix some typos
ArturT Aug 13, 2021
c81ca2e
text: you've added
ArturT Aug 13, 2021
390a1d3
add potential word
ArturT Aug 13, 2021
f69065d
add the
ArturT Aug 13, 2021
6a5036d
add link to support team
ArturT Aug 13, 2021
5d6e5b9
verify asterisk is present in gitignore file
ArturT Aug 17, 2021
e21f0a3
explicitly call ensure_temp_directory_exists! when needed
ArturT Aug 17, 2021
7756183
Update report_spec.rb
ArturT Aug 17, 2021
b7c3b15
Update base_adapter_spec.rb
ArturT Aug 17, 2021
8b8cf7b
Update rspec_test_example_detector_spec.rb
ArturT Aug 17, 2021
c96ab7d
Update rspec_test_example_detector_spec.rb
ArturT Aug 17, 2021
217639f
ignore .knapsack_pro directory so we can mock ensure_temp_directory_e…
ArturT Aug 17, 2021
2afdc5c
Use ruby 3.0.2 on CI
ArturT Aug 17, 2021
0a811c0
add ! to method names
ArturT Aug 17, 2021
eca0920
ensure line with asterisk
ArturT Aug 17, 2021
a58724b
update text
ArturT Aug 17, 2021
5b38f78
use expect to change
ArturT Aug 17, 2021
16dc997
Update temp_files_spec.rb
ArturT Aug 17, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Change Log

### 3.1.0

* Use `.knapsack_pro` directory for temporary files instead of the `tmp` folder in the user project folder
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should read user's project. I would also change "folder" to "directory" (despite the duplication).


https://github.com/KnapsackPro/knapsack_pro-ruby/pull/155

https://github.com/KnapsackPro/knapsack_pro-ruby/compare/v3.0.0...v3.1.0

### 3.0.0

* __(breaking change)__ Remove support for RSpec 2.x. This change was already done by accident in [the pull request](https://github.com/KnapsackPro/knapsack_pro-ruby/pull/143) when we added the RSpec `context` hook, which is available only since RSpec 3.x.
Expand Down
1 change: 1 addition & 0 deletions lib/knapsack_pro.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
require_relative 'knapsack_pro/config/ci/heroku'
require_relative 'knapsack_pro/config/env'
require_relative 'knapsack_pro/config/env_generator'
require_relative 'knapsack_pro/config/temp_files'
require_relative 'knapsack_pro/logger_wrapper'
require_relative 'knapsack_pro/client/api/action'
require_relative 'knapsack_pro/client/api/v1/base'
Expand Down
6 changes: 3 additions & 3 deletions lib/knapsack_pro/adapters/base_adapter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ class BaseAdapter

def self.adapter_bind_method_called_file
adapter_name = self.to_s.gsub('::', '-')
"#{KnapsackPro::Config::Env::TMP_DIR}/#{adapter_name}-bind_method_called_for_node_#{KnapsackPro::Config::Env.ci_node_index}.txt"
"#{KnapsackPro::Config::TempFiles.temp_directory_path}/#{adapter_name}-bind_method_called_for_node_#{KnapsackPro::Config::Env.ci_node_index}.txt"
end

def self.slow_test_file?(adapter_class, test_file_path)
Expand Down Expand Up @@ -39,14 +39,14 @@ def self.verify_bind_method_called
puts "\n\n"
KnapsackPro.logger.error('-'*10 + ' Configuration error ' + '-'*50)
KnapsackPro.logger.error("You forgot to call #{self}.bind method in your test runner configuration file. It is needed to record test files time execution. Please follow the installation guide to configure your project properly https://docs.knapsackpro.com/knapsack_pro-ruby/guide/")
KnapsackPro.logger.error("If you already have #{self}.bind method added and you still see this error then one of your tests must had to delete tmp/knapsack_pro directory from the disk accidentally. Please ensure you do not remove tmp/knapsack_pro directory: https://knapsackpro.com/faq/question/why-all-test-files-have-01s-time-execution-for-my-ci-build-in-user-dashboard")
KnapsackPro.logger.error("If you already have #{self}.bind method added and you still see this error then one of your tests must had to delete .knapsack_pro directory from the disk accidentally. Please ensure you do not remove .knapsack_pro directory: https://knapsackpro.com/faq/question/why-all-test-files-have-01s-time-execution-for-my-ci-build-in-user-dashboard")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty sure "must had to delete" is ungrammatical. How about "must have deleted"?

Also, I think we are missing "the" in "the .knapsack_pro directory" (2 occurrences)

Kernel.exit(1)
end
end
end

def bind
FileUtils.mkdir_p(KnapsackPro::Config::Env::TMP_DIR)
FileUtils.mkdir_p(KnapsackPro::Config::TempFiles.temp_directory_path)
File.write(self.class.adapter_bind_method_called_file, nil)

if KnapsackPro::Config::Env.recording_enabled?
Expand Down
1 change: 0 additions & 1 deletion lib/knapsack_pro/config/env.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ class Env
'info' => ::Logger::INFO,
'debug' => ::Logger::DEBUG,
}
TMP_DIR = 'tmp/knapsack_pro'

class << self
def ci_node_total
Expand Down
46 changes: 46 additions & 0 deletions lib/knapsack_pro/config/temp_files.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
module KnapsackPro
module Config
class TempFiles
def self.temp_directory_path
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered separating the ensuring the temp directory exists from returning the path? Or maybe memoizing the result or sth?

Do you think there is a way to do this check once at some point, or would it always be succeptible to errors?

The reason I'm wondering about this is that it seems that we are using this .temp_directory_path method in many places to just get the correct path, but each time we do this it would now trigger an I/O read operation. Would be great to avoid that if it's avoidable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about it and I decided that checking if file exists on the disk is good enough. We already do some IO operations in queue mode like saving reports and it shouldn't have significant impact on performance.

If I add memorization to class method then this causes random tests failures depending on the order of tests executed because class method memorization is preserved for each call of class method.

For instance I could do:

      def self.temp_directory_path
        @temp_directory_path ||=
          begin
            temp_files = new
            temp_files.ensure_temp_directory_exists
            temp_files.temp_directory_path
          end
      end

and then when I run tests I get randomly different tests failures:

[14:47:44] artur@Artur-MBP: ~/Documents/github/knapsack-pro/knapsack_pro-ruby temporary-files-directory! ruby-3.0.2
$ rspec ./spec/knapsack_pro/config/temp_files_spec.rb

Randomized with seed 42508

KnapsackPro::Config::TempFiles
  .temp_directory_path
    .gitignore file has content
    returns temporary directory path
    creates .gitignore file (FAILED - 1)

Failures:

  1) KnapsackPro::Config::TempFiles.temp_directory_path creates .gitignore file
     Failure/Error: expect(File.exist?(gitignore_file_path)).to be true

       expected true
            got false
     # ./spec/knapsack_pro/config/temp_files_spec.rb:14:in `block (3 levels) in <top (required)>'
     # /Users/artur/.rvm/gems/ruby-3.0.2/gems/webmock-3.13.0/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'

Finished in 0.02793 seconds (files took 0.53206 seconds to load)
3 examples, 1 failure

Failed examples:

rspec ./spec/knapsack_pro/config/temp_files_spec.rb:11 # KnapsackPro::Config::TempFiles.temp_directory_path creates .gitignore file

Randomized with seed 42508


[14:47:46] artur@Artur-MBP: ~/Documents/github/knapsack-pro/knapsack_pro-ruby temporary-files-directory! ruby-3.0.2
$ rspec ./spec/knapsack_pro/config/temp_files_spec.rb

Randomized with seed 56734

KnapsackPro::Config::TempFiles
  .temp_directory_path
    returns temporary directory path
    .gitignore file has content (FAILED - 1)
    creates .gitignore file (FAILED - 2)

Failures:

  1) KnapsackPro::Config::TempFiles.temp_directory_path .gitignore file has content
     Failure/Error: expect(File.read(gitignore_file_path)).to include '# This directory is used by knapsack_pro gem for storing temporary files during tests runtime'

     Errno::ENOENT:
       No such file or directory @ rb_sysopen - .knapsack_pro/.gitignore
     # ./spec/knapsack_pro/config/temp_files_spec.rb:19:in `read'
     # ./spec/knapsack_pro/config/temp_files_spec.rb:19:in `block (3 levels) in <top (required)>'
     # /Users/artur/.rvm/gems/ruby-3.0.2/gems/webmock-3.13.0/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'

  2) KnapsackPro::Config::TempFiles.temp_directory_path creates .gitignore file
     Failure/Error: expect(File.exist?(gitignore_file_path)).to be true

       expected true
            got false
     # ./spec/knapsack_pro/config/temp_files_spec.rb:14:in `block (3 levels) in <top (required)>'
     # /Users/artur/.rvm/gems/ruby-3.0.2/gems/webmock-3.13.0/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'

Finished in 0.01396 seconds (files took 0.39825 seconds to load)
3 examples, 2 failures

Failed examples:

rspec ./spec/knapsack_pro/config/temp_files_spec.rb:17 # KnapsackPro::Config::TempFiles.temp_directory_path .gitignore file has content
rspec ./spec/knapsack_pro/config/temp_files_spec.rb:11 # KnapsackPro::Config::TempFiles.temp_directory_path creates .gitignore file

Randomized with seed 56734

I could disable memorization with custom argument def self.temp_directory_path(cached: true). For the test environment I could usecache: false but I'm not sure if this is good idea to change method behaviour only to make tests pass :/

temp_files = new
temp_files.ensure_temp_directory_exists
temp_files.temp_directory_path
end

def ensure_temp_directory_exists
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to keep this method and the #temp_directory_path a part of the public interface of this class? From a quick glance, it seems that only the class method is being called from outside? 🤔 It's also the only one that's being (directly) tested in the spec.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I move code to private section:

private

      def ensure_temp_directory_exists
        unless File.exists?(gitignore_file_path)
          create_temp_directory
          create_gitignore_file
        end
      end

      # relative to the directory where you run knapsack_pro gem (user's project)
      def temp_directory_path
        '.knapsack_pro'
      end

then I get error:

$ rspec ./spec/knapsack_pro/config/temp_files_spec.rb

Randomized with seed 3043

KnapsackPro::Config::TempFiles
  .temp_directory_path
    .gitignore file has content (FAILED - 1)
    creates .gitignore file (FAILED - 2)
    returns temporary directory path (FAILED - 3)

Failures:

  1) KnapsackPro::Config::TempFiles.temp_directory_path .gitignore file has content
     Failure/Error: temp_files.ensure_temp_directory_exists

     NoMethodError:
       private method `ensure_temp_directory_exists' called for #<KnapsackPro::Config::TempFiles:0x00007f911c9ed6d0>
     # ./lib/knapsack_pro/config/temp_files.rb:8:in `temp_directory_path'
     # ./spec/knapsack_pro/config/temp_files_spec.rb:5:in `block (3 levels) in <top (required)>'
     # ./spec/knapsack_pro/config/temp_files_spec.rb:18:in `block (3 levels) in <top (required)>'
     # /Users/artur/.rvm/gems/ruby-3.0.2/gems/webmock-3.13.0/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'

When I move code to protected I get error as well:

  1) KnapsackPro::Config::TempFiles.temp_directory_path creates .gitignore file
     Failure/Error: temp_files.ensure_temp_directory_exists

     NoMethodError:
       protected method `ensure_temp_directory_exists' called for #<KnapsackPro::Config::TempFiles:0x00007fbb43a47c88>
     # ./lib/knapsack_pro/config/temp_files.rb:8:in `temp_directory_path'
     # ./spec/knapsack_pro/config/temp_files_spec.rb:5:in `block (3 levels) in <top (required)>'
     # ./spec/knapsack_pro/config/temp_files_spec.rb:13:in `block (3 levels) in <top (required)>'
     # /Users/artur/.rvm/gems/ruby-3.0.2/gems/webmock-3.13.0/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'

We call the following methods in class method temp_directory_path so they probably must be public.

 temp_files.ensure_temp_directory_exists
 temp_files.temp_directory_path

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is because we are trying to access the private method from outside the object.

If we are adamant about the public method remaining a class method, we could change all the other methods to be private class methods.

unless File.exists?(gitignore_file_path)
Copy link
Member

@shadre shadre Aug 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is deprecated in ruby 2.7.1, and doesn't exist in ruby 3.0.2. There is an #exist? method, but I'm not sure if they are effectively the same.

I was dealing with ensuring a directory exists in my past side-project. I looked up the code and noticed I was using the Dir class instead. I don't remember offhand why I chose it over File - you probably know the difference better. With Dir you can also create the directory.

In case it's helpful to you, here is how I handled a similar case back then:

create_needed_dirs!

def create_needed_dirs!
  create!('foo')
  create!('bar')
end

def create!(directory)
  return false if Dir.exist?(directory)

  Dir.mkdir(directory) && true
end

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I've used File.exist? in the whole codebase.

create_temp_directory
create_gitignore_file
end
end

# relative to the directory where you run knapsack_pro gem (user's project)
def temp_directory_path
'.knapsack_pro'
end

private

def create_temp_directory
FileUtils.mkdir_p(temp_directory_path)
end

def gitignore_file_path
File.join(temp_directory_path, '.gitignore')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly, that we'll create our own .gitignore file in our new temp directory?

Does it mean that there would now need to be the .knapsack_pro directory commited in the project (with just the .gitignore file commited)? Or is it just created during the gem's runtime and doesn't exist otherwise?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly, that we'll create our own .gitignore file in our new temp directory?

Yes.

Does it mean that there would now need to be the .knapsack_pro directory commited in the project (with just the .gitignore file committed)?

No. Because we also ignore .gitignore file with * char in .gitignore file.

.gitignore file is created during gem runtime. It does not have to be committed into repo even when you run knapsack pro locally in development because all content of .knapsack_pro is ignored so it means for git the .knapsack_pro directory is empty. It's not possible to commit an empty directory in git so this way .knapsack_pro with the .gitignore file won't be committed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I hoped it works the way you described it. Great to know it does indeed! 💚

end

def gitignore_file_content
"# This directory is used by knapsack_pro gem for temporary files during tests runtime.\n" <<
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that "for storing temporary files" sounds more proper.

"# Ignore all files, and do not commit this directory into your repository.\n" <<
"# Learn more at https://knapsackpro.com\n" <<
"*"
end

def create_gitignore_file
File.open(gitignore_file_path, 'w+') do |f|
f.write(gitignore_file_content)
end
end
end
end
end
13 changes: 6 additions & 7 deletions lib/knapsack_pro/report.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ def self.save

if test_files.empty?
KnapsackPro.logger.warn("No test files were executed on this CI node.")
KnapsackPro.logger.debug("When you use knapsack_pro regular mode then probably reason might be very narrowed tests list - you run only tests with specified tag and there are fewer test files with the tag than node total number.")
KnapsackPro.logger.debug("When you use knapsack_pro Regular Mode, the reason for no tests executing might be a very narrow tests list. Most likely, you run only tests with a specified tag, and the number of test files with the tag was lower than the number of parallel CI nodes.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvements. 👍

the last bit could be simplified to ", and there were fewer test files with the tag than parallel CI nodes."

end

create_build_subset(test_files)
Expand Down Expand Up @@ -36,8 +36,7 @@ def self.save_node_queue_to_api

if test_files.empty?
KnapsackPro.logger.warn("No test files were executed on this CI node.")
KnapsackPro.logger.debug("When you use knapsack_pro queue mode then probably reason might be that CI node was started after the test files from the queue were already executed by other CI nodes. That is why this CI node has no test files to execute.")
KnapsackPro.logger.debug("Another reason might be when your CI node failed in a way that prevented knapsack_pro to save time execution data to Knapsack Pro API and you have just tried to retry failed CI node but instead you got no test files to execute. In that case knapsack_pro don't know what tests should be executed here.")
KnapsackPro.logger.debug("This CI node likely started work late after the test files were already executed by other CI nodes consuming the queue.")
end

measured_test_files = test_files
Expand All @@ -46,9 +45,9 @@ def self.save_node_queue_to_api

if test_files.size > 0 && measured_test_files.size == 0
KnapsackPro.logger.warn("#{test_files.size} test files were executed on this CI node but the recorded time was lost due to:")
KnapsackPro.logger.warn("1. Probably you have a code (i.e. RSpec hooks) that clears tmp directory in your project. Please ensure you do not remove the content of tmp/knapsack_pro/queue/ directory between tests run.")
KnapsackPro.logger.warn("2. Another reason might be that you forgot to add Knapsack::Adapters::RSpecAdapter.bind in your rails_helper.rb or spec_helper.rb. Please follow the installation guide again: https://docs.knapsackpro.com/integration/")
KnapsackPro.logger.warn("3. All your tests are empty test files, are pending tests or have syntax error and could not be executed hence no measured time execution by knapsack_pro.")
KnapsackPro.logger.warn("1. Please ensure you do not remove the content of .knapsack_pro/queue/ directory between tests run.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"the contents"

I would add the before .knapsack_pro/queue/

Do you think it's a good idea to mention the more general, main .knapsack_pro directory here instead of the subdir?

KnapsackPro.logger.warn("2. Ensure you added Knapsack::Adapters::RSpecAdapter.bind in your rails_helper.rb or spec_helper.rb. Please follow the installation guide again: https://docs.knapsackpro.com/integration/")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should read "you've added"

KnapsackPro.logger.warn("3. Another reason for this warning is that all your tests are empty test files, pending tests, or they have syntax errors, and the time execution was not recorded for them.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say "Another potential reason" to emphasize this being a hypothesis and not a statement

end

create_build_subset(test_files)
Expand Down Expand Up @@ -80,7 +79,7 @@ def self.create_build_subset(test_files)

def self.queue_path
queue_id = KnapsackPro::Config::Env.queue_id
"#{KnapsackPro::Config::Env::TMP_DIR}/queue/#{queue_id}"
"#{KnapsackPro::Config::TempFiles.temp_directory_path}/queue/#{queue_id}"
end
end
end
11 changes: 7 additions & 4 deletions lib/knapsack_pro/slow_test_file_determiner.rb
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
module KnapsackPro
class SlowTestFileDeterminer
TIME_THRESHOLD_PER_CI_NODE = 0.7 # 70%
REPORT_DIR = "#{KnapsackPro::Config::Env::TMP_DIR}/slow_test_file_determiner"

# test_files: { 'path' => 'a_spec.rb', 'time_execution' => 0.0 }
# time_execution: of build distribution (total time of CI build run)
Expand All @@ -14,20 +13,24 @@ def self.call(test_files, time_execution)
end

def self.save_to_json_report(test_files)
FileUtils.mkdir_p(REPORT_DIR)
FileUtils.mkdir_p(report_dir)
File.write(report_path, test_files.to_json)
end

def self.read_from_json_report
raise 'Report with slow test files was not generated yet. If you have enabled split by test cases https://github.com/KnapsackPro/knapsack_pro-ruby#split-test-files-by-test-cases and you see this error it means that your tests accidentally cleaned up tmp/knapsack_pro directory. Please do not remove this directory during tests runtime!' unless File.exists?(report_path)
raise 'Report with slow test files was not generated yet. If you have enabled split by test cases https://github.com/KnapsackPro/knapsack_pro-ruby#split-test-files-by-test-cases and you see this error it means that your tests accidentally cleaned up .knapsack_pro directory. Please do not remove this directory during tests runtime!' unless File.exists?(report_path)
slow_test_files_json_report = File.read(report_path)
JSON.parse(slow_test_files_json_report)
end

private

def self.report_path
"#{REPORT_DIR}/slow_test_files_node_#{KnapsackPro::Config::Env.ci_node_index}.json"
"#{report_dir}/slow_test_files_node_#{KnapsackPro::Config::Env.ci_node_index}.json"
end

def self.report_dir
"#{KnapsackPro::Config::TempFiles.temp_directory_path}/slow_test_file_determiner"
end
end
end
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
module KnapsackPro
module TestCaseDetectors
class RSpecTestExampleDetector
REPORT_DIR = "#{KnapsackPro::Config::Env::TMP_DIR}/test_case_detectors/rspec"

def generate_json_report
require 'rspec/core'

Expand Down Expand Up @@ -69,8 +67,12 @@ def slow_test_files

private

def report_dir
"#{KnapsackPro::Config::TempFiles.temp_directory_path}/test_case_detectors/rspec"
end

def report_path
"#{REPORT_DIR}/rspec_dry_run_json_report_node_#{KnapsackPro::Config::Env.ci_node_index}.json"
"#{report_dir}/rspec_dry_run_json_report_node_#{KnapsackPro::Config::Env.ci_node_index}.json"
end

def adapter_class
Expand All @@ -86,7 +88,7 @@ def test_file_pattern
end

def ensure_report_dir_exists
FileUtils.mkdir_p(REPORT_DIR)
FileUtils.mkdir_p(report_dir)
end

def remove_old_json_report
Expand Down
3 changes: 2 additions & 1 deletion lib/knapsack_pro/tracker.rb
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ def set_defaults
end

def tracker_dir_path
"#{KnapsackPro::Config::Env::TMP_DIR}/tracker"
"#{KnapsackPro::Config::TempFiles.temp_directory_path}/tracker"
end

def prerun_tests_report_path
Expand All @@ -110,6 +110,7 @@ def save_prerun_tests_report(hash)
end

def read_prerun_tests_report
raise "Report #{prerun_tests_report_path} doest not exist on the disk. Most likely, it was removed accidentally. Please report the bug to the support team." unless File.exists?(prerun_tests_report_path)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can add a link here, or at least specify that we mean the Knapsack support team?

JSON.parse(File.read(prerun_tests_report_path))
end

Expand Down
18 changes: 11 additions & 7 deletions spec/knapsack_pro/adapters/base_adapter_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,13 @@
context 'when CI node index 0' do
let(:ci_node_index) { 0 }

it { expect(subject).to eq 'tmp/knapsack_pro/KnapsackPro-Adapters-BaseAdapter-bind_method_called_for_node_0.txt' }

it { expect(subject).to eq '.knapsack_pro/KnapsackPro-Adapters-BaseAdapter-bind_method_called_for_node_0.txt' }
end

context 'when CI node index 1' do
let(:ci_node_index) { 1 }

it { expect(subject).to eq 'tmp/knapsack_pro/KnapsackPro-Adapters-BaseAdapter-bind_method_called_for_node_1.txt' }
it { expect(subject).to eq '.knapsack_pro/KnapsackPro-Adapters-BaseAdapter-bind_method_called_for_node_1.txt' }
end
end

Expand Down Expand Up @@ -93,18 +92,21 @@
end

describe '.verify_bind_method_called' do
let(:temp_directory_path) { '.knapsack_pro' }

subject { described_class.verify_bind_method_called }

before do
expect(::Kernel).to receive(:at_exit).and_yield
expect(File).to receive(:exists?).with('tmp/knapsack_pro/KnapsackPro-Adapters-BaseAdapter-bind_method_called_for_node_0.txt').and_return(adapter_bind_method_called_file_exists)
expect(KnapsackPro::Config::TempFiles).to receive(:temp_directory_path).at_least(1).and_return(temp_directory_path)
expect(File).to receive(:exists?).with('.knapsack_pro/KnapsackPro-Adapters-BaseAdapter-bind_method_called_for_node_0.txt').and_return(adapter_bind_method_called_file_exists)
end

context 'when adapter bind method called' do
let(:adapter_bind_method_called_file_exists) { true }

it do
expect(File).to receive(:delete).with('tmp/knapsack_pro/KnapsackPro-Adapters-BaseAdapter-bind_method_called_for_node_0.txt')
expect(File).to receive(:delete).with('.knapsack_pro/KnapsackPro-Adapters-BaseAdapter-bind_method_called_for_node_0.txt')
subject
end
end
Expand All @@ -120,12 +122,14 @@
end

describe '#bind' do
let(:temp_directory_path) { '.knapsack_pro' }
let(:recording_enabled?) { false }
let(:queue_recording_enabled?) { false }

before do
expect(FileUtils).to receive(:mkdir_p).with('tmp/knapsack_pro')
expect(File).to receive(:write).with('tmp/knapsack_pro/KnapsackPro-Adapters-BaseAdapter-bind_method_called_for_node_0.txt', nil)
expect(KnapsackPro::Config::TempFiles).to receive(:temp_directory_path).at_least(1).and_return(temp_directory_path)
expect(FileUtils).to receive(:mkdir_p).with(temp_directory_path)
expect(File).to receive(:write).with('.knapsack_pro/KnapsackPro-Adapters-BaseAdapter-bind_method_called_for_node_0.txt', nil)

expect(KnapsackPro::Config::Env).to receive(:recording_enabled?).and_return(recording_enabled?)
expect(KnapsackPro::Config::Env).to receive(:queue_recording_enabled?).and_return(queue_recording_enabled?)
Expand Down
22 changes: 22 additions & 0 deletions spec/knapsack_pro/config/temp_files_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
describe KnapsackPro::Config::TempFiles, :clear_tmp do
describe '.temp_directory_path' do
let(:gitignore_file_path) { '.knapsack_pro/.gitignore' }

subject { described_class.temp_directory_path }

it 'returns temporary directory path' do
expect(subject).to eq '.knapsack_pro'
end

it 'creates .gitignore file' do
expect(File.exists?(gitignore_file_path)).to be false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI there is a nice DSL for expectations like that https://relishapp.com/rspec/rspec-expectations/v/2-0/docs/matchers/expect-change

e.g. expect{Counter.increment}.to change{Counter.count}.from(0).to(1)

(just wanted to share this in case you forgot about it, the current one is cool by all means)

subject
expect(File.exists?(gitignore_file_path)).to be true
end

it '.gitignore file has content' do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that the most important line in the file's content is the one with the asterisk (*)? If so, then maybe it'd be better to check the inclusion of it instead?

subject
expect(File.read(gitignore_file_path)).to include '# This directory is used by knapsack_pro gem for temporary files during tests runtime'
end
end
end
19 changes: 9 additions & 10 deletions spec/knapsack_pro/report_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@

expect(
JSON.parse(
File.read('tmp/knapsack_pro/queue/fake-queue-id/fake-subset-queue-id.json')
File.read('.knapsack_pro/queue/fake-queue-id/fake-subset-queue-id.json')
)
).to eq([
{ 'path' => fake_path }
Expand All @@ -56,7 +56,7 @@
queue_id = 'fake-queue-id'
expect(KnapsackPro::Config::Env).to receive(:queue_id).and_return(queue_id)

expect(Dir).to receive(:glob).with('tmp/knapsack_pro/queue/fake-queue-id/*.json').and_return([
expect(Dir).to receive(:glob).with('.knapsack_pro/queue/fake-queue-id/*.json').and_return([
json_test_file_a_path,
json_test_file_b_path
])
Expand Down Expand Up @@ -89,7 +89,7 @@
queue_id = 'fake-queue-id'
expect(KnapsackPro::Config::Env).to receive(:queue_id).and_return(queue_id)

expect(Dir).to receive(:glob).with('tmp/knapsack_pro/queue/fake-queue-id/*.json').and_return([
expect(Dir).to receive(:glob).with('.knapsack_pro/queue/fake-queue-id/*.json').and_return([
json_test_file_a_path,
json_test_file_b_path
])
Expand All @@ -102,9 +102,9 @@
logger = instance_double(Logger)
expect(KnapsackPro).to receive(:logger).exactly(4).and_return(logger)
expect(logger).to receive(:warn).with('2 test files were executed on this CI node but the recorded time was lost due to:')
expect(logger).to receive(:warn).with('1. Probably you have a code (i.e. RSpec hooks) that clears tmp directory in your project. Please ensure you do not remove the content of tmp/knapsack_pro/queue/ directory between tests run.')
expect(logger).to receive(:warn).with('2. Another reason might be that you forgot to add Knapsack::Adapters::RSpecAdapter.bind in your rails_helper.rb or spec_helper.rb. Please follow the installation guide again: https://docs.knapsackpro.com/integration/')
expect(logger).to receive(:warn).with('3. All your tests are empty test files, are pending tests or have syntax error and could not be executed hence no measured time execution by knapsack_pro.')
expect(logger).to receive(:warn).with('1. Please ensure you do not remove the content of .knapsack_pro/queue/ directory between tests run.')
expect(logger).to receive(:warn).with('2. Ensure you added Knapsack::Adapters::RSpecAdapter.bind in your rails_helper.rb or spec_helper.rb. Please follow the installation guide again: https://docs.knapsackpro.com/integration/')
expect(logger).to receive(:warn).with('3. Another reason for this warning is that all your tests are empty test files, pending tests, or they have syntax errors, and the time execution was not recorded for them.')

expect(described_class).to receive(:create_build_subset).with(
json_test_file_a + json_test_file_b
Expand All @@ -121,15 +121,14 @@
queue_id = 'fake-queue-id'
expect(KnapsackPro::Config::Env).to receive(:queue_id).and_return(queue_id)

expect(Dir).to receive(:glob).with('tmp/knapsack_pro/queue/fake-queue-id/*.json').and_return([])
expect(Dir).to receive(:glob).with('.knapsack_pro/queue/fake-queue-id/*.json').and_return([])
end

it 'logs warning about reasons why no test files were executed on this CI node' do
logger = instance_double(Logger)
expect(KnapsackPro).to receive(:logger).exactly(3).and_return(logger)
expect(KnapsackPro).to receive(:logger).exactly(2).and_return(logger)
expect(logger).to receive(:warn).with('No test files were executed on this CI node.')
expect(logger).to receive(:debug).with('When you use knapsack_pro queue mode then probably reason might be that CI node was started after the test files from the queue were already executed by other CI nodes. That is why this CI node has no test files to execute.')
expect(logger).to receive(:debug).with("Another reason might be when your CI node failed in a way that prevented knapsack_pro to save time execution data to Knapsack Pro API and you have just tried to retry failed CI node but instead you got no test files to execute. In that case knapsack_pro don't know what tests should be executed here.")
expect(logger).to receive(:debug).with('This CI node likely started work late after the test files were already executed by other CI nodes consuming the queue.')

expect(described_class).to receive(:create_build_subset).with([])

Expand Down
Loading