Code coverage is lost for first multi_scan run

[john-mejia]

New Issue Checklist

• Updated fastlane-plugin-test_center to the latest version
• I read the README.md
• I reviewed the example(s) for the action(s) I am using
• I have removed any sensitive data such as passwords, authentication tokens, or anything else I do not want to world to see
• I have reviewed the Discussions forum to see my question has already been addressed.

If you love this fastlane plugin, consider sponsoring it or asking your company to sponsor it. I would really appreciate any
gesture: https://github.com/sponsors/lyndsey-ferguson. 😍

Issue Description

When multi_scan tries to run tests multiple times, code coverage from the first run is lost. I have created a test project where this can be observed and have provided examples in the README:

https://github.com/john-mejia/testCoverage/blob/main/README.md

Environment

✅ fastlane environment ✅
Stack



Key
Value




OS
11.2.1


Ruby
2.6.1


Bundler?
true


Git
git version 2.24.3 (Apple Git-128)


Installation Source
~/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/bin/fastlane


Host
macOS 11.2.1 (20D74)


Ruby Lib Dir
~/.rbenv/versions/2.6.1/lib


OpenSSL Version
OpenSSL 1.0.2q  20 Nov 2018


Is contained
false


Is homebrew
false


Is installed via Fabric.app
false


Xcode Path
/Applications/Xcode.app/Contents/Developer/


Xcode Version
12.4



System Locale



Variable
Value





LANG
en_US.UTF-8
✅


LC_ALL




LANGUAGE





fastlane files:
`./fastlane/Fastfile`
# This file contains the fastlane.tools configuration
# You can find the documentation at https://docs.fastlane.tools
#
# For a list of all available actions, check out
#
#     https://docs.fastlane.tools/actions
#
# For a list of all available plugins, check out
#
#     https://docs.fastlane.tools/plugins/available-plugins
#

# Uncomment the line if you want fastlane to automatically update itself
# update_fastlane

default_platform(:ios)

platform :ios do

  desc 'Run tests (and re-run flakes)'
  lane :test do
    test_run_block = lambda do |testrun_info|
      failed_test_count = testrun_info[:failed].size

      if failed_test_count > 0
        UI.important('The run of tests would finish with failures due to fragile tests here.')

        try_attempt = testrun_info[:try_count]
        if try_attempt < 2
          UI.header('Since we are using :multi_scan, we can re-run just those failing tests!')
        end
      end
    end

    result = multi_scan(
      workspace: File.absolute_path('../testCoverage.xcworkspace'),
      try_count: 1,
      fail_build: true,
      include_simulator_logs: false,
      scheme: 'testCoverage',
      testrun_completed_block: test_run_block,
      code_coverage: true,
    )
    unless result[:failed_testcount].zero?
      UI.message("There are #{result[:failed_testcount]} legitimate failing tests")
    end
  end

end

`./fastlane/Appfile`
# app_identifier("[[APP_IDENTIFIER]]") # The bundle identifier of your app
# apple_id("[[APPLE_ID]]") # Your Apple email address


# For more information about the Appfile, see:
#     https://docs.fastlane.tools/advanced/#appfile

fastlane gems



Gem
Version
Update-Status




fastlane
2.176.0
✅ Up-To-Date



Loaded fastlane plugins:



Plugin
Version
Update-Status




fastlane-plugin-test_center
3.15.3
✅ Up-To-Date



Loaded gems



Gem
Version




did_you_mean
1.3.0


bundler
2.2.7


rake
13.0.3


CFPropertyList
3.0.3


public_suffix
4.0.6


addressable
2.7.0


artifactory
3.0.15


atomos
0.1.3


aws-eventstream
1.1.0


aws-partitions
1.429.0


aws-sigv4
1.2.2


jmespath
1.4.0


aws-sdk-core
3.112.0


aws-sdk-kms
1.42.0


aws-sdk-s3
1.88.1


babosa
1.0.4


claide
1.0.3


colored
1.2


colored2
3.1.2


colorize
0.8.1


highline
1.7.10


commander-fastlane
4.4.6


declarative
0.0.20


declarative-option
0.1.0


digest-crc
0.6.3


unf_ext
0.0.7.7


unf
0.1.4


domain_name
0.5.20190701


dotenv
2.7.6


emoji_regex
3.2.2


excon
0.79.0


faraday-net_http
1.0.1


multipart-post
2.0.0


ruby2_keywords
0.0.4


faraday
1.3.0


http-cookie
1.0.3


faraday-cookie_jar
0.0.7


faraday_middleware
1.0.0


fastimage
2.2.2


gh_inspector
1.1.3


jwt
2.2.2


memoist
0.16.2


multi_json
1.15.0


os
1.1.1


signet
0.14.1


googleauth
0.15.1


httpclient
2.8.3


mini_mime
1.0.2


uber
0.1.0


representable
3.0.4


retriable
3.1.2


google-api-client
0.38.0


rexml
3.2.4


webrick
1.7.0


google-apis-core
0.2.1


google-apis-iamcredentials_v1
0.1.0


google-apis-storage_v1
0.2.0


google-cloud-env
1.4.0


google-cloud-errors
1.0.1


google-cloud-core
1.5.0


google-cloud-storage
1.30.0


json
2.5.1


mini_magick
4.11.0


naturally
2.2.1


plist
3.6.0


rubyzip
2.3.0


security
0.1.3


simctl
1.6.8


slack-notifier
2.3.2


terminal-notifier
2.0.0


unicode-display_width
1.7.0


terminal-table
1.8.0


tty-screen
0.8.1


tty-cursor
0.7.1


tty-spinner
0.9.3


word_wrap
1.0.0


nanaimo
0.3.0


xcodeproj
1.19.0


rouge
2.0.7


xcpretty
0.3.0


xcpretty-travis-formatter
1.0.1


trainer
0.9.1


xctest_list
1.2.1


fastlane-plugin-test_center
3.15.3




generated on: 2021-02-25

[Patrick-Kladek]

I have the same issue. Is there a known workaround?

[lyndsey-ferguson]

Hi @Patrick-Kladek , I'm not aware of any workaround yet. However, you may want to use the testrun_completed_block to pass in a code block that finds and moves the code coverage file out and then add some code in your fastlane to combine them into one code coverage file.

Here is an example of how to use the block:
https://github.com/lyndsey-ferguson/fastlane-plugin-test_center/blob/master/docs/feature_details/multi_scan.md#examples

[lyndsey-ferguson]

I want to apologize that I have not responded to this issue. A family health issue is requiring me to narrow my focus on the essentials and I don't have the time I need to focus on the plugin more than a minute. If you can get some other contributors to this project to help with this, I would consider merging a resulting PR.