[lit] Show test script when '--no-execute -a' is specified to lit.py

[rintaro]

What's in this pull request?

Related: #4187 (comment)

Show test script when --no-execute -a is specified to lit.py
Updated utils/run-test to support that.

Note: This is a kind of hack. lit.py should implement this functionality.

Example:

$ utils/run-test --build-dir ../build/Ninja-ReleaseAssert --build=skip -v -n validation-test/stdlib/Dictionary.swift 
+ /usr/bin/python /home/rintaro/Documents/swift-oss/llvm/utils/lit/lit.py -a --no-execute --param swift_test_mode=optimize_none --param swift_test_subset=primary /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Dictionary.swift
lit.py: lit.cfg:268: note: using swift: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift
lit.py: lit.cfg:268: note: using swiftc: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swiftc
lit.py: lit.cfg:268: note: using sil-opt: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/sil-opt
lit.py: lit.cfg:268: note: using sil-extract: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/sil-extract
lit.py: lit.cfg:268: note: using lldb-moduleimport-test: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/lldb-moduleimport-test
lit.py: lit.cfg:268: note: using swift-ide-test: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-ide-test
lit.py: lit.cfg:268: note: using swift-reflection-dump: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-reflection-dump
lit.py: lit.cfg:268: note: using swift-remoteast-test: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-remoteast-test
lit.py: lit.cfg:268: note: using swift-format: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-format
lit.py: lit.cfg:268: note: using clang: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/llvm-linux-x86_64/./bin/clang
lit.py: lit.cfg:268: note: using llvm-link: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/llvm-linux-x86_64/./bin/llvm-link
lit.py: lit.cfg:268: note: using swift-llvm-opt: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-llvm-opt
lit.py: lit.cfg:268: note: using llvm-profdata: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/llvm-linux-x86_64/./bin/llvm-profdata
lit.py: lit.cfg:268: note: using llvm-cov: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/llvm-linux-x86_64/./bin/llvm-cov
lit.py: lit.cfg:316: note: Using resource dir: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/lib/swift
lit.py: lit.cfg:342: note: Using Clang module cache: /tmp/swift-testsuite-clang-module-cache72EMrh
lit.py: lit.cfg:343: note: Using test results dir: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/./swift-test-results/x86_64-unknown-linux-gnu
lit.py: lit.cfg:347: note: Using code completion cache: /tmp/swift-testsuite-completion-cacheKr7r6U
lit.py: lit.cfg:728: note: Testing Linux x86_64-unknown-linux-gnu
lit.py: lit.cfg:268: note: using swift-autolink-extract: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-autolink-extract
lit.py: lit.cfg:994: note: Using platform module dir: /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/lib/swift/%target-sdk-name/x86_64
-- Testing: 1 tests, 1 threads --
PASS: Swift(linux-x86_64) :: stdlib/Dictionary.swift (1 of 1)
Script:
--
rm -rf /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Dictionary.swift.tmp
mkdir -p /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Dictionary.swift.tmp
/home/rintaro/Documents/swift-oss/swift/utils/gyb /home/rintaro/Documents/swift-oss/swift/validation-test/stdlib/Dictionary.swift -o /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Dictionary.swift.tmp/main.swift
if [ native == "objc" ]; then    clang++ -target x86_64-unknown-linux-gnu -fmodules-cache-path='/tmp/swift-testsuite-clang-module-cache72EMrh' -fobjc-arc /home/rintaro/Documents/swift-oss/swift/validation-test/stdlib/Inputs/SlurpFastEnumeration/SlurpFastEnumeration.m -c -o /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Dictionary.swift.tmp/SlurpFastEnumeration.o;    /home/rintaro/Documents/swift-oss/swift/utils/line-directive /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Dictionary.swift.tmp/main.swift -- /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swiftc -target x86_64-unknown-linux-gnu  -module-cache-path '/tmp/swift-testsuite-clang-module-cache72EMrh'   /home/rintaro/Documents/swift-oss/swift/validation-test/stdlib/Inputs/DictionaryKeyValueTypes.swift /home/rintaro/Documents/swift-oss/swift/validation-test/stdlib/Inputs/DictionaryKeyValueTypesObjC.swift /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Dictionary.swift.tmp/main.swift -I /home/rintaro/Documents/swift-oss/swift/validation-test/stdlib/Inputs/SlurpFastEnumeration/ -Xlinker /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Dictionary.swift.tmp/SlurpFastEnumeration.o -o /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Dictionary.swift.tmp/Dictionary -Xfrontend -disable-access-control;  else    /home/rintaro/Documents/swift-oss/swift/utils/line-directive /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Dictionary.swift.tmp/main.swift -- /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swiftc -target x86_64-unknown-linux-gnu  -module-cache-path '/tmp/swift-testsuite-clang-module-cache72EMrh'   /home/rintaro/Documents/swift-oss/swift/validation-test/stdlib/Inputs/DictionaryKeyValueTypes.swift /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Dictionary.swift.tmp/main.swift -o /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Dictionary.swift.tmp/Dictionary -Xfrontend -disable-access-control;  fi
/home/rintaro/Documents/swift-oss/swift/utils/line-directive /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Dictionary.swift.tmp/main.swift --  /home/rintaro/Documents/swift-oss/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-linux-x86_64/stdlib/Output/Dictionary.swift.tmp/Dictionary
--

********************
Testing Time: 0.01s
  Expected Passes    : 1

---

Before merging this pull request to apple/swift repository:

• Test pull request on Swift continuous integration.

Triggering Swift CI

The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:

Smoke Testing

Platform	Comment
All supported platforms	@swift-ci Please smoke test
All supported platforms	@swift-ci Please smoke test and merge
OS X platform	@swift-ci Please smoke test OS X platform
Linux platform	@swift-ci Please smoke test Linux platform

A smoke test on macOS does the following:

1. Builds the compiler incrementally.
2. Builds the standard library only for macOS. Simulator standard libraries and
   device standard libraries are not built.
3. lldb is not built.
4. The test and validation-test targets are run only for macOS. The optimized
   version of these tests are not run.

A smoke test on Linux does the following:

1. Builds the compiler incrementally.
2. Builds the standard library incrementally.
3. lldb is built incrementally.
4. The swift test and validation-test targets are run. The optimized version of these
   tests are not run.
5. lldb is tested.

Validation Testing

Platform	Comment
All supported platforms	@swift-ci Please test
All supported platforms	@swift-ci Please test and merge
OS X platform	@swift-ci Please test OS X platform
OS X platform	@swift-ci Please benchmark
Linux platform	@swift-ci Please test Linux platform

Lint Testing

Language	Comment
Python	@swift-ci Please Python lint

Note: Only members of the Apple organization can trigger swift-ci.

[rintaro]

CC: @jrose-apple, @ddunbar, @gribozavr
I think, this satisfies our demand. Any thought?

[ddunbar]

I guess my main thought is we should land it in lit proper.

[rintaro]

I think so. This should be a temporary workaround.

[jrose-apple]

I'm with Daniel. The LLVM community is pretty receptive to additions like this to utilities, though they might ask for a test. Or one of us can take care of submitting the patch for you.

(Or as a compromise, you could submit the patch as a PR on apple/swift-llvm's upstream-with-swift branch, and then one of us will get it upstream.)

[rintaro]

OK, I will try to propose this to LLVM repo. :)
Closing.