[AutoDiff][stdlib] Add JVPs to SIMDDifferentiation.swift.gyb #32854

efremale · 2020-07-13T19:07:00Z

Adds JVPs to Add JVPs to SIMDDifferentiation.swift.gyb

efremale · 2020-07-13T19:07:29Z

efremale · 2020-07-13T19:12:30Z

Adding the following test (similar to the reverse-mode one in simd.swift) fails with a huge SIL dump and the following stack dump. Is it some issue with forward mode implementation?

Stack dump:

0.	Program arguments: /home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-frontend -frontend -c -primary-file /home/alex/swift-source/swift/test/AutoDiff/validation-test/forward_mode.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -module-cache-path /home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/swift-test-results/x86_64-unknown-linux-gnu/clang-module-cache -swift-version 4 -ignore-module-source-info -enable-experimental-forward-mode-differentiation -module-name main -o /tmp/forward_mode-1f50b8.o 
1.	Swift version 5.3-dev (LLVM 037eb4cde6d32db, Swift d31f3acaf46a10b)
2.	While verifying SIL function "@$sxq_Iegir_xq_Iegnr_SBRz16_Differentiation14DifferentiableRz13TangentVectorAaBPQzRszs4SIMDR_AaBR_AcDQy_Rs_6Scalars11SIMDStoragePQy_AERSr0_lTR".
 for expression at [/home/alex/swift-source/swift/test/AutoDiff/validation-test/forward_mode.swift:1606:12 - line:1606:38] RangeText="SIMDType.init(repeating: x"
 #0 0x000055f40c83c814 PrintStackTraceSignalHandler(void*) (/home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-frontend+0x53ce814)
 #1 0x000055f40c83a5ce llvm::sys::RunSignalHandlers() (/home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-frontend+0x53cc5ce)
 #2 0x000055f40c83cae5 SignalHandler(int) (/home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-frontend+0x53ceae5)
 #3 0x00007f2e89df1960 __restore_rt (/usr/lib/libpthread.so.0+0x14960)
 #4 0x00007f2e8986f355 raise (/usr/lib/libc.so.6+0x3c355)
 #5 0x00007f2e89858853 abort (/usr/lib/libc.so.6+0x25853)
 #6 0x000055f408971ee9 (/home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-frontend+0x1503ee9)
 #7 0x000055f4089737be (anonymous namespace)::SILVerifier::visitSILBasicBlock(swift::SILBasicBlock*) (/home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-frontend+0x15057be)
 #8 0x000055f40896d396 swift::SILFunction::verify(bool) const (/home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-frontend+0x14ff396)
 #9 0x000055f408970f3f swift::SILModule::verify() const (/home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-frontend+0x1502f3f)
#10 0x000055f407e51f50 swift::CompilerInstance::performSILProcessing(swift::SILModule*) (/home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-frontend+0x9e3f50)
#11 0x000055f407d2a994 performCompileStepsPostSILGen(swift::CompilerInstance&, std::unique_ptr<swift::SILModule, std::default_delete<swift::SILModule> >, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) (/home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-frontend+0x8bc994)
#12 0x000055f407d1bd68 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) (/home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-frontend+0x8add68)
#13 0x000055f407cafc7d main (/home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-frontend+0x841c7d)
#14 0x00007f2e8985a002 __libc_start_main (/usr/lib/libc.so.6+0x27002)
#15 0x000055f407caf80e _start (/home/alex/swift-source/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift-frontend+0x84180e)
<unknown>:0: error: unable to execute command: Aborted
<unknown>:0: error: compile command failed due to signal 6 (use -v to see invocation)

--

********************

Testing Time: 36.61s
********************
Failing Tests (1):
    Swift(linux-x86_64) :: AutoDiff/validation-test/forward_mode.swift

  Unexpected Failures: 1

The test:

ForwardModeTests.test("Generics") {
  let a = SIMD3<Double>(1, 2, 3)
  let g = SIMD3<Double>(1, 1, 1)

  func testInit<Scalar, SIMDType: SIMD>(x: Scalar) -> SIMDType
    where SIMDType.Scalar == Scalar,
          SIMDType : Differentiable,
          Scalar : BinaryFloatingPoint & Differentiable,
          SIMDType.TangentVector == SIMDType,
          Scalar.TangentVector == Scalar {
    return SIMDType.init(repeating: x)
  }
  func simd3Init(x: Double) -> SIMD3<Double> { testInit(x: x) }
  let (val1, df1) = valueWithDifferential(at: 10, in: simd3Init)
  expectEqual(SIMD3<Double>(10, 10, 10), val1)
  expectEqual(SIMD3<Double>(5, 5, 5), df1(5))

  // SIMDType + SIMDType
  func testAddition<Scalar, SIMDType: SIMD>(lhs: SIMDType, rhs: SIMDType)
    -> SIMDType
    where SIMDType.Scalar == Scalar,
          SIMDType : Differentiable,
          SIMDType.TangentVector : SIMD,
          Scalar : BinaryFloatingPoint,
          SIMDType.TangentVector.Scalar : BinaryFloatingPoint {
    return lhs + rhs
  }
  func simd3Add(lhs: SIMD3<Double>, rhs: SIMD3<Double>) -> SIMD3<Double> {
    return testAddition(lhs: lhs, rhs: rhs)
  }
  let (val2, df2) = valueWithDifferential(at: a, a, in: simd3Add)
  expectEqual(SIMD3<Double>(2, 4, 6), val2)
  expectEqual(g + a, df2(g, a))

  // Scalar - SIMDType
  func testSubtraction<Scalar, SIMDType: SIMD>(lhs: Scalar, rhs: SIMDType)
    -> SIMDType
    where SIMDType.Scalar == Scalar,
          SIMDType : Differentiable,
          Scalar : BinaryFloatingPoint & Differentiable,
          SIMDType.TangentVector == SIMDType,
          Scalar.TangentVector == Scalar {
    return lhs - rhs
  }
  func simd3Subtract(lhs: Double, rhs: SIMD3<Double>) -> SIMD3<Double> {
    return testSubtraction(lhs: lhs, rhs: rhs)
  }
  let (val3, df3) = valueWithDifferential(at: 5, a, in: simd3Subtract)
  expectEqual(SIMD3<Double>(4, 3, 2), val3)
  expectEqual(2 - g, df3(2, g))

  // SIMDType * Scalar
  func testMultipication<Scalar, SIMDType: SIMD>(lhs: SIMDType, rhs: Scalar)
    -> SIMDType
    where SIMDType.Scalar == Scalar,
      SIMDType : Differentiable,
      Scalar : BinaryFloatingPoint & Differentiable,
      SIMDType.TangentVector == SIMDType,
      Scalar.TangentVector == Scalar {
    return lhs * rhs
  }
  func simd3Multiply(lhs: SIMD3<Double>, rhs: Double) -> SIMD3<Double> {
    return testMultipication(lhs: lhs, rhs: rhs)
  }
  let (val4, df4) = valueWithDifferential(at: a, 5, in: simd3Multiply)
  expectEqual(SIMD3<Double>(5, 10, 15), val4)
  expectEqual(a * 3 + g * 5 , df4(g, 3))
}

dan-zheng · 2020-07-13T19:20:57Z

I'll try to reproduce the compiler crash.
Is there any assertion failure message in the stack dump right above the first line you shared (0.)?

Conclusion: we narrowed down the failing test case (SR-13210) and commented it out for now. We can investigate the underlying issue later.

efremale · 2020-07-13T19:27:00Z

I'll try to reproduce the compiler crash.
Is there any assertion failure message in the stack dump right above the first line you shared (0.)?

I'm only seeing a lot of SIL that ends with:

...
sil_witness_table public_external [serialized] Double: Numeric module Swift {
  base_protocol AdditiveArithmetic: Double: AdditiveArithmetic module Swift
  base_protocol ExpressibleByIntegerLiteral: Double: ExpressibleByIntegerLiteral module Swift
  associated_type_protocol (Magnitude: Comparable): Double: Comparable module Swift
  associated_type_protocol (Magnitude: Numeric): Double: Numeric module Swift
  associated_type Magnitude: Double
  method #Numeric.init!allocator: <Self where Self : Numeric><T where T : BinaryInteger> (Self.Type) -> (T) -> Self? : @$sSdSjsSj7exactlyxSgqd___tcSzRd__lufCTW// protocol witness for Numeric.init<A>(exactly:) in conformance Double
  method #Numeric.magnitude!getter: <Self where Self : Numeric> (Self) -> () -> Self.Magnitude : @$sSdSjsSj9magnitude9MagnitudeQzvgTW	// protocol witness for Numeric.magnitude.getter in conformance Double
  method #Numeric."*": <Self where Self : Numeric> (Self.Type) -> (Self, Self) -> Self : @$sSdSjsSj1moiyxx_xtFZTW	// protocol witness for static Numeric.* infix(_:_:) in conformance Double
  method #Numeric."*=": <Self where Self : Numeric> (Self.Type) -> (inout Self, Self) -> () : @$sSdSjsSj2meoiyyxz_xtFZTW	// protocol witness for static Numeric.*= infix(_:_:) in conformance Double
}



Stack dump:
0. ...

dan-zheng · 2020-07-13T21:32:51Z

@swift-ci Please smoke test

rxwei

Could you add these to swift-numerics as well?

dan-zheng · 2020-07-14T17:10:24Z

@swift-ci Please smoke test

rxwei · 2020-07-14T17:11:16Z

To clarify, I didn't mean moving SIMD derivatives. I meant adding JVPs to swift-numerics for operations that already have a VJP such as Complex. This PR looks good to me as is.

[AutoDiff][stdlib] Add JVPs to SIMDDifferentiation.swift.gyb

73864f9

efremale and others added 2 commits July 13, 2020 21:50

Add failing test

2f9d0b8

Narrow down SR-13210 failing test.

98c8360

dan-zheng requested review from marcrasi and rxwei July 13, 2020 21:32

dan-zheng approved these changes Jul 13, 2020

View reviewed changes

rxwei approved these changes Jul 13, 2020

View reviewed changes

rxwei requested a review from stephentyrone July 14, 2020 00:00

dan-zheng merged commit cff144a into swiftlang:master Jul 14, 2020

This was referenced Jul 16, 2020

[AutoDiff] [stdlib] Add a JVP for 'differentiableMap(_:)'. #32769

Closed

[AutoDiff] [stdlib] Fix a bug in _jvpMultiply in SIMDDifferentiation.swift.gyb #32963

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[AutoDiff][stdlib] Add JVPs to SIMDDifferentiation.swift.gyb #32854

[AutoDiff][stdlib] Add JVPs to SIMDDifferentiation.swift.gyb #32854

efremale commented Jul 13, 2020

efremale commented Jul 13, 2020

efremale commented Jul 13, 2020

dan-zheng commented Jul 13, 2020 •

edited

Loading

efremale commented Jul 13, 2020

dan-zheng commented Jul 13, 2020

rxwei left a comment

dan-zheng commented Jul 14, 2020

rxwei commented Jul 14, 2020

[AutoDiff][stdlib] Add JVPs to SIMDDifferentiation.swift.gyb #32854

[AutoDiff][stdlib] Add JVPs to SIMDDifferentiation.swift.gyb #32854

Conversation

efremale commented Jul 13, 2020

efremale commented Jul 13, 2020

efremale commented Jul 13, 2020

dan-zheng commented Jul 13, 2020 • edited Loading

efremale commented Jul 13, 2020

dan-zheng commented Jul 13, 2020

rxwei left a comment

Choose a reason for hiding this comment

dan-zheng commented Jul 14, 2020

rxwei commented Jul 14, 2020

dan-zheng commented Jul 13, 2020 •

edited

Loading