Skip to content

Conversation

@tedyu
Copy link
Contributor

@tedyu tedyu commented Apr 9, 2016

What changes were proposed in this pull request?

After [SPARK-14462] went in, build shows the following error:

[error] missing or invalid dependency detected while loading class file 'SparkFunSuite.class'.
[error] Could not access type Logging in package org.apache.spark.internal,
[error] because it (or its dependencies) are missing. Check your build definition for
[error] missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
[error] A full rebuild may help if 'SparkFunSuite.class' was compiled against an incompatible version of org.apache.spark.internal.
[error] one error found
[error] Compile failed at Apr 9, 2016 9:31:47 AM [0.382s]

This was due to missing dependency of spark-core module where Logging resides.
SparkFunSuite imports Logging.

I tried creating SparkFunNoLoggingSuite but then realized that it becomes FunSuite.

Current PR lets DummyTestingSuite inherit FunSuite so that mllib-local module doesn't depend on spark-core.

How was this patch tested?

Gone through maven build.

@tedyu
Copy link
Contributor Author

tedyu commented Apr 9, 2016

@mengxr @dbtsai
Please take a look.

@holdenk
Copy link
Contributor

holdenk commented Apr 9, 2016

So I believe the plan is to not have mllib local depend on spark-core, so this probably isn't the fix we want (it might be better to roll back mllib-local if the master build is broken rather than trying to fix it)

@tedyu
Copy link
Contributor Author

tedyu commented Apr 9, 2016

From SPARK-14462:

The test scope will still depend on spark-core and spark-core-test in order to use the common utilities

If spark-core shouldn't be a dependency, we need to roll back mllib-local in order to unblock the build.

@holdenk
Copy link
Contributor

holdenk commented Apr 9, 2016

ah ok if its intended then that makes sense. I think its still better to revert rather than try and fix master build.

@SparkQA
Copy link

SparkQA commented Apr 9, 2016

Test build #55443 has finished for PR 12276 at commit 5e4ba99.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@tedyu
Copy link
Contributor Author

tedyu commented Apr 9, 2016

@holdenk
You made a good point.

@SparkQA
Copy link

SparkQA commented Apr 9, 2016

Test build #55450 has finished for PR 12276 at commit bf2a42f.

  • This patch fails Scala style tests.
  • This patch does not merge cleanly.
  • This patch adds the following public classes (experimental):
    • class DummyTestingSuite extends FunSuite

@tedyu
Copy link
Contributor Author

tedyu commented Apr 9, 2016

@mengxr
I saw you reverted [SPARK-14462].
[SPARK-14462] + this PR would pass the build.

Let me know your preference.

@tedyu tedyu changed the title [SPARK-14462] [HOTFIX] Add dependency for Logging [SPARK-14462] [HOTFIX] Let DummyTestingSuite inherit FunSuite Apr 9, 2016
@mengxr
Copy link
Contributor

mengxr commented Apr 10, 2016

+@dbtsai We plan to follow sketch and use FunSuite with no:scalastyle funsuite.

@tedyu
Copy link
Contributor Author

tedyu commented Apr 10, 2016

@mengxr
Is there anything I can do on this PR ?

@dbtsai
Copy link
Member

dbtsai commented Apr 11, 2016

@tedyu I created a new PR to address this issue. #12298

Please close this PR. Thanks.

@tedyu tedyu closed this Apr 11, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants