Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate dependecy conflicts between worker and user code #183

Closed
pragnagopa opened this issue Sep 18, 2018 · 7 comments · Fixed by #379
Closed

Investigate dependecy conflicts between worker and user code #183

pragnagopa opened this issue Sep 18, 2018 · 7 comments · Fixed by #379

Comments

@pragnagopa
Copy link
Member

An example of conflict #182
Need to investigate how users can bring in their dependencies that conflict with Java worker

@asavaritayal
Copy link

asavaritayal commented Oct 5, 2018

At the moment, we have the following runtime dependencies for the language worker:

  • protobuf-java : 3.6.1
  • grpc-protobuf : 1.15.0
  • grpc-stub : 1.15.0
  • grpc-netty : 1.15.0
  • guava : 26.0-jre
  • commons-lang3 : 3.8
  • commons-cli : 2.9.6
  • jackson-databind : 2.9.6
  • jackson-annotations : 2.9.6
  • javax.annotation-api : 1.3.2
  • junit : 4.12

@asavaritayal asavaritayal added this to the Backlog milestone Oct 5, 2018
@asavaritayal
Copy link

From @brunoborges -

Only way I can think of is if you ensure user-defined dependencies are loaded first in the clasapath. But that may break the worker if user chooses an incompatible version of a dependency.

In other words: I don't think it is possible.

As I mentioned couple months ago, right now the architecture has worker and user code tightly coupled.

The ideal solution is to have user code running in one Java process, and the worker in another one.

This is how OpenFaaS and FN Project are designed.

@brunoborges
Copy link
Member

Just to provide more details: the other option is to isolate user code with a different ClassLoader. But that imposes several other challenges.

@pragnagopa
Copy link
Member Author

This issue becomes more important when we start supporting rich types as that would require worker to take dependency on azure libraries for each component

@pragnagopa
Copy link
Member Author

Related issue #117
Reducing worker dependencies on third party libraries and allowing the users to configure classpath might help

@pragnagopa
Copy link
Member Author

Following are is complete list that is included in the azure-functions-java-worker

[INFO] Including com.microsoft.azure.functions:azure-functions-java-library:jar:1.0.0-beta-7-SNAPSHOT in the shaded jar.
[INFO] Including com.google.protobuf:protobuf-java:jar:3.6.1 in the shaded jar.
[INFO] Including io.grpc:grpc-protobuf:jar:1.15.1 in the shaded jar.
[INFO] Including io.grpc:grpc-core:jar:1.15.1 in the shaded jar.
[INFO] Including io.grpc:grpc-context:jar:1.15.1 in the shaded jar.
[INFO] Including com.google.code.gson:gson:jar:2.8.5 in the shaded jar.
[INFO] Including com.google.errorprone:error_prone_annotations:jar:2.2.0 in the shaded jar.
[INFO] Including com.google.code.findbugs:jsr305:jar:3.0.0 in the shaded jar.
[INFO] Including org.codehaus.mojo:animal-sniffer-annotations:jar:1.17 in the shaded jar.
[INFO] Including io.opencensus:opencensus-api:jar:0.12.3 in the shaded jar.
[INFO] Including io.opencensus:opencensus-contrib-grpc-metrics:jar:0.12.3 in the shaded jar.
[INFO] Including com.google.guava:guava:jar:26.0-jre in the shaded jar.
[INFO] Including org.checkerframework:checker-qual:jar:2.5.2 in the shaded jar.
[INFO] Including com.google.j2objc:j2objc-annotations:jar:1.1 in the shaded jar.
[INFO] Including com.google.api.grpc:proto-google-common-protos:jar:1.0.0 in the shaded jar.
[INFO] Including io.grpc:grpc-protobuf-lite:jar:1.15.1 in the shaded jar.
[INFO] Including io.grpc:grpc-stub:jar:1.15.1 in the shaded jar.
[INFO] Including io.grpc:grpc-netty:jar:1.15.1 in the shaded jar.
[INFO] Including io.netty:netty-codec-http2:jar:4.1.27.Final in the shaded jar.
[INFO] Including io.netty:netty-codec-http:jar:4.1.30.Final in the shaded jar.
[INFO] Including io.netty:netty-codec:jar:4.1.30.Final in the shaded jar.
[INFO] Including io.netty:netty-handler:jar:4.1.30.Final in the shaded jar.
[INFO] Including io.netty:netty-buffer:jar:4.1.30.Final in the shaded jar.
[INFO] Including io.netty:netty-common:jar:4.1.30.Final in the shaded jar.
[INFO] Including io.netty:netty-handler-proxy:jar:4.1.30.Final in the shaded jar.
[INFO] Including io.netty:netty-transport:jar:4.1.30.Final in the shaded jar.
[INFO] Including io.netty:netty-resolver:jar:4.1.30.Final in the shaded jar.
[INFO] Including io.netty:netty-codec-socks:jar:4.1.30.Final in the shaded jar.
[INFO] Including org.apache.commons:commons-lang3:jar:3.8.1 in the shaded jar.
[INFO] Including commons-cli:commons-cli:jar:1.4 in the shaded jar.
[INFO] Including javax.annotation:javax.annotation-api:jar:1.3.2 in the shaded ja

@asavaritayal / @rloutlaw - Any ideas where we should document this? As these are brought in at runtime it is not transparent to use. I will update the readme on azure-functions-java-library repo.

@Fabiest
Copy link

Fabiest commented Jan 29, 2020

Hello,

I'm suffering from similar problem where I want to use lib A which depends on Jackson 2.9.6 (annotations, binding, core) and lib 8 which depends on Jackson 2.4.0 (annotations).

If i force version 2.9.6 in the pom and use my code in a "console" app, it works fine, but in a function it always complains with error "NoSuchFieldError: USE_DEFAULTS" which only exists after 2.6.0 in jackson annotations.

My work around, so far, was to force jackson annotations to 2.5.5 (the lowest maximum version i managed to make it work with) which was fine to work with lib A, but now this causes issues with lib azure-identity which needs 2.7.0 of jackson otherwise I get other errors...

Am I supposed to set something on the host.json file for the function?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants