Cassandra Dependency (scope/security)

[planetf1]

Currently we are pulling cassandra oriented dependencies into our compile dependency tree:

[INFO] |  +- org.odpi.egeria:connector-configuration-factory:jar:1.5-SNAPSHOT:compile
[INFO] |  |  +- org.odpi.egeria:open-metadata-archive-file-connector:jar:1.5-SNAPSHOT:compile
[INFO] |  |  +- org.odpi.egeria:audit-log-file-connector:jar:1.5-SNAPSHOT:compile
[INFO] |  |  +- org.odpi.egeria:audit-log-console-connector:jar:1.5-SNAPSHOT:compile
[INFO] |  |  +- org.odpi.egeria:audit-log-slf4j-connector:jar:1.5-SNAPSHOT:compile
[INFO] |  |  +- org.odpi.egeria:audit-log-event-topic-connector:jar:1.5-SNAPSHOT:compile
[INFO] |  |  +- org.odpi.egeria:cohort-registry-file-store-connector:jar:1.5-SNAPSHOT:compile
[INFO] |  |  +- org.odpi.egeria:inmemory-repository-connector:jar:1.5-SNAPSHOT:compile
[INFO] |  |  +- org.odpi.egeria:graph-repository-connector:jar:1.5-SNAPSHOT:compile
[INFO] |  |  |  +- org.janusgraph:janusgraph-cassandra:jar:0.4.1:runtime

This causes exposure of all modules - potentially to any security issues. We currently know that org.apache.thrift:libthrift:0.9.2 is subject to CVE-2018-1320. Even if this isn't exploitable for other components we should be limiting the scope of all third dependencies to the minimum required.

The dependency tree above shows we have a compile & runtime (default) dependency from a core egeria component - the connector configuration factory.

Ideally

• All connectors should only be loaded dynamically. They may not need to be expressed as a dependency at all, since they are optional and dependent on configuration. If necessary they could be 'provided' or 'runtime'
• The connector will need to be added in the distribution definition egeria-omag.xml so that they get placed in server/lib
• The maven dependencies can be removed

-- at least that's.a theory ....

Furthermore We should note :

org.odpi.egeria:graph-repository-connector:jar:1.5-SNAPSHOT
[INFO] +- org.janusgraph:janusgraph-cassandra:jar:0.4.1:runtime
[INFO] |  +- io.dropwizard.metrics:metrics-jvm:jar:3.2.2:runtime
[INFO] |  +- ch.qos.logback:logback-classic:jar:1.2.3:compile
[INFO] |  |  \- ch.qos.logback:logback-core:jar:1.2.3:compile
[INFO] |  +- org.apache.cassandra:cassandra-all:jar:2.2.13:runtime
[INFO] |  |  +- com.ning:compress-lzf:jar:0.8.4:runtime
[INFO] |  |  +- commons-cli:commons-cli:jar:1.1:runtime
[INFO] |  |  +- org.apache.commons:commons-math3:jar:3.2:runtime
[INFO] |  |  +- com.googlecode.concurrentlinkedhashmap:concurrentlinkedhashmap-lru:jar:1.4:runtime
[INFO] |  |  +- org.antlr:antlr:jar:3.5.2:runtime
[INFO] |  |  |  \- org.antlr:ST4:jar:4.0.8:runtime
[INFO] |  |  +- com.googlecode.json-simple:json-simple:jar:1.1:runtime
[INFO] |  |  +- com.addthis.metrics:reporter-config3:jar:3.0.0:runtime
[INFO] |  |  |  \- com.addthis.metrics:reporter-config-base:jar:3.0.0:runtime
[INFO] |  |  +- com.thinkaurelius.thrift:thrift-server:jar:0.3.7:runtime
[INFO] |  |  |  \- com.lmax:disruptor:jar:3.0.1:runtime
[INFO] |  |  +- com.clearspring.analytics:stream:jar:2.5.2:runtime
[INFO] |  |  |  \- it.unimi.dsi:fastutil:jar:6.5.7:runtime
[INFO] |  |  +- net.sf.supercsv:super-csv:jar:2.1.0:runtime
[INFO] |  |  +- org.apache.thrift:libthrift:jar:0.9.2:runtime
[INFO] |  |  +- org.apache.cassandra:cassandra-thrift:jar:2.2.13:runtime

In addition we should raise an issue against JanusGraph (cc: @grahamwallis ) to see if they can/should be using a more recent version of org.apache.cassandra:cassandra-all . Currently Janus is using 2.2.13 from July 2018. Cassnadra is currently in 4.x beta.

[planetf1]

A further issue is that the currentgraph repository connector is explicitly depending on org.codehaus.jackson:jackson-mapper-asl.jar. This is documented as a runtime dependency, and the maven dependency checks are marked as an exception (as is normal for a runtime only dependency)

We should change this in our own code -- much easier than the changes above -- to use current versions of jackson and thus address CVE-2017-7525 and CVE-2019-10172

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 20 days if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 20 days if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 20 days if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 20 days if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 20 days if no further activity occurs. Thank you for your contributions.

[planetf1]

Current dependency chains:

[INFO] -------------< org.odpi.egeria:graph-repository-connector >-------------
[INFO] org.odpi.egeria:graph-repository-connector:jar:2.6
[INFO] +- org.janusgraph:janusgraph-cassandra:jar:0.5.2:runtime
[INFO] |  +- org.apache.cassandra:cassandra-all:jar:2.2.13:runtime
[INFO] +- org.janusgraph:janusgraph-cql:jar:0.5.2:runtime
[INFO] |  +- com.datastax.cassandra:cassandra-driver-core:jar:3.9.0:runtime



[INFO] org.odpi.egeria:open-lineage-janus-connector:jar:2.6
[INFO] +- org.odpi.egeria:open-connector-framework:jar:2.6:compile
[INFO] +- org.janusgraph:janusgraph-core:jar:0.5.2:compile
[INFO] +- org.janusgraph:janusgraph-cql:jar:0.5.2:runtime
[INFO] |  +- com.datastax.cassandra:cassandra-driver-core:jar:3.9.0:runtime

But in addition we pull in 'datastax' libraries, in part from janusgraph-sql (addressed above), but also for example:

[INFO] org.odpi.egeria:cassandra-data-store-connector:jar:2.6
[INFO] +- com.datastax.oss:java-driver-core:jar:4.9.0:compile

Since this clearly is to support cassandra, and it's a build time dependency, I think this is valid, but probably should be in another repo.

[INFO] org.odpi.egeria:cassandra-metadata-extractor-connector:jar:2.6
[INFO] +- com.datastax.oss:java-driver-core:jar:4.9.0:compile

Similar...

And indirectly via this route:

[INFO] org.odpi.egeria:open-connector-archives:jar:2.6
[INFO] +- org.odpi.egeria:cassandra-data-store-connector:jar:2.6:compile

I'm not sure we should be building this at all...

So in summary I think we should:

• remove janusgraph cassandra/cql report from runtime dependency or the graph repository connector. This is a deployment specific decision, so we should add information to our guides to explain how to configure the way we use janusgraph, and point to the upstream documentation.
• Similarly the open lineage janus connector should not depend on cassandra/cql - again a deployment specific decision
• Move the cassandra connectors to a new repo longer term (it is ok to leave them as is for now) as they will only be picked up as needed Cassandra metadata extractor connector move to a separate repository? #4095
• consider removing the open connector archives - do we need them?

cc: @lpalashevski @bogdan-sava @mandy-chessell @grahamwallis

[planetf1]

I believe you are looking at this @mandy-chessell ?

[mandy-chessell]

Yes :)

[Shrinivas-Kane]

@mandy-chessell / @planetf1 is it possible to get this fix back ported to 2.8 ?

[mandy-chessell]

@Shrinivas-Kane Sorry we do not back-port fixes

[Shrinivas-Kane]

is there ETA when fix will be available ?

[Shrinivas-Kane]

@mandy-chessell can you please confirm release target for these security fixes ?

[mandy-chessell]

@Shrinivas-Kane When we have identified a resource to do this, we will identify the release in the milestones field.

[lenawoolf]

@mandy-chessell Mandy, Can you please help with identifying resource to fix this soon? For WKC, critical and high severity security defects are becoming a gating factors for release so fix for this issue is quite urgent because its a blocker for security issue https://github.ibm.com/InformationServer/Information-Server-Tracking-Board/issues/13692

[planetf1]

@lenawoolf @lpalashevski has been looking at this, and one step to remove the cassandra oriented chain is addressed by a current draft PR (should be merged soon) in #5243 . Cassandra does end up being pulled in & through it libthrift and the old jackson-mapper-asl, so those pathways should be addressed. The only other occurrences are via hadoop which is limited to, for example, the ranger connector.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 20 days if no further activity occurs. Thank you for your contributions.

[lpalashevski]

This issue is no logner relevant since outdate cassandra libs and connectors are now removed.
We are going to re-introduce cassandra-data-store-connector and cassandra-metadata-extractor-connector under dedicated data connectors repository. Ref. #4095