Goblin and Janusgraph--Amazon DynamoDB Storage Backend for JanusGraph

[jbrow70]

Hi @davebshow,

So Janusgraph works with Goblin, right?

There's an option to use AWS DynamoDB as the backend storage for Janusgraph. We were looking at that as an option for a managed cloud solution but using AWS. I know you're looking at getting Goblin working with Cosmos DB, but we thought maybe we'd try janusgraph with DynamodDB but wanted to know if you (or anyone else) has tried Goblin out on Janusgraph with DynamodDB option.

Here's the link: https://github.com/awslabs/dynamodb-janusgraph-storage-backend

We're trying it right now but running across issues.

Thanks,

Jordan

[davebshow]

Yes, Goblin tests and runs well against current versions of TinkerPop and Janus graph. I'm not sure about DynamoDB. The README on Github says: "JanusGraph 0.1.1 and TinkerPop 3.2.3 compatibility". If those are indeed the required versions, I don't think Goblin or aiogremlin or Gremlin Python will work. If you can deploy with TP 3.2.6 though, you should be good to go.

[jbrow70]

Ok yeah we'll try that. We saw that the version was older so that was a concern for us. We actually tried running just gremlin python against it a couple of weeks ago and wouldn't even work there. I don't think we ever tried upgrading to a later version of Tinkerpop though. We'll try that if not.

-Jordan

[davebshow]

Well, I'm sorry that you are having so much trouble getting going here. It seems like a lot of providers are way behind on their TinkerPop support, which causes problems due to the fact that gremlin python wasn't even around pre 3.2.4, and was just a toy implementation until 3.2.5 (or maybe 3.2.6). Hopefully as these technologies mature, this won't be so much of an issue, as you will simply just match framework and driver versions. Its great to see that you guys are following through though with TP--that is what will drive providers to stay up to date.

[jbrow70]

Hi @davebshow,

If you don't mind, can you give us a working example of TikerPop, Goblin, and Janus graph implementation? Do you have any simple app to test its implementation, including the configuration (yaml file), and the versions you used?

We're planning on standing up Janus Graph With Tinker Pop just using the standard back end (think it uses Casandra). That way, for now, we can use Goblin and Gremlin language so we can build our app in a more agnostic approach. Then later on, we can port over to DynamoDB (if that catches up) or Cosmos DB if that ever gets going.

[davebshow]

Sounds like a good approach. Here is a link to an example project. I just built Janus from source and it runs fine. To build Janus, I do:

$ mvn clean install -Pjanusgraph-release -Dgpg.skip=true -DskipTests=true

The snapshot will end up here: janusgraph-dist/janusgraph-dist-hadoop-2/target/

To run the example, do:

$ git clone https://github.com/davebshow/goblin-janus-examples.git
$ cd goblin-janus-examples
$ pip install -e .
$ pip install gremlinpython --no-deps
$ python examples/create_graph.py

It doesn't do much, but I think it will get you going.

[jbrow70]

Thanks @davebshow!

[jbrow70]

Hi @davebshow,

So we got Goblin up and running with Janusgraph using the preconfigured server bundles on their release page (https://github.com/JanusGraph/janusgraph/releases). What's interesting is their release version of JanusGraph, 0.1.1 is compatible with Apache TinkerPop 3.2.3, and it works with Goblin which requires TinkerPop 3.2.4. Based on that, it seems like dynamodb-janusgraph-storage-backend (https://github.com/awslabs/dynamodb-janusgraph-storage-backend) which also compatible with JanusGraph 0.1.1 and TinkerPop 3.2.3 compatibility should work. We're going to explore that option a little more too but for now we're using the preconfigured server bundle for janusgraph since it's working.

If you're interested, see this thread: amazon-archives/dynamodb-janusgraph-storage-backend#242

-Jordan

[davebshow]

Which goblin are you using?

…

On Wed, Oct 18, 2017 at 10:34 AM, jbrow70 ***@***.***> wrote: Hi @davebshow <https://github.com/davebshow>, So we got Goblin up and running with Janusgraph using the preconfigured server bundles on their release page (https://github.com/ JanusGraph/janusgraph/releases). What's interesting is their release version of JanusGraph, 0.1.1 is compatible with Apache TinkerPop 3.2.3, and it works with Goblin which requires TinkerPop 3.2.4. Based on that, it seems like dynamodb-janusgraph-storage-backend ( https://github.com/awslabs/dynamodb-janusgraph-storage-backend) which also compatible with JanusGraph 0.1.1 and TinkerPop 3.2.3 compatibility should work. We're going to explore that option a little more too but for now we're using the preconfigured server bundle for janusgraph since it's working. If you're interested, see this thread: awslabs/dynamodb-janusgraph- storage-backend#242 <amazon-archives/dynamodb-janusgraph-storage-backend#242> -Jordan — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#85 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADj62YPaM09nVfMUNdqklCviDdYwRG5Iks5stjatgaJpZM4P6_wU> .

-- David M. Brown R.A. CulturePlex Lab, Western University

[jbrow70]

goblin 2.1.0rc3 Same one that you're using in link. We're using gremlinpython 3.2.6.

[jbrow70]

By the way, does Goblin have a way to add indexes or enforce uniqueness on properties or vertex classes / edge classes? We couldn't find a feature for that in the OGM. Additionally, running commands using session.submit() to construct the indexes seems to fail silently. Is there a way to ensure errors come back?

[jbrow70]

Hey we figured it out. We're using the cluster object and creating the client. That way we can access the response and get errors back (for creating indexes and the like). We're building out a JanusGraph integration with Goblin currently. (Not using DynamoDB yet, just a standard JanusGraph-Cassandra instance on an EC2). We'd be happy to share what we build out for Janus as we get farther along.

[davebshow]

Yes perfect. All raw scripts should be submitted to the server using the aiogremlin.Client class. Please share any relative code. Ideally, I would like to create a goblin-janus plugin that contains useful utilities for Janus. For the record, I wrote a really simple schema generator for Janus as a demo, and though it isn't configurable and is lacking a lot of functionality , it may be of some use to you.

If you don't mind me asking, what is your use case for goblin/Janus? Academic or industry?