Create official Java driver

[deontologician]

https://github.com/npiv/rethink-java-driver might be a good jumping off point. It seems both of the 3rd party drivers for Java are abandoned or very out of date

[danielmewes]

I think we should try to get a beta release out by 2.1, especially since we might abandon protocol buffer support with that release and @npiv's driver is still using it.

Later we'll need to integrate it into our polyglot testing framework.

Another possible starting point for a driver is @dkhenry's Java driver. I've already ported it to the JSON protocol for benchmarking purposes in https://github.com/danielmewes/rethinkjava , but in contrast to @npiv's driver it's missing support for lambdas.

[deontologician]

Most likely the serialization stuff is easier to port from one to the other. There are some language design issues to consider as well though, I think the default should return a cursor, rather than needing runForCursor. So in any case, there would be a lot of fiddling to get it into shape

[kclay]

I've been meaning to add more javaness to my Scala driver( https://github.com/kclay/rethink-scala) since it enforces type safety and provides methods for casting, blocking and async modes ,jackson for json,etc. On @deontologician note about the need for runForCursor. The driver is built in a way that its able to determine if the result should be a single value or an cursor. Pretty sure I can refactor some of this to work with java since some parts relying strongly on Scala implicits. Just a thought.

[stephanos]

@deontologician
Will the new driver support Java 7?

[stephanos]

@danielmewes
I can't even get your fork to compile! After some fixing I am stuck with Error:(296, 37) java: incompatible types: long cannot be converted to com.google.protobuf.CodedInputStream (RqlConnection.java) @ https://github.com/stephanos/rethinkjava

[danielmewes]

@stephanos I saw you did some patches in your branch, did you get it to compile eventually? I'm not sure why it's not compiling (though I'm not surprised, I've only ever used it on one type of system for a single purpose). My guess would be an incompatible protocol buffer library or compiler.

The official driver will not be based on that code base by the way.

[stephanos]

@danielmewes no, I didn't :( -will have to wait for the official driver.

[danielmewes]

@stephanos another driver you can try until then is https://github.com/dropbox/rethink-java-driver

[deontologician]

@stephanos I'm building it with Java 8, and we will probably release the beta version tested only with Java 8, but supporting Java 7 shouldn't be too hard and I think we should support it as part of release version

[deontologician]

Actually @stephanos, what is your opinion on the Java 7 EOL announcement?

I assume there are lots of people still on Java 7, but I can't imagine everyone is able to switch right away. So as someone using Java 7, I'm curious whether upgrading is on your roadmap or what the feeling is towards Java 8 from professional Java developers in general

[stephanos]

@deontologician Well, I'd (almost) give a kidney to be able to use Java 8 right now. Problem is that we run on App Engine which only supports Java 7 - and it looks like this won't change, ever. The new option is ManagedVM, but it's only in beta status, has no SLA and is super slow to develop for (it is based on boot2docker). So we're stuck on Java 7 :(

[deontologician]

@stephanos thanks, that's really useful info

[iboumedien]

Thanks

[scottwleonard]

@deontologician I wanted to add my $.02 to your question about Java 8. We have been using Java 8 for awhile now, and there are a ton of useful features. streams are very well implemented from a syntax perspective, and they have allowed us to write our code in a more functional way. We didn't have any issues converting our codebase over from Java 7.

[deontologician]

Thanks @scottwleonard, that's helpful. I'm a fan of Java 8, I think (despite being Oracle) that they did a good job integrating functional language features into the existing language in a way that is really nice to work with.

[gmethvin]

I'd also like to see good Scala driver support but once the low level stuff is officially supported in a Java driver I think it'd be fairly easy to write a small Scala layer on top of it with a nice DSL and maybe some additional type safety.

For the Java driver, ideally the serialization framework should be pluggable, and it'd be good if cursors implemented reactive streams so they'd easily interoperate with Play iteratees and Akka streams.

[danielmewes]

@gmethvin Thanks for the suggestions.

I think we'll first make sure that we ship a Java driver on par with our other official drivers and with a comparable interface that will be based on synchronous cursors.
Once we have that working reliably, I suspect that adding a reactive stream API is going to be fairly easy from what I can tell (I had only a very brief look at https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.0/README.md , so there might be some complications that I missed).

@deontologician do you have additional comments on that?

@gmethvin What is your use case for a pluggable serialization framework? Are you talking about the JSON encoder/decoder, the connection logic, or some higher-level API?

[deontologician]

I don't have much to add except that it will take me a bit of research to find out what the best approach is for async etc. The Java ecosystem is really big, so it might make sense to do something like the python driver where the async library is swappable.

[gmethvin]

@danielmewes I'm talking about the JSON serialization. It would be nice if I'm already using Jackson, Gson, Play Json, etc., to easily write an adapter to serialize/deserialize my objects.

[danielcompton]

I maintain the Clojure RethinkDB library, and would be very interested in using an official RethinkDB Java driver for connection management and cursors, and wrapping/adapting this to be idiomatic in Clojure.

[gmethvin]

Also, the type system should try to prevent you from doing things that are obviously wrong. It shouldn't be possible to have a runtime error like Indexed order_by can only be performed on a TABLE or TABLE_SLICE since the type system prevents you from calling the method.

[danielmewes]

I'm quite sceptical about putting anything but trivial type constraints into the driver.
It adds additional code and increases the maintenance burden, since we would need to update the driver when adding new type signatures to ReQL.

[igorlukanin]

@mattias800 There're also tests with usage examples for the POJO stuff mentioned by @deontologician.

@deontologician I bet @tyth would really like if we make r.table(...)'s signature not just r.table(Object tableName), but provide a more strongly typed r.table(ReqlExpr tableNameExpr) among with probably more frequently used r.table(String tableName) which would just wrap the string with the appropriate ReqlExpr.

I think there's no harm in doing this in means of maintainability, and it could be done anytime after 2.2 (kind of 1.0 for Java driver) is released.

[mattias800]

Nice! Gonna try it out tomorrow!

[pires]

@crockpotveggies and everyone else, just be aware that the artifact available on OSS Sonatype atm is dated Oct 26th and is outdated when compared to current code. Better build manually and install it locally.

[dklotz]

@mattias800

Having it return instances of your own classes would be a great feature though.

This (in essence: ODM / object-document-mapping) could be compared to e.g. what Morphia is doing for MongoDB on top of their normal java driver and I would strongly suggest to have it in a library separate from the basic driver. Definitely needed, but definitely a different beast than the basic DB driver.

[deontologician]

OK guys, I uploaded a new snapshot. We've also merged the driver to next (the main development branch) so things are coming along!

[crockpotveggies]

Nice work! Unless someone found a fix, I think compatibility with Scala case class is currently broken, so not sure I can test the new version. It will require some thought around reflection since case classes are structured a little differently.

[pires]

@deontologician 👏

[radicaled]

I've noticed that the POJO mapping code intentionally ignores fields that are inherited.

So, simple hierarchies like this fail:

class Model {
  // too lazy to write out getter / setter, pretend I did
  public String id;
}

// id will always be null if you pass this class, like so: `...run(connection, Person.class)`
class Person extends Model {
  // too lazy to write out getter / setter, pretend I did
   public String name;
}

Is there any particular reason why the POJO mapper does this?

[pires]

I've also noticed it doesn't support Enum.

[kclay]

I would suggest  looking into Jackson for POJO support there are a lot if corner cases as well if the intended use was to also be able to support Scala case classes as well. I ended up using Jackson for the rethink-scala library and it worked out pretty well.
On Nov 21, 2015 6:12 AM, Paulo Pires notifications@github.com wrote:I've also noticed it doesn't support Enum.
On Nov 21, 2015 11:33, "Arron Washington" notifications@github.com wrote:

I've noticed that the POJO mapping code intentionally ignores fields that
are inherited.

So, simple hierarchies like this fail:

class Model {
public String id;
}
// id will always be null if you pass this class, like so: ...run(connection, Person.class)class Person extends Model {
public String name;
}

Is there any particular reason why the POJO mapper does this?

—
Reply to this email directly or view it on GitHub
#3930 (comment)
.

—Reply to this email directly or view it on GitHub.

[deontologician]

@kclay you may be right. I opened #5128 to look at what would be required

[berndwarmuth]

When I try to call insert in parallel I noticed that the driver crashes with a "java.lang.OutOfMemoryError" Exception. This happens as soon as more than one calls to insert are fired in parallel. Any suggestions how to fix this?

Here is the stacktrace:

java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
Caused by: java.lang.OutOfMemoryError: Java heap space
at com.rethinkdb.net.Util.leByteBuffer(Util.java:25)
at com.rethinkdb.net.SocketWrapper.recvall(SocketWrapper.java:100)
at com.rethinkdb.net.ConnectionInstance.readResponse(ConnectionInstance.java:74)
at com.rethinkdb.net.ConnectionInstance.readResponse(ConnectionInstance.java:58)
at com.rethinkdb.net.Connection.runQuery(Connection.java:156)
at com.rethinkdb.net.Connection.run(Connection.java:204)
at com.rethinkdb.ast.ReqlAst.run(ReqlAst.java:66)

kind regards,

[bchavez]

When I ported over the Java driver code to C#, I noticed some internal data structures were not thread-safe in C# land. All the threading issues are resolved C# driver & documented in this internal design note:

bchavez/RethinkDb.Driver#15

Although, this design note is for C#, it might help identify thread-sensitive spots in the Java driver since both drivers share the same architecture.

[danielmewes]

@triggetry I think @bchavez is right that this is likely a threading problem. For now I recommend using one connection per thread.

[danielmewes]

Opened #5166 to add thread-safety to the Java driver.

[a1russell]

It's released! Yay!

https://www.rethinkdb.com/docs/install-drivers/
http://rethinkdb.com/blog/official-java-driver/

[deontologician]

This is fixed now.

[mglukhovsky]

Great work, @deontologician! ☕

[crockpotveggies]

Just wanted to say nice work to everyone who got this developed. Not sure if Scala support made it for the release, however it's very meaningful to the JVM community :)