Converting Ref<?>s to @DocumentField(type = FieldType.Text)

[tj-recess]

Can doco deal with Ref and Key fields automatically? If not, how should I specify that values need to be fetched and stored in document?

[tj-recess]

I think we can allow users to specify a converter function for any kind of field which translates field value to text. This will not only be useful for Key and Ref but also for types like Set, List etc.

For Set and List, a simple invocation of toString() will give us TEXT FieldType but it won't work for Ref and Key.

Thoughts?

[oliverhausler]

IMO Ref/Key and Objectify integration should not be extendable/configurable. Objectify is "the" library for datastore, and doco wants to be the same for the document store [great idea BTW], so the two should work together. If I use @id and @documentid I would like Doco see convert the Ref/Key without any additional code necessary. Objectify can convert from/to string and I would think this is unique.

A converter function would be nice for other datatypes, though, similar to Google's endpoint transformers.

[tj-recess]

Converting Document back to POJO isn't really much useful IMO because not all the fields are saved in Document so reconstructed Entity will be always have partial information. If we move towards integration with Objectify, then just we should just retrieve back the Key from index.search() and fetch entire Entity using ofy().
Also, configuring Key to DocumentId automatically is very useful. I chose alternate route without depending on Objectify in #6.

[oliverhausler]

I agree, I did not want to say that there should be an integration between the two, but the same terminology and syntax should be used, so they go well together.

Without looking at your code, let me comment about this:

Converting Ref<?>s to @DocumentField(type = FieldType.Text)

Why do we need to specify the type? If Ref<> is String then we have text, if Ref<> is Long/long then we have number/numeric. I think I can say I don't care how my Ref<> is stored, as I only need to restore it to get my datastore entity back.

I would suggest fixed conversions between datatypes and use type= optionally. Why let the developer make a decision they cannot make? I would prefer that doco made the decision for me.

[tj-recess]

I would suggest fixed conversions between datatypes and use type= optionally.
Agree.

Any thoughts on integrating with ofy() and fetching entity using key?

[oliverhausler]

This question cost me two hours tonight ;-) and I am not sure if I should write about it as most of the things that came to my mind are just thoughts and speculations. But let me share them with you:

I think I would probably not tightly integrate because you would have dependencies on objectify. In my opinion, the Java POJO entity must serve as common ground between objectify and doco, and not an integration between the two. Integration should always go via the entity (or objectify keys, which you probably should use to build on).

So a call would look something like this (pseudo-code):

List<Key> keys = doco().find("searchterm").keys().list(); // returns keys
List<Entity> entities = ofy().load().keys(keys).list(); // get-by-key

Alternatively I would also want to be able to store and retrieve the full entity from doco, without using objectify.

List<Entity> entities = doco().find("searchterm").entities().list(); // returns full entities

This would work as long as the Id is the same for both systems. As long as keys can be used interchangeably in both systems, the developer can switch back and forth between them. The developer could also persist using doco from the objectify @onsave lifecycle event.

Let me tell you a few things that came to my mind:

• I would not let the user choose how to save a key in doco. Just like in objectify, you should decide what's the best possible approach and then set it forth as convention. Then call it "key" in the document no matter what the field is called (same as in datastore/objectify).
• I would also keep things simple for other datatypes. Decide how you convert datatypes and then set it forth as convention. If you want to implement type-converters, expose an interface, but don't mess with it yourself as it would make your project heavy and nobody would want to use it. I would guess custom type converters are needed in less than 5% of use cases.
• I would make shorter annotations and keep them similar to objectify. @id -> @docid, @Index -> @docindex, @ignore -> @DocIgnore. This keeps the learning curve low and helps foster adoption. Probably I would even honour objectify annotations if no contrary @doc annotations are present (and if this doesn't require dependency on the objectify library). So the developer needs to annotate only once.
• I would keep the fluent interface from objectify. Probably, what I would do if this was my project, I would fork objectify and then change everything for the document store, instead of the datastore.

I would love to work on this but my time doesn't allow, so sorry. It's a beautiful project and a great approach to make things easier for developers in Google cloud. doco is a great name, too.

[tj-recess]

I wrote some code to add all fields of referenced entity (Ref) to search document with their name as Foo_. All user needs to do is annotate Ref entity with @documentref(type=Foo.class) annotation.
For this piece of code, I've added dependency on Objectify. Check out the diff here : tj-recess@6b6c0cc

I can create a pull request if everyone thinks that this is the right way to go. I'm using this in my own code and it works pretty well for me.