Usecontribdjango

[sum12]

I am not entirely sue this is how the code should be organized.
Or even if the code should even go here.

[alanjds]

Well, this is a nice hack.

It will work until Django's implementation changes. Should not occur a lot, but will need to be tracked.

I vote for being accepted after polished a bit, but keep pursuing a way to transfer client-side functions code to serverside.

[otoolep]

I need more context here -- I don't really understand what this is about.

Also, I'm unlikely to add support for a particular ORM such as Django. Not ruling it out, but I prefer to keep rqlite focused.

[sum12]

Let me give you more context on why a PR to rqlite.

sqlite3 has a mechanism to define custom functions and use them while running queries.
example: django_extract_year
select * from expenses where django_extract_year(spent_on)=2017

So now I have the list of all the expenses from 2017.

both sqlite3 module from python and gp-sqlite3 have api to register these custom functions.
go-sqlite3 expects param Connection_hook which is a function. The functions receive a param *sqlite.SQLiteCon. This params has a method create_function which can be used register custom functions.

Here is an example of how it is done https://github.com/mattn/go-sqlite3/blob/master/_example/custom_func/main.go.

Because sqlite3 has this ability to have custom functions. There would be other ORMs who would have such custom functions. And to use these ORMs with rqlite one will need those functions.

I agree that rqlite is not necessarily the place to hold these extra functions. But since rqlite uses go-sqlite3, rqlite should have mechanism to either register these functions "Dynamically" (I dont know if that is even possible in golang) and/or have them in contrib(folder?/repo?) and register when asked.

Does that help in understanding or is further confusing ? :-)

[alanjds]

I understand the need to "bring the code along" because Golang does not allow dynamic bindings usually, but narrowing to Django needs is wrong.

The Right Way is to have a db-side programming language. But how?

Talking with @sbneto we thought about wire-transfering the code to the servers and compile + link locally, but is hard in Go.

If something like a Lua runtime got embedded, at least the ORM could send its functions ahead of compilation time, without needing the whole compilation toolchain on the server.

[alanjds]

Another decision: should this PR be accepted as a keg-leg until a db-side language got integrated, just to support at least one ORM? Is this a good compromise?

I do love to see Django and RQLite working, but I an undecided.

[otoolep]

What is a "keg-leg"?

[alanjds]

@otoolep: Is a weird word for "handicapped by too much beer last night". Sorry for that.

In the context, the idea is intended to be of "so ugly that should be madness to be merged and will be replaced very soon".

[otoolep]

OK, thanks @alanjds

I am planning more major work on rqlite for 5.0 in a couple of weeks or so (busy with other stuff right now). So I am reluctant to merge anything that may not make sense (and this code change isn't ready yet). The changes could be significant, including a brand new consensus module.

[sum12]

That sounds exciting. Thanks for planning it. Meanwhile can you share your view on how rqlite should handle these cases of registering extensions and/or functions . Since sqlite3 provides that mechanism i feel rqlite should not shadow that feature simply because it does not have an API. If I may get into details all rqlite has todo is take in a list of extensions which it has to register while creating sqlite connection. And these extensions could reside in a contrib folder. Think of these contrib extensions as low priority util functions which do not affect the functionality of rqlite however they increase/facilitate the environments in which rqlite can be used.

…

On Wed, Jul 25, 2018, 05:55 otoolep ***@***.***> wrote: OK, thanks @alanjds <https://github.com/alanjds> I am planning more major work on rqlite for 5.0 in a couple of weeks or so (busy with other stuff right now). So I am reluctant to merge anything that may not make sense (and this code change isn't ready yet). The changes could be significant, including a brand new consensus module. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#523 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADIhE5pDH6Jo_1Q0Eu-0wtD4HVteLv8-ks5uJ7rzgaJpZM4VOxoY> .

[otoolep]

I would be open to a contrib directory as you suggest. However any code in there would still need to be high-quality, and I would need an HTTP API exposed that would allow people explicitly hook the stuff in.

The connection model is changing is significantly in 5.0, and since this functionality is based on that, anything you do right now is liable to need serious rework. But the general idea you have should work.

[sum12]

Its awesome that you don't oppose the idea. Then how about for now we only get the contrib part in the repo for later as you put in the HTTP API. If this sounds like a good plan to you. I can modify the PR.

…

On Wed, Jul 25, 2018, 18:48 otoolep ***@***.***> wrote: I would be open to a contrib directory as you suggest. However any code in there would still need to be high-quality, and I would need an HTTP API exposed that would allow people explicitly hook the stuff in. The connection model is changing is significantly in 5.0, and since this functionality is based on that, anything you do right now is liable to need serious rework. But the general idea you have should work. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#523 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADIhE1zz7PJ9f7YSOne-DF5MC_4_oVfvks5uKHASgaJpZM4VOxoY> .

[otoolep]

OK, please generate a PR and I'll take a look -- thank you.

[sum12]

@otoolep I have updated the PR.
Do you have any new reviews for me ?

[otoolep]

Thanks @sum12

Before I merge anything (not sure if that's what you're suggesting now), a few more things need to happen. Commented-out code needs to be removed, tests need to be added, and a README needs to be added.

Alternatively you can wait until I play with what you have now. However it's going to be 3-4 weeks because I have non-programming commitments right now that mean I don't have time for coding right now. :-/

[tomdesair]

I also want to use rqlite in a new Django project. Therefor I wanted to check if this PR is still waiting on the 5.0 release or if it could already be merged in a new 4.x release?

Thank you all for all the effort you've put in rqlite and best regards,
Tom

[otoolep]

I have no plans to merge this at the moment. Right now the goals for rqlite mean I want to keep it quite focused, and not have any ORM or web-framework -specific features.

[alanjds]

I understand, @otoolep. However, please keep in mind that any application needing DB-side custom procedures will not be able to be ported for now.

I am sure you have not spare hands to do everything. It is just a reminder of the consequences of this decision.

[sum12]

@tomdesair you should be able to use rqlite with django without this PR too.

This PR just add the functionality to call custom time.Time based functions. So rqlite should work just fine if you are not using these special functions in your project

[alanjds]

@tomdesair you should be able to use rqlite with django without this PR too.

I cannot assure that. Without this PR the ORM of Django does not pass the tests for the SQLite backend. You may be able to do something, but YMMV.

[otoolep]

v5 development is now highly speculative, it may not continue in its current form.

Thank you for your contribution though.