refactor gee_tools

[12rambau]

Dear all,

We recently decided to give gee_tools a spring refresher to make it more stable, useful and fast (see #79 and #96). Multiple tasks are ahead of us but we want to start with a complete refactoring of the lib. The main idea is to transform gee_tools into an extention of the earthengine-api making our function available for all ee objects as methods. To avoid issues with gee updates we decided to hide them behind a namespace "geetools".

Here is a use case example to create a FeatureCollection from a shapefile:

Before	import geetools fc = geetools.batch.featurecollection.fromShapefile("toto.shp")
After	import ee, geetools fc = ee.FeatureCollection.geetools.fromShapeFile("toto.shp")

As the methods are hidden behind the "geetools" namespace they should not conflict with new gee developments nor other extention libs. The exact name of the namespace is not yet completely fixed.

As the amount of methods is collosal, it will take time to migrate everything from v0.x to v1.x but we alrready started in this branch. We realize that some of them are already broken and should thus be dropped and other could be replaced by one liner so maybe don't worth the extra method.

Would you mind telling us in the comments which functions of geetools you are using the most ?

Note
to make it as efficient as possible please create a comment per function as the poll discussion doesn't allo for commenter to add extra answers. if the function already exist, like it.
I will create a small list of links follwing this note to fasten the search.

[mgravey]

Interesting! May I inquire about the rationale behind this change? I rarely use gee_tools, so it's not a pressing issue for me. However, I appreciate package independence and quick readability to understand dependencies.

Reasons to Reconsider:

1. Substitutability: The change contradicts Python's philosophy, as it hampers easy replacement via import geetoolsCompetitor as geetools.
2. JS-Python Mapping: Earth Engine was designed to maintain a near one-to-one mapping between JavaScript and Python. This change disrupts that, making code portability between the two languages cumbersome.
3. Code Adaptation: Existing users will have to modify their code to accommodate this change.

I had contemplated implementing this for the Open Earth Engine Library, but ultimately decided against it after careful consideration.

If you're still convinced this is beneficial, why not employ function mapping? This would allow both the old and new versions to coexist and eliminating the need for a complete library overhaul.

BTW, I like the idea of #79 :)

[12rambau]

the inspiration for me is the eemont and wxee libs. both of them use the accessor mechanic to extend the normal scope of the earthengine-api. In both cases the extension allows the methods to be available for all ee objects. I preferred the wxee approach (using a named accessor, wx) as it prevent competition. Also with the rise of the commercial version of GEE, the methods from gee_tools need to be optimized to avoid getInfo() calls and remain server-side for all GEE outputs (that's not always the case in the current implementation). That's why I don't want to follow the path of eemont to override magic methods (item, add ...) to maintain a visual fence between server-side operation and client-side one.
The name of the current accessor "geetools" is too long I might change it afterward to "gt" but that will be done at the end.

The change contradicts Python's philosophy, as it hampers easy replacement via import geetoolsCompetitor as geetools

Yes that's on purpose because here we want to overload an API that is clearly not Python like. that strategy is commonly used to create xarray plugins.

Earth Engine was designed to maintain a near one-to-one mapping between JavaScript and Python. This change disrupts that, making code portability between the two languages cumbersome.

It's already the case right ? if you are using the current implementation of gee_tools you can't port your code from Python to Javascripts "as is". In a second phase of the refactoring I would like to port the geetools method in a javascript lib. but that's another story (only made possible because we use server-side calls)

Also it allows to do more javascript like chain computation like:

string = (
    ee.FeatureCollection(dicto)
    .toList()
    .geetools.transpose()
    .distinct()
    .geetools.delet(1)
    .geetools.join(",")
)

Existing users will have to modify their code to accommodate this change.

I'm not deleting the previous ones, they are all wired to new implementation and decorated with the deprecated lib: https://github.com/gee-community/gee_tools/blob/1.x/geetools/tools/_deprecated_ee_list.py. They will obviously stay there for a long time.

[mgravey]

But I can create a geetools object in JS, so it's possible. I doubt, however, that the approach you're taking will work in JS, especially in the Code Editor where ee is not extendable. Therefore, your chaining method seems unachievable.

If calling JS from Python is possible, the reverse is likely doable in Node.js but not in the Code Editor. I agree that simple computations in Python can be serialized in python and then deserialized in JS, but direct copy-pasting would probably suffice.

The whole idea behind OEEL is to write in JS and have a way to access and use the code from Python.

I respect your design choices and won't debate them further. I believe I've made my points clear, and the decision is yours to make. After all, progress is better than perfection. We all have design decisions we regret, but only those who take action learn.

By the way, gt stands for "greater than" (e.g., ee.Number, ee.Image) so you might want to consider another abbreviation.
Maybe gtl, gtls, or even tl or tls

[12rambau]

thanks a lot for your feedback, that's super helpful ! I'll dive a bit into the JS/Python compatibility issue that you raised.
I think the main approach difference is that you maximized interoperability, I want to maximize code efficiency for Python developers.

[mgravey]

bon resumé :)