Organisation of MLJ interfaces

[ablaom]

In response to query about competing algorithms: Competing algorithms should be fine and they can even have the same name, which one handles by only importing the relevant packages:

import BobsTrees
import JanesTrees

model1 = BobsTrees.TreeRegressor()
model@ = JanesTrees.TreeRegressor()

However, this doesn't work with lazy loaded models (as currently implemented) because import DecisionTree (with DecisionTree begin lazily loaded) imports all the DecisionTree models into the global namespace without the qualifying package name. I don't know how to fix this unwanted behaviour, except to have something ugly like:

import DecisionTree    # lazily loads interface module _DecisionTree

model = _DecisionTree.TreeRegressor

I suppose an alternative to lazy loading would be to have a separate repo MLJInterfaces.jl which implements the MLJBase interface for all models that do not natively implement it (currently none do). (This is what IterableTables does for the the Query TableTraits interface). The workflow for using a model in a package that does not natively support the MLJ API is then identical to the normal one, except for one step. First import MLJInterfaces which will load all the interfaces (and dependencies) of those packages.

[ablaom]

earlier discussion of lazy loading

[tlienart]

Packages for interfaces is a good idea I think. And they would have their own round of tests as well which could allow pinning versions that we're sure work.
It might make sense to start a separate MLJ github organisation then so that all these can be found in one spot though (the alan-turing-institute github is really quite messy).

(And if the ATI in its wisdom would like to keep its name on things, we could move MLJBase and all subsidiray packages to the organisation and keep MLJ here...)

[fkiraly]

Ah, but we could use teams, no, @tlienart ? I.e., create an MLJ team and give it ownership of MLJ related repositories?

Also, we do no longer call ourselves "ATI" since this is used by the "Aerospace Technology Institute" as well as by a defunct semiconductor manufcaturer.

[tlienart]

Teams sound good though I haven't used it before. And ok for the name 😁 .

[fkiraly]

I think we need to ask @jamespjh for a team.
Last time I chatted policies for team and user right management were being worked out, so there might be a process now.

[ablaom]

Okay, so the plan to create a new repo - tentatively named MLJModels - to reflect the following new organisation:

MLJBase - imported by any code wanting to implement the MLJ API for some algorithm; contains just enough for this purpose plus some convenience methods (e.g., for dealing with tabular data). Not for general MLJ user.

MLJ - user's point-of-entry and the package with all the meta-algorithms and some built-in transformers (e.g., one-hot encoding) and maybe a few model implementations for testing.

MLJModels - a package to be imported by the general user after MLJ if they need a model from a package that does not natively implement the MLJ API. To be made progressively redundant as it's content makes it's way to such packages via PR's.

Please share comments on the name "MLJModels"" soon on #mlj slack. (I prefer this name to MLJInterfaces, which might be mysterious to the general user). Name to become final Thu 31st, 10.30pm, London time.

Could @fkiraly and @sjvollmer please determine the possibilities for an umbrella repo (is that what "teams" are?) and report back. This should be sorted before we register any packages for least pain. It would be convenient to have MLJBase.jl registered soon.

[ablaom]

Now implemented.