Rename export_files() to to_storage()

[dmpetrov]

Description

export_files() needs to be renamed in order to get a consistent naming with from_storages(), to_parque(), to_json() etc. It's also need too support all params acceptable in from_storage() like cloud paths s3://mybkt/dir1/

• Rename to to_storage()
• Support clouds

Open question: should we do a bit opposite - rename from_storage() --> from_files(), export_files() --> to_files() in order to get a consistency with from_parquet(), from_json(). It seems like a cleaner way.

@iterative/datachain WDYT folks?

def predict_dog(File: file) -> float:
   model = tf.keras.applications.ResNet50(weights='imagenet')
   img = tf.keras.preprocessing.image.load_img(io.BytesIO(file.read()), target_size=(224, 224))
   pred = model.predict(tf.expand_dims(tf.keras.preprocessing.image.img_to_array(img), axis=0))
   return sum(pred[0][151:269])

(
    DataChain
    .from_storage("s3://mybucket/data/*.jpg'")
    .map(dog_score=predict_dog)
    .filter(C("dog_score") > 0.96)
    .to_storage("s3://mybucket/out/dogs")
)

PS:

• it's ok to break compatibility with export_file since it always was a workaround

[skshetry]

One thing to keep in mind is that export_files also supports checking out (for a lack of a better term) with different names.

datachain/src/datachain/lib/file.py

Line 41 in 143b4be

ExportPlacement = Literal["filename", "etag", "fullpath", "checksum"]

I don't particularly like the to_storage suggestion. In my mental model, from_storage creates a dataset from a storage. This will have a signal called file, which basically has a pointer to the remote file. But it could have other things besides the file signal, or not have one at all.
to_storage would be something that dumps the index/dataset to the storage.

If I ignore different placement strategies, the current function could be best explained as a pull or a fetch. But if we support clouds, I don't know what to call it - maybe copy or copy_to, or just pull (or, is it push?). 😅