Media Manager Design

Design

https://g0v.hackmd.io/C8dW2cFiR1-N5Z0wcOefuA#Processing-images

insert()

Under the hood, instead of just downloading the whole file from the given URL and process it, all processing of insert() is done in a stream fasion. When insert({url}) is called, it will take the following steps in practice:

1. Get the NodeJS readable stream to read the URL
2. Connect the readable stream to to all variant transformers, and a temporary location on GCS.
3. Connect the readable stream to file identifier that consumes the file and generates its media entry ID.
4. Wait until the ID is generated. Resolve insert() promise as soon ID is generated.
5. Check if the media entry with the ID already exists on GCS.
   • If so, delete the temporary files.
   • If not, move the temporary files to the folder of the media entry.

Experiments

https://g0v.hackmd.io/LHhF_VQ1RdS12C0k0ESQ_Q https://g0v.hackmd.io/lkGJU4TpQ7evxDoXYK1rtQ#%E5%9C%96%E7%89%87-hash-%E6%95%88%E6%9E%9C