Migrate the scripts in S3 to use `script._id` instead of the path value

[Martii]

Currently we are unable to upgrade the aws-sdk dependency because "something" changed that makes getObject to fail.

This has been explained in detail at aws/aws-sdk-js#430 . With all the test deploys done trying to figure out what the steps to reproduce are it appears that normalizing the casing when we save to the DB is an option.

Looking for more options related to advancing the goal of updating the dep... e.g. perhaps we could validate the current aReq.params.username against the real username (name) that is typically in the Raw JSON Data for an account. e.g. a lookup to correct the casing around here... but I suspect this is going to need to be a DB migration with a BLOCKING label until it is finished.

See also:

• Rethrown error when streaming aws/aws-sdk-js#397 (comment) error trap suggestion for loosely related
• https://github.com/OpenUserJs/OpenUserJS.org/blob/71ef808762ccf14a048c8a639d6d620e4e4e91e4/controllers/script.js#L350 Probably related and should be fixed

Test points:

• https://openuserjs.org/scripts/Marti/oujs_-_Meta_View/source vs https://openuserjs.org/scripts/marti/oujs_-_Meta_View/source e.g. manual url entry in the addy bar
• https://openuserjs.org/scripts/JRI/Geocaching_Map_Enhancements/source Since jri is currently in the DB not JRI
• https://openuserjs.org/users/jri (http://localhost:8080/users/jri with local pro)

[Martii]

With #487 we have another option of inline live migration... that patch currently fixes the net timeout in dev when selecting some other script source with fakeS3 and it usually just spins the browser spinner for an incredibly long time since our local dev DB doesn't usually have that source.

We can either correct the casing to what the user entered, if applicable, (e.g. a quick user search in the DB and if it is a hit compare it to the username portion generated from installName and migrate) from what we store in the Raw JSON Data or we can just lowercase migrate everything username wise... It is probably not recommended that we should alter the scriptname value though in casing with the Key.

Updated the above for some test points and other linkage.

[Zren]

Normally you wouldn't base the filename/keys based on the username/scriptname. Create an id for each source code and store it in the Script model. If we wanted to future proof ourselves, we wouldn't use the Script._id either as that means only 1 source code per Script. But since we're not versioning it atm, we could just use that.

We have to fetch the Script model first before we doing any AWS stuff anyways.

Edit: Also noticed this when I tried to refactor the scriptEditSourceCodePage that never got finished. Most of the callbacks need to follow the callback(err, ...) pattern so we can chain error up to the route handlers.

[sizzlemctwizzle]

I agree with @Zren. We should use Script._id instead since we have to do a lookup anyway.

[Martii]

Create an id for each source code and store it in the Script model.

Yah I'm not entirely sure how lower casing a Unicode character would affect things so a unique id would probably be best. EDIT: Searching will need to be retested for this as well.

[Martii]

Just a FYI here's a short snippet from recent production log with the isPro console message I added in to check for this on pro e.g. the version of aws-sdk we are currently using hides/mishandles the error and possibly returns an empty payload as one of the maintainers stated... jri's script does show source and does install.

[12/08 23:36:21 EST][err] S3 Key Not Found jri/Geocaching_Map_Enhancements.user.js
[12/08 23:40:33 EST][out] Group(imgur) Rating Updated
[12/08 23:45:24 EST][out] Group(deviantART) Rating Updated
[12/08 23:48:10 EST][err] S3 Key Not Found mienaikage/OkChoices.user.js
[12/08 23:48:53 EST][out] Group(Subtitles) Rating Updated
[12/08 23:49:42 EST][out] Group(Steam) Rating Updated
[12/08 23:49:42 EST][out] Group(dota2lounge) Rating Updated
[12/08 23:49:47 EST][out] Group(download) Rating Updated
[12/08 23:51:02 EST][out] GitHub client authenticated
[12/08 23:51:10 EST][out] Group(bookmarks) Rating Updated

---

Hmmm their clock appears to be a bit skewed too.

[Martii]

Hmmm... retested this randomly with aws-sdk@2.1.25 and doesn't appear to crash dev nor local pro anymore... not sure on actual production. All prior test points appear to be okay (without crashing local pro). :\

@sizzlemctwizzle
Any objections to trying it on production? I'll give this a day or so for any responses.