Meteor 3.0: Adding Async Methods to server #884

bratelefant · 2023-12-21T20:18:35Z

This is a first try on async methods as a PR draft. Updated eslint and mocha testing as well; core.js and cursor.js were a starting point. Work is progressing on server.js.

Related to #865

@Prinzhorn

📦 v2.2.0 __Breaking Changes__ - ⚠️ Changes in `namingFunction`, — now naming function acts the same on the Client and Server, upon insert, load, and write. Test your implementation with changed logic. Output of Server function supersedes Client's function output __Changes__ - 📔 Merge veliovgroup#843 and fix veliovgroup#820, thanks to @Prinzhorn - 📔 Documentation refactoring focused on examples and its simplifications - 👨‍💻 Support nested custom path returned from `namingFunction` - 👨‍💻 Fix `namingFunction` behavior on Client and Server in upload, load, and write methods, closing veliovgroup#842; Thanks to @chrschae - 👷‍♂️ Now library exports its helpers `import { FilesCollection, helpers };` - 👷‍♂️ Add `.meteorignore` to minimize package's footprint

@chrschae

📦 v2.2.1 - 👨‍💻 Fix veliovgroup#842, a newly detected bug by @chrschae; Fixing case when `namingFunction` returns new nested path cause exception in `.write()` and `.load()` methods

@xet7

📦 v2.3.0 __New features:__ - ✨ `opts.sanitize` method, read more in [*Constructor* docs](https://github.com/veliovgroup/Meteor-Files/blob/master/docs/constructor.md); Thanks to @xet7 and @mfilser __Other Changes:__ - 👷‍♂️ Minor codebase enhancements and cleanups

@Leekao

v2.3.1 __Changes:__ - 👨‍💻 Improve `createIndex` helper - 👨‍💻 Improve error output when FileSystem destination not writable; Related to veliovgroup#857, thanks to @Leekao - 🐞 Fix custom `allowedOrigins` option for CORS; Closing veliovgroup#850, thanks to @djlogan2; __Notes:__ - 👨‍🔬 Tested with latest release of `meteor@2.8.0`

📔 Improve AWS S3 documentation

📦 v2.3.2 - 👨‍💻 Potential fix for veliovgroup#857 (windows)

👷‍♂️ Update .eslintrc

@Gobliins

📦 v2.3.3 __Major changes:__ - no __Changes:__ - 👨‍🔧 Fixed veliovgroup#870, thanks to @Gobliins - 🤝 Compatibility with `meteor@2.11.0`

bratelefant · 2023-12-27T22:26:18Z

Hi guys, please comment on this first attempt. In order to better understand what's going on I decided to add some tests for the server part, FilesCollection, core and cursor. Unfortunately this adds overall dependencies, especially in package.js; if anyone knows how to limit these dependencies to dev / testing, pleas let me know. Also tweaked the .eslintrc a little bit. Core and cursor now additionally have async methods. Work on server.js is in progress.

I suggest that the async versions of all the methods do not use any callbacks but throw errors instead and return promises resolving the results, cf. FilesCollection.addFileAsync for example.

Which callbacks need async versions? E.g. for my project, I need an async version of protected(), since I do policy checks against the db here, which will be async as well of course. I guess an async version onAfterUpload could be helpful, since the aws sdk v3 for example also uses async functions to send putObject requests etc.

dallman2 · 2024-03-24T19:11:49Z

EDIT: Do any of you guys use Cordova? I run into a couple of issues when async changes paired with Cordova.

We don't, unsure of others in the thread. What issues are you having?

harryadel · 2024-03-25T08:03:26Z

server.js

@@ -22,6 +29,7 @@ import nodePath from 'path';
 const bound = Meteor.bindEnvironment(callback => callback());


Just out of sheer curiosity, is there still a need for Meteor.bindEnvironment? Meteor 3.0 no longer uses fibers/futures.

@harryadel good catch, it should be safe to remove it now.
@bratelefant wdyt?

dr-dimitru · 2024-03-25T19:48:44Z

@harryadel please give a freshly published v3.0.0-beta.5 a try. Issue from your screenshot should be solved now. LMK

harryadel · 2024-03-26T05:37:14Z

@dallman2 Still trying to pint point the root cause, though I figured to share my problems in case someone else ran into them.

Thank you Dimitry @dr-dimitru

dallman2 · 2024-03-26T05:39:41Z

Will try this update tomorrow and let you know how it goes (I'm on Pacific coast time :P). I've been deep into css modules all day so I'm going to have to shift a bit to deal with the files collection.

dallman2 · 2024-03-28T22:31:38Z

No issues for me using beta.5. That is to say, my app starts and runs. Have not done totally comprehensive testing.

harryadel · 2024-04-29T07:14:37Z

@bratelefant @dr-dimitru Maybe it's time to release a new beta that relies on the latest RC instead of beta?

jankapunkt · 2024-04-29T09:15:46Z

A beta release would be awesome so I can do some integration tests with our projects and give you early feedback

dr-dimitru · 2024-04-29T10:19:42Z

@harryadel @bratelefant @jankapunkt 3.0.0-beta.6 is out with support for meteor@3.0.0-rc.0

harryadel · 2024-04-29T10:36:07Z

Thanks for acting so quickly @dr-dimitru 👏

harryadel · 2024-05-21T14:55:10Z

Can you make onBeforeUpload on the client side async please?
https://github.com/bratelefant/Meteor-Files/blob/69cddf91c6432ac872d70218949d9537a143d667/upload.js#L688

onBeforeUpload is an isomorphic method that runs on both server and client so sync operations would only work on client side and fail on serverside on 3.0 but async operation works on both. So, I suggest to make all the callers of isomorphic methods support async functions.

cc @bratelefant @dr-dimitru

make-github-pseudonymous-again · 2024-05-23T14:13:33Z

Can you make onBeforeUpload on the client side async please? https://github.com/bratelefant/Meteor-Files/blob/69cddf91c6432ac872d70218949d9537a143d667/upload.js#L688

onBeforeUpload is an isomorphic method that runs on both server and client so sync operations would only work on client side and fail on serverside on 3.0 but async operation works on both. So, I suggest to make all the callers of isomorphic methods support async functions.

cc @bratelefant @dr-dimitru

I think in general all hooks should be async. Any of them might require interacting with the database which is now something that is async by default.

make-github-pseudonymous-again · 2024-05-23T16:14:36Z

@bratelefant My only pain point so far with the current state of the PR is the lack of documentation: took me some time to figure out that the callback version of Meteor.write was gone. The error you get by trying to use the old signature on the new one is just a timeout, if you are lucky to have configured one (tests). This PR could have been done without this change, right? If we are aiming at a new version of ostrio:files without any callback-style functions, then we should also change the client signatures (insert, etc.).

bratelefant · 2024-05-23T16:32:37Z

Can you make onBeforeUpload on the client side async please? https://github.com/bratelefant/Meteor-Files/blob/69cddf91c6432ac872d70218949d9537a143d667/upload.js#L688

That'd be definitely one major point to add to this pr. Did all changes with focus on the server side, client only methods did not get touched.

onBeforeUpload is an isomorphic method that runs on both server and client so sync operations would only work on client side and fail on serverside on 3.0 but async operation works on both. So, I suggest to make all the callers of isomorphic methods support async functions.

Yes, in our discussion here at some point we decided to make this an update with breaking changes by removing all sync methods on the server side.

@make-github-pseudonymous-again I tried to keep documention in the server side method comments aligned with the code changes, but that was definitely not checked agains tests / code.

I would be really grateful if someone could jump on the bandwagon and contribute to the PR, I am having a pretty 'crunchy' time at work atm

make-github-pseudonymous-again · 2024-05-24T07:01:06Z

server.js

+    let _id;
+    try {
+      _id = await this.collection.insertAsync(helpers.clone(result));
+      try {
+        await this._preCollection.updateAsync({_id: opts.fileId}, {$set: {isFinished: true}});
+        if (_id) result._id = _id;


Is it conceivable that this id-setting logic sometimes fail? I have an existing test that yields fileRef._id = undefined in the on('end') handler of FileCollection#insert (client-side call)`.

onUploaded is even worse: the fileRef is simply undefined.

@bratelefant do you think we should address this as part of this PR?

I'm getting similar undefined results after an upload via

ContactFileCollection.insert({ file: file, }).on('uploaded', (uploadError, file) => { console.log('===== uploaded', uploadError, file)

…oreUpload

StorytellerCZ · 2024-09-25T15:22:37Z

Any update on this?

harryadel · 2024-09-26T09:35:23Z

Hello, I created a PR to allow us to use async onBeforeUpload bratelefant#2

Make onBeforeUpload async compliant

hluz · 2024-10-08T21:45:09Z

Hi everyone... Not trying to be pushy, but this is one of the few remaining hurdles in migrating to Meteor 3 for many of us. Even if it was only a new beta version adding the async onBeforeUpload changes would help... @dr-dimitru @StorytellerCZ @jankapunkt

wolasss · 2024-11-14T10:56:06Z

Hey, any updates on this ?

dr-dimitru and others added 25 commits June 29, 2022 16:08

Merge pull request veliovgroup#845 from veliovgroup/dev

313e842

📦 v2.2.1 - 👨‍💻 Fix veliovgroup#842, a newly detected bug by @chrschae; Fixing case when `namingFunction` returns new nested path cause exception in `.write()` and `.load()` methods

Merge pull request veliovgroup#859 from veliovgroup/dev

df7a6c9

📔 Improve AWS S3 documentation

Merge pull request veliovgroup#860 from veliovgroup/dev

4f58852

📦 v2.3.2 - 👨‍💻 Potential fix for veliovgroup#857 (windows)

Merge pull request veliovgroup#864 from veliovgroup/dev

b312ef9

👷‍♂️ Update .eslintrc

Merge pull request veliovgroup#871 from veliovgroup/dev

49f3bed

📦 v2.3.3 __Major changes:__ - no __Changes:__ - 👨‍🔧 Fixed veliovgroup#870, thanks to @Gobliins - 🤝 Compatibility with `meteor@2.11.0`

implement async methods

93b7a73

getLink async

a6c6da3

lint fix

926e8e2

fix Match.OneOf etc lint rule

c811a75

tiny tests npm script

b37aba8

added some mocha testing

f7bbba1

script for mocha

4af4af6

Merge remote-tracking branch 'original-meteor-files/dev'

aac6555

tests

e502124

few more tests

91fbf48

link() is not async

c7fe2a3

more tests, remove tests fails

5c31934

fixed tests

743c12f

rest of cursor tests

ff30780

few more tests

1ea4e38

Add devDependencies and update removeAsync method

bf63873

addFileAsync

e727ae8

bratelefant added 4 commits December 28, 2023 01:57

test for download

3edc652

downloadAsync tests

f97b4a9

prepareUploadAsync

497e8c2

_finishUploadAsync and tests

2dfa4de

harryadel approved these changes Mar 25, 2024

View reviewed changes

harryadel mentioned this pull request Apr 1, 2024

[WIP] Meteor v3 Meteor-Community-Packages/meteor-collection-hooks#309

Merged

bratelefant mentioned this pull request Apr 18, 2024

User can not be determined in protected callback in Firefox #888

Open

make-github-pseudonymous-again reviewed May 24, 2024

View reviewed changes

dr-dimitru changed the title ~~Adding Async Methods to server~~ Meteor 3.0: Adding Async Methods to server Jun 28, 2024

make-github-pseudonymous-again mentioned this pull request Jul 5, 2024

Upgrade Meteor to 3.0 infoderm/patients#995

Draft

donstephan mentioned this pull request Jul 29, 2024

[3.0.0-beta.6] fileObj._id on upload end ? #889

Open

bratelefant and others added 3 commits August 7, 2024 11:47

return fileObj in uploaded and end event

d573741

Initial commit

22969a0

Merge branch 'master' of github.com:harryadel/Meteor-Files into onBef…

ed9e253

…oreUpload

Merge pull request #2 from harryadel/onBeforeUpload

0d6707e

Make onBeforeUpload async compliant

hluz mentioned this pull request Nov 5, 2024

Version 3.0.0-beta.6 onBeforeUpload does not support async db calls #892

Open

xet7 mentioned this pull request Nov 27, 2024

Wekan Windows 64bit version On-Premise wekan/wekan#5591

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Meteor 3.0: Adding Async Methods to server #884

Meteor 3.0: Adding Async Methods to server #884

bratelefant commented Dec 21, 2023

bratelefant commented Dec 27, 2023

dallman2 commented Mar 24, 2024

harryadel Mar 25, 2024

dr-dimitru Mar 25, 2024

dr-dimitru commented Mar 25, 2024 •

edited

Loading

harryadel commented Mar 26, 2024 •

edited

Loading

dallman2 commented Mar 26, 2024

dallman2 commented Mar 28, 2024

harryadel commented Apr 29, 2024

jankapunkt commented Apr 29, 2024

dr-dimitru commented Apr 29, 2024

harryadel commented Apr 29, 2024

harryadel commented May 21, 2024 •

edited

Loading

make-github-pseudonymous-again commented May 23, 2024

make-github-pseudonymous-again commented May 23, 2024

bratelefant commented May 23, 2024

make-github-pseudonymous-again May 24, 2024 •

edited

Loading

make-github-pseudonymous-again May 24, 2024 •

edited

Loading

dr-dimitru Jun 28, 2024

67726e Aug 27, 2024

StorytellerCZ commented Sep 25, 2024

harryadel commented Sep 26, 2024

hluz commented Oct 8, 2024

wolasss commented Nov 14, 2024

		@@ -22,6 +29,7 @@ import nodePath from 'path';
		const bound = Meteor.bindEnvironment(callback => callback());

Meteor 3.0: Adding Async Methods to server #884

Are you sure you want to change the base?

Meteor 3.0: Adding Async Methods to server #884

Conversation

bratelefant commented Dec 21, 2023

bratelefant commented Dec 27, 2023

dallman2 commented Mar 24, 2024

harryadel Mar 25, 2024

Choose a reason for hiding this comment

dr-dimitru Mar 25, 2024

Choose a reason for hiding this comment

dr-dimitru commented Mar 25, 2024 • edited Loading

harryadel commented Mar 26, 2024 • edited Loading

dallman2 commented Mar 26, 2024

dallman2 commented Mar 28, 2024

harryadel commented Apr 29, 2024

jankapunkt commented Apr 29, 2024

dr-dimitru commented Apr 29, 2024

harryadel commented Apr 29, 2024

harryadel commented May 21, 2024 • edited Loading

make-github-pseudonymous-again commented May 23, 2024

make-github-pseudonymous-again commented May 23, 2024

bratelefant commented May 23, 2024

make-github-pseudonymous-again May 24, 2024 • edited Loading

Choose a reason for hiding this comment

make-github-pseudonymous-again May 24, 2024 • edited Loading

Choose a reason for hiding this comment

dr-dimitru Jun 28, 2024

Choose a reason for hiding this comment

67726e Aug 27, 2024

Choose a reason for hiding this comment

StorytellerCZ commented Sep 25, 2024

harryadel commented Sep 26, 2024

hluz commented Oct 8, 2024

wolasss commented Nov 14, 2024

dr-dimitru commented Mar 25, 2024 •

edited

Loading

harryadel commented Mar 26, 2024 •

edited

Loading

harryadel commented May 21, 2024 •

edited

Loading

make-github-pseudonymous-again May 24, 2024 •

edited

Loading

make-github-pseudonymous-again May 24, 2024 •

edited

Loading