Public User Feedback request - Promises

[benjamingr]

Following discussion in #76 and nodejs/promise-use-cases#26 we want to get some questions going for the Node.js promises usability and debuggability survey.

Going to try something - let me know if it's inappropriate :)

Post questions as comments - use 👍 and 👎 in order to indicate whether you think other questions should be included in the survey or not.

You are welcome to post question suggestions even if you are not Node.js project core. If you would like to consult me before posting a suggestion - feel free to contact me personally at benjamingr@gmail.com

[benjamingr]

What is your primary role in your workplace?

• Developer, I typically am not the person to debug production issues
• Developer, I typically am the person to debug production issues
• DevOps / Operations Engineer, I typically am not the person to debug production issues
• DevOps / Operations Engineer, I typically am the person to debug production issues
• Other (please specify)

[benjamingr]

Which do you use the most often in order to write async code:

• Promises without async/await.
• Promises with async/await.
• Node.js standard (err, data) callbacks.
• I don't use any of the above at all.

[benjamingr]

Do you use async iterators and for await( loops with streams?

[benjamingr]

Which of the following tools do you use in order to debug Node.js code at least once a week:

• The Node.js inspector or an IDE debugger (nodejs --inspect)
• Long Stack traces with an external library like Bluebird
• Printed stack traces with native promises (for example mocha errors).
• core dumps + lldb
• gdb
• Adding console.log around code and running it

[benjamingr]

What should the policy Node.js takes when an unhandled rejection occurs? For example - when an exception is thrown in an async function (or a then handler) but no code handles the error (.catch, then second param or catch in an async function).

• I would expect Node.js to abort the process like on uncaught exceptions.
• I would expect Node.js to stay silent about the error.
• I would expect Node.js to warn about the error. Note: this is the current behavior.
• I would expect something else.

Note: this is only the default and users would still have the ability of code to opt into different behaviour.

[benjamingr]

Do you use promises with async_hooks?

• I do not know what async_hooks are.
• I do know what async_hooks are but am not using them at all or with promises.
• I am using a third-party library that uses async_hooks with promises
• I don't know if I am using promises with async_hooks
• I am using promises with async_hooks in my code

If you answered "I am using promises with async_hooks in my code" to the previous question - please elaborate how?

_Enter answer here_

[benjamingr]

Is the performance of Node.js promises sufficient to you:

• Node.js promises are still slow (even on the latest version) and significant effort needs to be put in before I can use them.
• Node.js promises are slow - and while I am using them I think more time should be spent on optimizing them .
• Node.js promises are fast and more effort should be spent on making them more debuggable rather than faster.
• Node.js promises are fast enough now and I wouldn't mind them being slightly slower if that provides better debuggability features such as async stack traces in production apps.
• Node.js promises are not a limiting factor in my app's performance and I wouldn't mind them being slightly slower if that provides better debuggability features such as async stack traces in production apps.
• None of the above apply.

If you have answered that Node.js promises are slow, please write down why that factor limits your usage of promises:

__Enter answer here__

Note: see this note

[boneskull]

@benjamingr

What is your primary role in your workplace?

What's the necessity of this?

A large proportion of Node.js users are using it for tooling; Node.js may only be used to bundle a site or run tests. Those users may say they aren't using Node.js in production (I would).

Which do you use the most often in order to write async code:

Is this "which do you use most often" or "which do you prefer?"

Do you use async iterators and for await( loops with streams?

Maybe this question could use a code example. 😉

Which of the following tools do you use in order to debug Node.js code at least once a week:

Maybe "IDE or Editor" should be added to this list. I debug from VSCode wherever possible.

Is the performance of Node.js promises sufficient to you:

Users may not have an opinion, not care, or don't know.

But otherwise, unless the user is running Node.js 10, will the feedback be useful? If they complain it's slow, and they're using Node.js < 10, would it be an impetus to backport Promise performance improvements?

(Apologize if some of these have been answered already; I was not in attendance at relevant meeting)

[benjamingr]

@boneskull

What's the necessity of this?

A large proportion of Node.js users are using it for tooling; Node.js may only be used to bundle a site or run tests. Those users may say they aren't using Node.js in production (I would).

To validate that our survey reached a diverse set of developers - and to check the bias of various groups. For example in the meeting some participants mentioned they believe operations engineers use post-mortem debugging techniques differently from developers.

Is this "which do you use most often" or "which do you prefer?"

It is "which do you use most often".

Maybe this question could use a code example. 😉

Not really sure, it's basically "whether or not you have for await in your code or not.

Maybe "IDE or Editor" should be added to this list. I debug from VSCode wherever possible.

👍 added thanks

But otherwise, unless the user is running Node.js 10, will the feedback be useful? If they complain it's slow, and they're using Node.js < 10, would it be an impetus to backport Promise performance improvements?

Well - if people complain they're slow then sure - but there is a possibility they won't. If people complain about performance in older versions that's something to keep in mind as well.

More question suggestions welcome by the way.

[giltayar]

Which of the following tools do you use in order to debug Node.js code at least once a week:

I believe you forgot:

• Regular stack traces with native promises

Which is what I use to debug my tests when I'm not running them under a debugger (looking at the stack trace helps me figure out what the problem is.)

[benjamingr]

@giltayar Regular stack traces with native promises can you show me an example of what you mean exactly? Just looking at hte output of unhandled rejections?

[giltayar]

How about adding a question about the use of domain with promises? While I know they're deprecrated and async_hooks are replacing them, I believe a lot of people still use them as they are higher-level abstraction use than async_hooks.

[giltayar]

This is a stack trace from my mocha test (which was run just now...). I don't use bluebird, and so don't have long stack traces (and I wish native promises had them!), but I still use these stack traces to debug my code.

[benjamingr]

@giltayar

I don't use bluebird, and so don't have long stack traces (and I wish native promises had them!)

Native promises have them with the inspector attached - as a "protip" you can run mocha with the node inspector running which enables long stack traces. Following the Berlin summit the V8 team are prioritizing and working on async stack traces in production. I will add the option - thanks for clarifying.

[justsml]

I thought bluebird's long stack trace support was largely from native functionality Error.stackTraceLimit = 50 or something similar?

[naugtur]

On the async_hooks question, most people aren't aware that a tool like APM or a library is using hooks. A question more towards the goal of hooks with promises might give better results.
Do you track (or wish to) metrics on the time each function in your promise chains runs.

As for the current question, it could use clarification whether it's about just using promises and hooks in the same codebase or something specific about using them together.
I think you're aiming at getting examples of what needs to be supported in hooks, but I wouldn't know how to answer that question while I do plenty of promises and wrote blocked-at npm package using hooks.

[benjamingr]

@justsml no, we actually keep context. V8 does it on their side. stackTraceLimit helps with synchronous stack traces but not async ones.

---

@naugtur thanks for the suggestion - I agree that it's not optimal - I think "Do you track (or wish to) metrics on the time each function in your promise chains runs." will go above peoples' heads a little - I'm very open to suggestions though. Maybe we should ask about the most popular packages with async hooks? (That is, I use a tool such as ....)

[mhdawson]

@yunong it would be great to get your input on these questions.

[mhdawson]

For a number of them "Which of the following tools do you use in order to debug Node.js code at least once a week:"

I think adding an other where people can add their own would be good.

[mhdawson]

I think adding Chrome developer tools as an option to "Which of the following tools do you use in order to debug Node.js code at least once a week:" would be good.

[mhdawson]

@benjamingr what might also help is separately listing out what we want to understand based on the survey. That will help when looking at the questions to see if there are any gaps.

[ktrott]

What is your primary role in your workplace?

At Netflix, everyone is full cycle engineers that are responsible for development and devops.

If your intent is to understand responsibilities, just separate out into explicit questions on each responsibility and whether that is part of their work/role. You could also provide a scale of how frequently or what percent of their role/time is that responsibility.

"Typically" is vague and open to many interpretations. Example: larger teams with longer on call rotations.

On primary role, I find your choices may have some possible implicit bias built in it. Again would focus on responsibilities not titles.

Importantly, what do you mean by "debug"? There are many ways to debug production issues and often depends on the issue. Is that meaningful to you? If so, be explicit, or your responses won't help you as much.

Example:

• I debug issues of Node.js services? Yes/No
• Do these run in production? Yes/No
• How frequently do you debug? Never, Daily, Weekly, Monthly, Yearly
• How do you debug? Multi checkbox: metrics, logs, flamegraphs, coredumps, other (writein)

[ktrott]

Which do you use the most often in order to write async code

Should you have a other writein option? What about Generators or Observables? Is it useful to learn about those cases too?

[ktrott]

Which of the following tools do you use in order to debug Node.js code at least once a week:

Why does it have to be "at least once a week"? Are my usages not valid otherwise? What about "which do you use frequently"

I would allow an other writein.

[joesepi]

Was this discussed in #95 ? I'm interested in helping move this and #96 forward in tandem if that makes sense and is possible.

[joesepi]

Hey everyone @/here. I updated the Google Doc trying to collect the questions there. I am still parsing the comments on the questions though, so if someone wants to help, that would be great. The folks from the Foundation are ready to help us get this out there, so I'd like to get something together ASAP. Thanks! /cc @benjamingr @misterdjules @dshaw @ktrott @justsml @naugtur @justsml @boneskull @giltayar

https://docs.google.com/document/d/1ZwS8ITwsgzcmIdoqFw5jKlx4_C0S1_pY4vqSPSI_Sfs/edit?usp=sharing

[justsml]

Thanks for the update @joesepi - i added minor notes to the google doc.

There's one big thing that would really help people w/ Promises that might fit alongside the unhandledRejection default behavior question...

Should node add an opt-in warning: To always return inside your .then functions. Especially if you are nesting another async call, or using a Promise.reject?

I realize this could be desired behavior (e.g. log something async & ignore its result, while continuing to load app data), however 99% of the time it's a bug that leads to race conditions & some of the nastiest bugs I've encountered.

Example:

// ❌ Should warn that you aren't returning the Promise.reject
getUsers({active: true})
  .then(users => {
    if (users.length === 0) Promise.reject(new Error('No users found'))
    return users
  })

// ❌ Should warn that you aren't returning anything
// This might be perfectly OK functionality (in a CLI tool writing out requested data)
getUsers({active: true})
  .then(users => {
    console.log(users)
  })

This may be the responsibility of a linter. Currently, eslint-plugin-promise struggles to 'see' actual runtime differences.

[misterdjules]

@joesepi That's great, thank you for doing that! I'll try to take a closer look soon-ish, but I don't want to commit to something when my schedule is already very busy.

One thing I'd like to do is ask @SMotaal about his experience designing the ESM survey. He presented that survey during the latest collaborators summit and it seemed like he had a lot of good insights into how to design a survey that would reach a large chunk of Node.js users.

[SMotaal]

I am happy to help... since our efforts have paused at the drafting stage, I think I can only share some key insights on how we made Google Forms work for our kind of questions because it does come with a bit of learning curve to get things done:

• Formatting for code is not provided so we ended up using the monospaced math charset.

• It does not have a good collaborative editing experience (no versions/history) so it falls onto the editors to find a good process.

• A feature we found useful is to use MCQs as "gatekeeper" questions allowing us to divide out our survey and personalize which sections each user gets to see.

• There are some additional stylistic insights but I found that if you are not willing to test your questions on someone seeing them for the first-time, your best intuition on how well they will work is likely nothing more than exactly that. It takes more than one person, willing to be proven wrong, and mutually open to actually discovering that it is not intuitive but it's what actually works (and the buy-in of everyone else, that was where I failed to catch up) to actually get the results you need.

[misterdjules]

Those are great insights, thank you @SMotaal!

[joesepi]

@benjamingr @misterdjules would you two be able to pull the questions together in a way that gets satisfactory survey results since you have more context around this issue than I do currently? I want to facilitate this happening and am happy to take the survey questions through the process but I dont want it to get held up by me trying to craft the questions adequately. Thanks!