Usage statistics on optional chaining in CoffeeScript

[alangpierce]

Hi! I've been following a lot of the discussions here, and to help inform these discussions, I wrote a tool that gets statistics of how the ?. operator is used in practice in real-world CoffeeScript projects. CoffeeScript includes basically all of the features under discussion (short circuiting, soaked new, soaked delete, soaked assignments, parens to block short-circuiting), so I think it's a good case study to see how these details play out in the real world.

(For a little more background about me, I've been the main person working on the decaffeinate project for quite a while now, and I implemented ?. and similar operations in decaffeinate, so I've worked with the nitty-gritty details of these operators quite a bit. My personal preference for JS optional chaining is to keep things simple; I was unpleasantly surprised when I learned how magical ?. is in CoffeeScript. But I'm a little biased because I've generally viewed this stuff from an implementer's perspective.)

Here's the repo: https://github.com/alangpierce/coffeescript-soak-stats

The README has a detailed explanation of the different stats when run on about 500,000 lines of "typical" code found on GitHub (Atom, ShareLaTeX, CodeCombat, SwitchyOmega, Trix, Vimium, YakYak, and Bacon.js). Here are the results:

Total files: 2489
Total lines: 475208
Total soak operations: 4627
Total soaked member accesses: 3811
Total soaked dynamic member accesses: 240
Total soaked function applications: 576
Total soaked new invocations: 0
Total soak operations using short-circuiting: 1522
Total soak operations using short-circuiting (excluding methods): 233
Total soaked assignments (including compound assignments): 37
Total soaked deletes: 1
Total cases where parens affected the soak container: 0
Total soak operations chained on top of another soak: 564

Some observations:

• Soaked new was never used and soaked delete was only used once.
• Even though parens can be used to block short-circuiting, e.g. (a?.b).c, it never came up in practice (probably because it would just make the code crash).
• Soaked assignments (like a?.b = c) had some usages, although of course they were much more rare than other uses of ?..
• Short-circuiting wasn't used in 67% of use cases. If short-circuiting only applied to direct method syntax like a?.b(), that would cover 93% of all use cases.
• It was about twice as common for people to write chains like a?.b?.c (which don't rely on short-circuiting) than a?.b.c (which rely on short-circuiting).

So I think this is at least an argument for leaving out new, delete, and meaningful parens, unless they somehow makes things simpler.

Happy to dig into the specific examples for these or run other statistics if people want. And of course it's open source and published on npm, so feel free to hack on it or try it on your own CoffeeScript code. I was reasonably careful and there are tests, but it's still possible some of these numbers are buggy.

[alangpierce]

One thing to keep in mind when interpreting these numbers is that "number of usages" probably isn't the best way to evaluate how important/useful a feature is, it's just the easiest thing to measure. Still, I think it's a rough proxy for usefulness. Ideally, you should also weight by how much of an improvement the syntax is over the alternative. Simple cases like a?.b (with no chaining) are common, but also only a minor syntactic improvement over the common alternative a && a.b. Cases like a?.b?.c?.d?.e are rare, but are probably more important because they're a major improvement over the alternative in JS.

[claudepache]

@alangpierce Thanks for the analysis.

Soaked assignments (like a?.b = c) had some usages, although of course they were much more rare than other uses of ?..

I wonder what are examples of such usages?

[alangpierce]

@claudepache here's a gist with all 37 examples: https://gist.github.com/alangpierce/34b7aa40cda51b0a089a44680bdfed7e

[claudepache]

Thanks. That provides arguments for including optional assignment: #18

[littledan]

This analysis is awesome!

[gisenberg]

@alangpierce Thanks so much for this detailed analysis! It's definitely appreciated.

[alangpierce]

@gisenberg asked if I could get numbers on how often out-of-scope globals are accessed in soak operations. (In CoffeeScript, a?.b evaluates to undefined rather than crashing if a is an undeclared variable and isn't a global.) They're certainly rare, but It turns out they're used more than I expected:

Total accesses of undeclared globals in soak operations: 74

Here's a gist with all 74 examples, all of which look intentional to me:
https://gist.github.com/alangpierce/9ca4eda80b148d7aba8e4b17d412f8f2

See also tc39/proposal-nullish-coalescing#13 for some more discussion about that behavior.

Note that the total amount of CoffeeScript in those repos has gone down by about 5% since my original post. I updated the README in the repo with new numbers, but the scale is pretty much the same for all of them.

[jazeee]

I found CoffeeScript's a?.b to be quite useful. It significantly simplified our code and made it much easier to reason with.
Optional operators are the one remaining feature I sorely miss when developing EcmaScript.
We also used a ?= b; frequently. (The Elvis operator).
This assigns b to a, only if a is null or undefined.
Very useful in scientific domains since a = a || b; is not desirable.

[alangpierce]

Actually, it looks like one of the usages I linked is unintentional, the use of bookmarksView in atom/bookmarks/lib/main.coffee:

https://github.com/atom/bookmarks/blob/master/lib/main.coffee#L38

There's a bookmarksView variable in the activate function and the deactivate function has bookmarksView?.destroy(). CS scoping rules make these different variables, so it's always undeclared in deactivate and the destroy function is never called, which is probably a memory leak or other similar bug.

That sort of mistake is probably the main argument against the "evaluate to undefined on undeclared variables" behavior.

[0x24a537r9]

Cases like a?.b?.c?.d?.e are rare

In most codebases, I would imagine so, but it's worth noting that this is actually very common if you are using Relay, where any part of the query could be null. Of course not everyone uses Relay--just something to consider.

[littledan]

@0x24a537r9 That's an interesting case.

Do you have any data to share about your effort to use ?.?

[MatthiasKunnen]

@littledan When using GraphQL, for example, the data you need can be deeply nested. At any moment, any field might be null.

[littledan]

I think we're all in agreement that optional chaining should support these deeply nested use cases well.