Roadmap for Node Intellisense?

I was comparing WebStorm 10 with VS 2015 earlier this week and when I had some issues with WebStorm's autocomplete, I told JetBrains support how unfavorably their feature compared to Visual Studio's with regard to third party NPMs (my particular problem was with lodash). Their support staff responded:

"For WebStorm 10, the possible workaround is [manually setting up the] typescript stubs ...

As for Visual Studio... Well, indeed it provides more accurate completion - as it's based on JSLS service that works by executing the JavaScript code to dynamically provide syntactically correct completions. But, as you can imagine, need to execute code dynamically in background causes a huge memory and CPU overdraft, especially when working with Node.js projects that are much larger and more complicated than average client-side apps. That's why MS is planning to abandon this service and start using typescript stubs for completion when working with Node.js"

I told them that I don't think you'd replace your current system with something that's less automatic.
But, now I'm wondering where this is coming from (maybe here) and I want to confirm - are we going to be forced to find and setup Typescript stubs for all third-party libs at some point?

Thanks for your work!

[mousetraps]

Yeah, it sounds like they got that from here and from the comment you referenced. Though it's a little weird that their answer focuses on JSLS and completely disregards the static analysis we do in NTVS to provide completions.

The new ES6 preview feature is completely experimental at the moment, and also happened to be the quickest way for us to get ES6 support.

Our primary goals with our efforts here are to:

• reduce duplication of effort (thus freeing up our time to iterate more quickly on other elements of the experience)
• reduce memory/CPU overhead
• provide a more consistent experience throughout Visual Studio.

Throughout this process, we're investigating all of our current JavaScript-based language services (JSLS, TypeScript, and NodeLS), and trying to understand how we can leverage the best features of them all.

That said, these efforts are highly guided by user feedback. So try downloading the latest release, flipping the ES6 Preview flag on, and letting us know what you love / hate so we can work to improve in the next iteration.
https://aka.ms/ntvslatest

Hope that helps, and lmk if you have any more questions!

@mousetraps Thanks for your detailed response. My chief concern is that I'd rather not have to go find a definition file for every external library that I want Intellisense for. So my question is - do you think that is a direction we're heading in for ES5 code? Or, will what we have for ES5 just continue to get better and faster and more reliable? (I understand that you said things depends on user feedback, so I'll take your personal opinion if you care to provide it.)

The ES6 announcement you referenced was unclear to me if Typescript definition files were required for every single external lib. It says "[the ES6 preview] can take advantage of TypeScript definition files", but I'm not sure if it requires them. Does it?

I would try out the ES6 mode, but I'm afraid for a few reasons:

• I don't want to install Typescript 1.6 today for various reasons.
• I don't want it to show me ES6 things that I can't use. I'm not using ES6 and I won't until it has been widely available in most browsers for some time.

Here is my general feedback for this feature though - I want things to be as accurate and automatic as possible. That's the main reason why I prefer VS and Microsoft tech in general. Javascript libs move too fast and it's not worth the amount of maintenance time spent to make sure things are up to date.

Thanks again!

[mousetraps]

My personal opinion is that we're taking a step back in the interest of taking two steps forward. I'm also excited because there are also a lot of very smart people with diverse opinions on the matter working towards this common goal, and that tends to be a good recipe for building something awesome 😃.

Regarding automaticity, I agree that we should strive towards a "just works" experience. What remains to be seen is just how close we can realistically get. With tsd files in particular, we could totally take advantage of our existing npm integration to automatically attempt to grab the tsd file when a new package is installed.

That doesn't fully address the issue, though. My primary concern with a tsd-only approach is the state of DefinitelyTyped. Specifically...

• # of packages:
  npm registry has hockey-stick growth, DefinitelyTyped does not (and only 1,235 entries compared to 188,201 npm pkgs)... I've often compared our Node.js Tools static analysis support of particular library/framework to the support a browser has for a particular website. Essentially, we try and understand the patterns people are using, and we'll do well so long as there weren't any unforeseen patterns.

  Taking that analogy a bit further... one of the biggest lessons we've learned with Internet Explorer is that even though the top 9000 websites gets 88% of traffic, addressing just those top 9000 of sites isn't sufficient - in other words, the "long tail" matters. And based on some of the numbers and stats about npm and various user-reported issues with our own analysis engine, my hunch is that this same tail end phenomenon exists with npm, so we may run up against the same challenges if DefinitelyTyped only addresses the top 1% of packages.

• versioning of packages:
  New npm versions are released all the time, so what happens to IntelliSense if a particular version of a package is required? DefinitelyTyped doesn't have a good story for this yet afaik

• general quality concerns:
  The nice thing is that there's incentive for library authors to create and share good tsd files so that they'll provide a better experience for their users. However, that hasn't been a strong enough incentive for everyone to write great docs, so I question whether people's behaviors will change here.

My hope is that both the DefinitelyTyped community will continue to grow and we can eventually land on a hybrid approach that enables a simultaneously lightweight and seamless experience. Some ideas regarding how we might be able to do that include:

• automatic tsd download as I mentioned above
• an additional level of static analysis
• using tsd files where we have them, and otherwise falling back to static analysis.
• maybe we can repurpose our existing static analysis engine to help people generate tsd files automatically and decrease the barrier of writing them from scratch.
• providing a good extensibility model that will enable ppl to customize the experience further.
• Perhaps a hybrid dynamic / static approach is possible.
• one complaint about our NTVS intellisense is that it's sort of a black box from a user-perspective because it's not entirely clear which patterns work and which ones don't. So finding a way to communicate those expectations better would be valuable.

That said, the above was just a personal braindump, and there certainly other ideas floating about. Long story short, this whole effort is going to take a lot of work, and one of our biggest challenges is trying to understand what aspects of our current IntelliSense experience people care most about so we can prioritize pulling those best-parts into our newly combined experience (which is especially difficult thanks to the last bullet point above 😃).

So any feedback you have to offer would be amazing. I understand you don't want to mess up your existing environment. Maybe one of our VMs would make it easier? http://blogs.msdn.com/b/visualstudio/archive/2015/04/24/node-js-tools-1-0-on-github-and-vms-available.aspx

[mousetraps]

And regarding general direction... the "ES6 IntelliSense Preview" may as well have been labeled "IntelliSense Preview", but

• we wanted a user-friendly name for it that made the advantages clear (the biggest value add at this point is ES6)
• we know it's not up to snuff to our current IntelliSense experience in some ways (in others - e.g. consistent browser/node.js intellisense experience - it's better) and we don't want to make it the default until we can improve on it enough and bridge that gap. That's not to say that there won't be sacrifices, but we're aiming for a better overarching experience.

One concern that I have about using tsd for Javascript Intellisense is that I don't want those files to become part of my project source code in any way. I don't want non-Visual Studio users to have to deal with them. If Visual Studio needs them to do stuff internally, that's fine with me as long as it's mostly automatic. Something like the way NuGet works sounds reasonable to me at first - the only thing that would have to be stored in my project repo would be a list of tsd packages. Of course that involves running a package server or depending on third parties like GitHub and that's probably a deal-breaker. I'm really hoping for something that just runs the Javascript, creates and caches some tsds for internal use and doesn't make me do anything.

I bit the bullet and installed a bunch of updates for VS2015 today, including Typescript 1.6. I was hesitant because I had to burn an MSDN support incident a couple months ago to fix a problem after an update. So, I'll certainly try it out when I can!

[code-Review-2015]

@mousetraps I'd be happy to send reproductions of any issues, even my VHD, but I'd probably need to use what, Azure(?) to send you 40GB+ of a .VHD in a reasonable amount of time. Have you guys already looked at the Lexical Analysers that Microsoft uses for other languages? V8 compiles .js files into native code, right?

@code-Review-2015 V8 compiles js directly to machine code and I don't think there are any lexical analyzers that can reverse that into an abstract syntax tree that resembles the original js code.

[EdMaurer]

Bill outlines the current thinking for future JavaScript/TypeScript editing plans in Visual Studio and beyond here. @waynebloss your feedback on the provisioning and managing of assets like TypeScript typings in a JavaScript project is exactly what we are looking for.

[bowdenk7]

Hey @waynebloss I work directly on the VS 2015 IntelliSense (auto complete) experience and would love to hear more about your perspective on code completion. Any chance you would be interested in a quick skype/phone call?