TypeScript compiler in Rust

[kitsonk]

This is long, but please read before you comment. If you are interested in following this, use the subscribe button to the right instead of just adding a comment.

Since the Deno v1 announcement there has been a lot of community interest in the TypeScript compiler in Rust that was mentioned. This issue is intended to provide information on that. First a few points:

• I am a long term collaborator in Deno. I am not a committer. This means these are entirely my own opinions, but ones I have formed with discussion on a wide range of people and my personal experiences.
• There is a highly likely hood that this issue will get noisy. Please think first before adding to that noise. A lot of people are eager to contribute to Deno and TypeScript. If you haven't been contributing to Deno and related projects or the TypeScript compiler itself for a while, it would be best to be an observer here and see if you can help out. We will all make better progress if we continue in a semi-organised fashion.
• Deno ❤️ TypeScript. TypeScript is here to stay. A lot of hard work, sweat and tears have been invested by lots of people to get TypeScript as a language a reality, and that has always been built on a compiler that is written in TypeScript. My opinion, which I know is not alone, is that TypeScript would not be as great as it is if it hadn't been written in TypeScript. That being said, the language TypeScript is built on, JavaScript is a slippery beast when it comes to performance. One could argue one of the things that makes it hard to easily optimise is the lack of strong types. That irony has a direct impact on the ability for tsc to maximise performance.
• Sometimes we are imprecise in our language. In the v1 announcement, the intent was born out of the challenges of making TypeScript a first class language in Deno, and want to get to a better situation where the gap in performance between starting up JavaScript in Deno and starting up TypeScript in Deno narrows. We have one opinion on approach, which we will express here, but we are open to take the path the delivers the best outcomes.

The Problem

We need a bit of context of what problem we are really trying to solve here. Type checking, especially with a structural type system built on top of a weakly typed underlying language, is expensive. As TypeScript as a language has grown, and the type checking become more and more advanced and safe, means that it is increasingly "expensive" to do this type of checking. Deno treats TypeScript as a first class language. At the moment Deno also always runs TypeScript through the TypeScript compiler and treats any type errors as it would treat a JavaScript runtime error.

This is "expensive". If you look at Deno benchmarks you will see that there is currently a cost of about 10x just spinning up the compiler to get some TypeScript transpiled into JavaScript.

We have tried a few things to improve that, but there is still a big cost, and we want to try to narrow that gap significantly. For other things in Deno we have had really good experience in moving to Rust things we used to do in JavaScript. There are several reasons for the performance improvement, but the most compelling is that it is just plain easier to get performant code in Rust. If you resort to non-obvious structures in JavaScript, you can often get lots of performance from v8, but it is really really really hard work. Our text encoder is an example. We started with a spec complaint implementation, following the structures laid out in the IDL. It was abysmal performance. We then implemented a fairly "magical" non-obvious implementation in JavaScript/TypeScript which dramatically increased performance, but that wasn't even enough, we got eve more performance moving that to Rust, with even the overhead of "copying" that data in and out of the JS sandbox is faster than what we could get out of heavily optimised JavaScript. Could we have gotten more out of our JS implementation... maybe... but it just gets too hard.

How does it work today?

I realise a lot of folks might not understand well where we are at today, so some background context is helpful. It sort of works like this...

How the CLI is put together

• v8 runs JavaScript in a sandbox, called an isolate.
• rusty_v8 is a Rust crate that provides bindings to v8 from Rust, to allow Deno to interface with v8 (which is built in C++).
• Deno core is a Rust crate that provides a basic communication path between a v8 isolate and Rust. Sending a message, including data, between an isolate and Rust is what we call an "op", and so Deno internal isolate code "ops" to Rust and Rust replies back.
• deno_typescript is a Rust crate that allows us to transpile TypeScript ES Modules into a single file JavaScript System bundle which can be run in an isolate. This is effectively bootstrap code which allows us to author TypeScript for internal code. It also allows us to create snapshots, which is a v8 feature which allows a running JavaScript sandbox's state to be serialised and dumped to a file.
• During the build process we create two snapshots: the runtime/cli snapshot, which provides all the basic runtime environment where Deno CLI code runs; and the compiler snapshot. The compiler snapshot is effectively a web worker which contains typescript.js (tsc if you will) and our infrastructure code to manage it.
• In the compiler bundle that the snapshot is built on, we instantiate a program with the TypeScript compiler, which with our infrastructure generates TypeScript source files for each of the lib files that we use in our runtime environment. We also currently use that as an "old program" we feed the compiler, but recent conversations have made me realise that this likely doesn't do anything, that speed we are seeing is that having the source files in memory for the lib files is what is efficient as that being part of the snapshot, as they are reused on subsequent compliations.

"Running" TypeScript code

• When you try to use some TypeScript (most commonly via deno run), the Deno CLI will take the module passed and see if it is in the Deno cache, or if it needs to be fetched. If it is in the cache and it is a local module, it checks if the source has been modified or the --reload flag is true, and if it is remote, just if the --reload flag is true. If the transpiled version of the modules is in the cache and "valid" Deno will just load it (see below).
• If there is a TypeScript file that needs to be compiled/transpiled, Deno will spin up the compiler isolate from the snapshot and then will send it a message to compile the module.
• Inside the infrastructure code for the compiler in Deno, we currently use the ts.preProcessFile() plus other logic to determine the dependencies of the root file. As those dependencies are identified, the compiler "ops" to Rust to resolve and fetch those modules.
• There is logic in Rust to deal with identification of "type substitutions" for TypeScript files. For example if you are loading a JavaScript file, but you want to use type definitions for the compiler to type check against instead of the JavaScript file, Deno supports using things like X-TypeScript-Types header to identify those files. The fetching logic will resolve all of that before returning the type definition to the compiler instead of the JavaScript file.
• When the sources are resolved, they are returned back to the compiler and the compiler infrastructure caches them in memory in order to be ready for them to be requested by the TypeScript compiler.
• Deno then creates a program with the TypeScript compiler. This will then trigger the TypeScript compiler to request source files, which we start feeding via the provided Deno CompilerHost. We substitute .d.ts files for JavaScript files based where appropriate.
• We check the pre-emit diagnostics and if there are any (that we don't ignore), we serialise them and return them to Rust and stop the compilation.
• Deno then does an emit on the program, which will cause the TypeScript compiler to start to "write" out files. The Deno CompilerHost will take these writes and "op" back into Rust to have them added to the cache.
• Finally we finish the compilation and return to Rust, which usually spins down the compiler.

Loading JavaScript into the isolate

• When a file is in the cache, where it is either direct JavaScript, or JavaScript transpiled from a TypeScript source, Deno loads it into the runtime isolate (or a worker isolate) as an ES Module. Everything from "userland" is considered an ES module.
• Loading the module triggers v8's dependency analysis, which in turn will start requesting other modules to be loaded into the isolate. Those are then fetched from the cache. If the root module was TypeScript, most of the time the compilation will already be done and the emitted file is in the cache ready to go. In the case of not statically identifiable dynamic imports or if the root module was just JavaScript, hitting a non-transpiled module is possible, and the compiler would be loaded and the compilation would happen like above.
• Once v8 has all the modules, it instantiates them and code starts running. 🎉

The Approach

Ok, so what do we do about it. The discussion I have had on the subject with various people have always been about an evolutionary approach. Also this approach is what I personally feel is right for Deno. Solving everyone's problems is tough, but if solving Deno's problems helps out everyone else 🥳.

Here are the major items that I think we need to tackle, and likely in the order presented here:

• Move the dependency graph analysis into Rust. @bartlomieju has this in progress in refactor: rewrite TS dependency analysis in Rust #5029. It also needs to be integrated into the compiler which Bartek and I planned on working on together. This means we would be able to fully fetch all the sources that the compiler would need before we spin up the compiler. While ts.preProcessFile() has been super useful, there are a few bugs with it that have been long outstanding, and with our added logic of supporting type substitutions it is just better/easier/more testable to have all of that in Rust. It will speed up things little bit from not having to "op" back and forth sending individual files.
• Provide a "no check" capability for Deno, where TypeScript is simply stripped of its types (and transformed for things like experimental decorators) and then loaded into Deno. This is discussed in Slow TypeScript compile time #4323 but likely needs its own issue to deliver the feature (which I will take care of). This is covered in Not Type Checking TypeScript #5436. Some of the conversations I have had with people is that making this the default might actually make more sense. Something we should discuss. Not type checking TypeScript would certainly speed things up, and especially when you are developing, and saving type checking for those special occasions, like a pre-commit hook.
• Get a really good performance analysis of the TypeScript compiler under Deno. We have the v8 inspector working under Deno now, we should be able to get flame charts for compilations, as well as there is a lot of instrumentation infrastructure in the TypeScript compiler that we haven't looked at enabling or using in Deno. Some of it is quite Node.js opinionated, so contributing back enhancements that make it a bit more isomorphic or enable it a bit more I am sure would benefit everyone, as well as making it easy to observe the performance of the compiler. Admittedly we don't know specifically where we are eating up time.
• Look at lexing/parsing of TypeScript in Rust and sending a serialised AST to the compiler isolate. This is a seriously complex issue. swc is our preferred choice for Rust lexing/parsing of JavaScript and TypeScript. It is the engine behind deno fmt (along with the excellent dprint built on top of it, which is how we became aware of swc in the first place) and deno doc. SWC, like most ECMAScript parsers, produces a estree like AST structure. The TypeScript AST is significantly different than that... it is far more aligned to a Roslyn AST. It would be naïve of me (or anyone) to think that it would be trivial to transform swc's AST to a TypeScript one, but it maybe worth the effort, or at least exploring a simple transform to see if we can generate an AST (TS SourceFile) that TypeScript can consume. If this did work, we could use the TypeScript compiler to just do type checking. (It was pointed out that a reference implementation of estree like AST to TS AST exists in typescript-eslint)
• Investigate further improvements in type checking performance in TypeScript that could be gained by optimising for v8. This would benefit everyone, not just Deno. It would be hard work, but I believe the TypeScript core team are amenable to contributions along those lines. Once we have a baseline of performance, there maybe a lot we can do there.
• Try to move type checking to Rust. Personally, I am not convinced on this, but it is a logical conclusion of "moving to Rust". I see lots of downsides to this, and very few upsides, unless we find that 99% of the time spent in TypeScript was type checking, even then it is naïve to think that Rust in and of itself would be the solution to that problem. There is a lot of blood sweat and tears put into the type checking in TypeScript from a lot of people that understand that a lot better than any of us ever will. What I do understand about it is that what the type checking is doing is a lot different than saying the "hot paths" that tend to work fast in Rust, like parsing and lexing large text files. So even if it were ported over and kept in sync with the TypeScript based type checker, it may not perform that much faster. My opinion is there is a lot of other things to do before we even consider this, if ever.

[Bnaya]

I'm not sure how feasible to integrate with deno, but tsc have the incremental flag to speedup re runs, or spin up language server and keep it alive
https://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-4.html#faster-subsequent-builds-with-the---incremental-flag
https://github.com/microsoft/TypeScript/wiki/Using-the-Compiler-API#incremental-build-support-using-the-language-services

[dsherret]

@Bnaya see #5094

[kitsonk]

Thanks @dsherret. Yes it might speed up big projects where there are incremental changes. Once we get to the point where we are only type checking in the compiler (I guess it really becomes the "type checker" at that point) we would want to see if incremental compiles would help.

[matklad]

it may not perform that much faster

I think there's a possibility that perf gains here would be substantial, for two reasons.

First, tsc is a compiler geared towards interactive IDE/LSP server use-case. My experience as an author of such IDE tooling tells me that this has non-trivial perf implications (pervasive lazyness, lossless-ish syntax trees, and the tax of additional complexity). By shifting focus to batch compilation, it should be possible to simplify and speed-up things on the architecture level.

Second, while type-checking can't really benefit from something like simd, there's still a lot that can be gained by using a language with full control over data layout. Type checking is mostly pointer chasing and cache invalidation, and having tightly packed arena-allocated data structures helps a lot. HashMap performance is also super important. Sorbet, the typechecker for Ruby, was written in C++ for these reasons (some info 1, 2).

The fact that type-checking could be made faster at significant dev cost, of course doesn't necessary mean that it should. --no-type-check for running tests and tsc in your IDE for on-the-fly type checking seem to be cost/benefit sweet spot.

[kitsonk]

@matklad thanks for the feedback and thoughts.

Your second point is really interesting. There might be some data structures, or operations that are simply not performant. Because of the legacy of the compiler, evolving over a language that has moved a lot, and not specifically optimised for the engine it primarily runs in (v8), there could actually be a lot of ground to be gained in making sure the performance of the data structures is optimised.

TypeScript core team, for lots of valid reasons, is never going to move off of building the compiler in TypeScript. Therefore my biggest fear is the while getting 100% syntax parity is "easy" getting 100% type checking parity would be hard or impossible. That is why I am hopeful that the long road to get there means we may never get to the final destination.

@95th that is effectively what we are talking about here.

[znck]

esbuild can be used for transpiling typescript without type checking.

[kitsonk]

@znck esbuild is built in Go and is not designed to be embeddable... It offers no advantages over swc which is built in Rust and which we have already integrated in Deno.

[elektronik2k5]

Go is an implementation detail and shouldn't matter. Quoting @evanw, esbuild's author:

The Go code in this repo isn't intended to be built upon. Go is just an implementation detail of how I built this tool. The stable interfaces for this project are the command-line API and the JavaScript API, not the internal Go code. I'm may change the internals in a backwards-incompatible way at any time to improve performance or introduce new features.

The real issue IMHO is that esbuild doesn't (currently) support decorators - so that's a show stopper.

I also recently learned about sucrase, which seems substantially faster than swc. Should we consider it?

[kitsonk]

@elektronik2k5 it matters if you are thinking of leveraging it. Why port something to Rust when we already have something that does exactly the same thing, written in Rust? That just doesn't make sense. Plus esbuild's focus is on being a bundler, Deno's need is for single file asynchronous transforms.

Sucrase's what it is not would likely not be suitable based on its own statements: https://github.com/alangpierce/sucrase#what-sucrase-is-not. In particular it does not transform decorators, saying it needs to be supported by your runtime, but there are no runtimes that support decorators. It also says itself that it isn't intended for production use. swc's metric comparison there is sort of obscured, because of the way swc is being used. We have yet to really baseline what swc could do. swc is significantly faster when it is targeting more modern versions of JavaScript then when it is targetting older version. The benchmark's on Sucrase's website don't really indicate what the configuration of swc was, nor what the inputs were. Coupled with the fact that all those stats are some obfuscated by using the Node.js APIs to orchestrate everything. Because we have already indicated swc, and aren't going to get rid of it any time soon, and we don't have any strong indication that it is going to be a performance barrier, it doesn't make sense to pursue Sucrase.

Also, all of these would only solve bullet point two, which is important but clearly not the whole picture. None of these solutions do type checking. They all just do type stripping and transforms. The bigger fish to fry is always how to improve the speed of the type checking.

[apiel]

As I mention in this issue #6173 why not to give the possibility to the user to decide for their transpiler? Of course having by default a super efficient TS transpiler build-in inside Deno would be a huge advantage and would maybe be the reason of people switching from Node to Deno. But giving the possibility to give the choice to user to decide for their transpiler is an easy move and would also open the door to other programming language like dart, coffescript...

swc might be a great option but this would mean that we might not be up to date to the latest version of TypeScript. When a new TS version is release, we would first have to wait that swc update their code and then you could update Deno and finally user would have the latest version of TS. In the other way, if you give the users the chance to use their own transpiler, they could just pull the latest version of TSC and use it right away inside Deno, till the whole process from swc to Deno is done.

[HKhademian]

Rewriting typescript in rust can truly improve its speed and memory consumption but (a big one) separating deno typescript from main typescript development line can cause many problems:

1. any bug fixes and ... must patched in rust code as another process which need additional maintainer.
2. new features and improvement has delay to come in rust equivalent environment also in deno.
3. rewriting may cause new errors and ... problems in its implementation.

All of those, at the end, cause deno to use an outdated and much bug-proof version of TypeScript, and I think it does not worth unless TypeScript maintainers decide to port entire language to Rust or any low level language.

But (another big one)

We can use tools like AssemblyScript,or any other similar tools to Compile/Convert latest versions of TypeScript's compiler with some little modifications to WebAssembly version and use its higher performance version.

Yes, I know, AssemblyScript is so young and ..., but we can put/redirect any required affords from TypeScript->Rust rewrite process to help these projects.

At last, yes Rust conversions are good if they don't have costs and problems, like V8 engine in rust, windows kernel in rust, Linux complete rewrite in rust, but are they possible for any projects that use these libraries? (I think it's not possible even for their main maintainer, unless use a progressive work to convert like what Mozilla team did in Firefox)

[kitsonk]

AssemblyScript (and Web Assembly) have come up a number times. Let's make it clear, it is not suitable. There are fundamental differences between how JavaScript and WebAssembly work. JavaScript is a garbage collected language, WebAssembly is a memory allocated language, like Rust and C++ is, where the code itself manages the language.

AssemblyScript makes it clear on their GitHub page: "Definitely not a TypeScript to WebAssembly compiler". It is similar to TypeScript, which makes it easier for those familiar with TypeScript to write Web Assembly, but it is not a general purpose TypeScript/JavaScript to WebAssembly compiler, which is what would be needed. Why isn't there a general purpose JavaScript to WebAssembly compiler? Because that is effectively v8 or any other JavaScript engine, which provides all the other infrastructure needed, like a highly tuned garbage collector, to be able to deal with general purpose JavaScript. Even if there was a general purpose JavaScript/TypeScript to WebAssembly compiler, it wouldn't speed up the code, for two reasons. First, Web Assembly is an abstraction on top of another language. In the case of v8, C++. C++ is abstracting WebAssembly from the underlying CPU, still morphing the op codes to something the native architecture of the CPU understands. The second is that v8 is really really really really good at optimising JavaScript, a lot better than some other compiler could be, and types don't significantly help with runtime performance of JavaScript (Google experimented with this a while ago, it effectively didn't work).

AssemblyScript is great, if you are a TypeScript developer, and you need to move a hot path out of JavaScript/TypeScript to WebAssembly. It is a lot lower overhead for you then having to learn Rust or C++. Which is an awesome thing. AssemblyScript is a really cool project.

So if we accept that converting the TypeScript compiler to WebAssembly is impossible and wouldn't work, why not move some of the hot paths AssemblyScript/WebAssembly? In Deno, we simply wouldn't do that. Why? Because as state before, WebAssembly is still an abstraction on top of another language, it is "cheap" and "easy" for us to right things in Rust, and native compiled Rust will always run faster than Rust to WebAssembly. Working around the limitations of AssemblyScript, for Deno, wouldn't make it any easier.

The wider community shouldn't worry, people are talking to each other. Since Deno was started there has been communication between the Deno team and the TypeScript core team. Like all groups of people, we don't always see eye-to-eye, that doesn't mean we can't have productive conversations about how to improve the things for everyone.

[HKhademian]

First of all, I know and said that everything in Rust is Fast (compair to JIT langs, but not even every nodeJS task!, suggest to watch Ashley Williams - How I Convinced the World's Largest Package Manager to Use Rust, and So Can You! from npm team) and Secure (memory-wise). It's a fact, an obvious one.
But all I said is the Ideal transformation has some cost, issues and problems.

I see deno, as alternative to nodejs mess in future, but like every other thing in this world it has its own limited budget, eager followers, volunteer maintainers, ... . if deno maintainers spend big chunk of these resources where are not much important (compare to others), denos fate will be like WeWork and other prodigal ideas, platforms and services.

One of TypeScript compiler problems is it's boot time and JIT speed, we can (with some modification, maybe automatic ones) convert TypeScript compiler source code to make it compatible with tools like AssemblyScript then compile it to WASM or even find some tricks to use LLVM and generate native binaries for top used platforms (like Windows, Linux, MacOS, ...) to speed this part up and use main typescript compiler on other not supported platforms.

This conversion is fast (maybe some weeks to setup tools, develop solutions and CI/CD tasks and ...) and already available. It costs very low.

Even if we work together and reach our Ideal purpose (some day, full functional TypeScript Compiler in Rust), then we can switch to that better solution, till then we have our good temporary solution to TypeScript compiler's problem.

[kitsonk]

@HKhademian you are not understanding...

if deno maintainers spend big chunk of these resources where are not much important

I am glad you know what is and isn't important for us.

we can (with some modification, maybe automatic ones) convert TypeScript compiler source code to make it compatible with tools like AssemblyScript

This cannot be done, and even if it could it would run slower then it currently does.

This conversion is fast (maybe some weeks to setup tools, develop solutions and CI/CD tasks and ...) and already available. It costs very low.

Great, I await your proof of concept.

[rezonant]

Regardless of the WASM thing, I think @HKhademian has a point about the maintainability of such a solution, which is why I've watched this issue for some time somewhat worried about this Rust TS compiler plan. @kitsonk is the plan to rely on pulling in TS' automated test suites to ease tracking advancements in TS, or is there some other strategy long term?

[rezonant]

Apologies for the double post but I do want to also say that provided the TS specification is sufficiently detailed, it ultimately isn't inconceivable that multiple implementations of the language can be handled-- after all, it is commonly a sign of language maturity (and an insurance policy) when there are multiple distinct compiler implementations. But it will be a large effort. I do think it will be necessary to decouple this effort from Deno so you can get contributors who just want a faster TS compiler without losing some decent level of type checking.

[kitsonk]

Regardless of the WASM thing, I think @HKhademian has a point about the maintainability of such a solution,

Which I also extensively made as the last bullet point of my original post, and might even be unnecessary.

@kitsonk is the plan to rely on pulling in TS' automated test suites to ease tracking advancements in TS, or is there some other strategy long term?

If we end up at that point, ensuring the language doesn't become fractured would be a goal.

[HKhademian]

I am glad you know what is and isn't important for us.

I didn't decide anything for you, I said deno can work without faster Typescript compiler. so it has less importance than fixing some bugs, issues and even add new features.

I love deno idea and it's existence, I love to help it and anything I said was in direction of my thoughts for its good.
They are just my opinions.

[phaux]

Try to move type checking to Rust.
even if it were ported over and kept in sync with the TypeScript based type checker, it may not perform that much faster.

If type checking was ported to Rust then what's the point of keeping it in sync with TypeScript? I would understand it, if there was a huge ecosystem (like npm) that needs to be supported, then it would be valuable to be compatible with all of that old TypeScript code, but that's not the case.

If Deno had it's own type checker, I wouldn't care if it supports code already written in TypeScript, because most of that code is written for Node anyways. It could reuse existing TypeScript syntax (or even Flow, Hegel or a custom one), but it wouldn't be a huge problem if it worked differently than one of the existing type checkers. Even the current approach of using slightly modified tsc already isn't compatible and needs editor plugins to work properly.

Also, having a custom type checker would be a great opportunity to create something that is more strict and correct by default. TypeScript has had a lot o strict options added and had to make a lot of compromises for backward compatibility, so it's not that great at all.

[rezonant]

No, there already is a lot of Typescript code out there and I think a lot of us want to be able to use it both on Node and Deno. If Deno were to become (even more) incompatible to tsc, I would opt to simply compile/transform my TS code to Deno-compatible JavaScript and ship that for use with deno. Honestly, it already looks like the better path given the performance and compatibility headaches that already exist.

[Ebuall]

If Deno had it's own type checker, I wouldn't care if it supports code already written in TypeScript, because most of that code is written for Node anyways.

Compatibility with old ecosystems is the biggest reason TypeScript is popular. And the are a lot of code. Most of which doesn't even care if it's written for node or browser.

[CreatCodeBuild]

@HKhademian Deno's compile time is not a problem if you are talking about the production environment where most of the time we don't care about the start-up time.

But, when it comes to development, it is a major developer experience killer.

I am working on an HTML5 game and was using Deno at the beginning because Deno has bundle options so that I don't need to set up a whole webpack/babel thing.

It costs 20s to compile while my game was just several thousands lines of code. The speed of iteration was killed by this slow compile time. I switched to Node and setup a webpack/babel thing.

I am not sure if Deno has incremental compilation enabled for TSC or if that is even possible for Deno. However, I believe if even Deno team doesn't implement TSC in Rust, there are places where TSC can be faster in JS implementations or Deno's internal pipeline.

[apiel]

@CreatCodeBuild completely agree with you. If developer experience is not good, people will not use Deno. Right now, Deno is still in a very early stage and I would not recommand it to anybody in a professional project. For the moment, we still have to wait till we get reasonable performance and flexibility. But at least, we could help to build the eco-system by creating new modules, compatible with Deno.

In another thread, we point out that one of the main problem of TSC is the types checking of the code. Would be great if we could just deactivate this type check when we are in dev mode.

Also another point that you mention, is that in node many people use babel because it is much much faster than TSC. So would be really great to give the possibility to decide for our own transpiler. Of course by default, it would be TSC but if someone want to tune it, he could just mention to deno to use something else (for example to use babel). And doing this, would also open lot of room to developers, we could then use a different module loader than the native one from v8 and therefor use nodejs module.

[HKhademian]

Maybe it's a silly idea but could we just convert TS code to JS by removing all type-chekings and type-system and see the bare ts code as JS one (which it is), feed it to V8 and skip typescript compiler?

We could use deno check (something like in rust) to check for types and ... ,
Aslo to bundle the final product in CI/CD work-follows we can have flags like --type-system to enable this feature.

This idea (maybe it's silly) came from where I saw deno can run JS like TS codes, if we don't have any problems with JS code, so by fast convert (not compile or analyse) it to JS, we can use JS's benefits.

[dsherret]

@HKhademian this has been discussed before. See #5436

[kitsonk]

@HKhademian to add to @dsherret is specifically discussed as a step (second one!) in the approach in the original post that we are working towards. 😕

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ghost]

Can't Deno fire off two threads, one running SWC stripping types and emitting into a temporary file, the other thread running TSC --no-emit for performance, then, depending on the result of TSC, running the emit file from SWC, or returning with a non-zero exit status?

Would that even help?

[schickling]

That's also how https://github.com/rsms/estrella works (except it's using esbuild instead of SWC).

[ghost]

In that case, if it's viable, based on that benchmark,

Parse Time - 15%
Bind Time - 12%
Check Time - 49%
Emit Time - 24%

we could cut out the 15% and the 24%, reducing the overall time by roughly 40%.
That sounds pretty good, but is Deno already doing something like this?

[kitsonk]

but is Deno already doing something like this?

Yes.

[ghost]

@kitsonk In that case, what does "bind" refer to from that benchmark?
Could we help there?

[VienDinhCom]

Is this good new for JavaScript/TypeScript?

Rome will be written in Rust 🦀

Rome started off written in JavaScript because that is the language of choice for our team and it made it easier for others in the community to join as contributors. We love JavaScript and TypeScript (and HTML and CSS) at Rome, and we want to build the very best tooling possible for these languages. For a number of reasons, we’ve decided that Rust will provide a better foundation for this tooling.

We are also taking the opportunity to explore fundamental shifts in the architecture of Rome. These changes give us more flexibility and will allow us to build tooling JavaScript and the web has not had before.

Read more: https://rome.tools/blog/2021/09/21/rome-will-be-rewritten-in-rust

[kitsonk]

@Maxvien the core teams between both projects are in communication. There are certainly things that we can collaborate on that are mutual interest. Having more JavaScript/TypeScript projects focused on Rust certainly will help things. One thing we both agreed on is that we are a ways away from full type checking TypeScript in Rust, though Rome has some interesting ideas at ways of chipping away at the problem, as well as we are keeping an eye on what is happening with swc's work on type checking in Rust as well.

[charrondev]

as well as we are keeping an eye on what is happening with swc's work on type checking in Rust as well.

Looks like the SWC doesn't have any intention of releasing their type-checker under an OSS license (and development has stalled on it), instead intending to monetize the software under a proprietary license to fund it's development. swc-project/swc#571 (comment)

So far SWC -> TSC AST conversion seems to be the avenue that would be fastest to gets some speedups. Is there a separate tracking issue for looking into that and/or has someone already done some evaluation of the work required for that.

[majo44]

In the context of the discussion: I’m porting tsc to Go - DongYoon Kang

[GerbenRampaart]

@kitsonk Sorry if I lag behind the discussion or bring up something uninformed, but just to ask; isn't the long-term (v2) aim for Deno transpiling the deno_ast to rust ast and skipping v8? Making typescript generate rust and therefore effectively making the deno runtime a typescript abstraction over rust would simplify the architecture no?

Probably a ludicrous idea. For one thing that, in this hypothetical situation, you would have to drop support for js. But are there really many people who started with deno from a js angle instead of ts?

[lucacasonato]

@GerbenRampaart We are not dropping JS support (who said that?). Secondly it is not possible to "compile" TS to Rust. TS does not require sound static typing, while Rust does. The type systems and allocation primitives of Rust and JS are too different to transpile one to the other. User code will continue to be executed in V8 for the foreseeable future.

[GerbenRampaart]

@lucacasonato Thanks for the answer, did not realise the type systems were that different. No one said deno is dropping js support, I implied it would be a consequence of a ts => rust transpiler, that's all. (edit: I re-read my original spit-ball and see how you could read that the deno team said js support would be dropped, so I made it clearer, thanks.)

[azdavis]

stc, a WIP typechecker for TS in Rust, was just open-sourced. It's by the author of swc.

• https://github.com/dudykr/stc
• https://kdy1.dev/posts/2022/10/open-sourcing-stc

[notramo]

There is another TS compiler in Rust, based on the OXC project.

https://github.com/kaleidawave/ezno
https://kaleidawave.github.io/posts/introducing-ezno/

Please... don't start a third one. These projects are great, utilize one from these, contribute there, because we do better with 1 fully compliant implementation than 3 partially compliant ones.

[marvinhagemeister]

FYI: This thread is a bit older. There are no plans on our side here at Deno to write a TypeScript type checker at the moment.

[ry]

Won't implement. Too hard.

[dsherret]

Just to follow-up, in Deno v1.0 type checking was indeed a bottleneck, but that's no longer the case. There have been significant changes to Deno's TypeScript support that reduces the need for a separate TypeScript implementation in a more performant language like Rust.

1. Deno no longer type checks by default for every subcommand and a new deno check some_file.ts command was introduced.
2. Deno has gotten a lot better at leveraging caches to speed up type checking.
3. Deno no longer type checks if it knows a script has no diagnostics (try running deno check some_file.ts twice and you'll see it's instant the second time)
4. Dependency analysis is now done in Rust.
5. TS to JS emit is now done in Rust.
6. JSR introduced the concept of discouraging the use of "slow types" (https://jsr.io/docs/about-slow-types). When using JSR, Deno now reduces the amount of code that's sent to TypeScript so that type checking is much faster.

Overall, in my opinion, it's best to align as much as possible with the official implementation of TypeScript. Creating a new implementation is a full time job for a large team of developers.