From fb6938ce11d9620250eed212c5370eb9ccb5473b Mon Sep 17 00:00:00 2001 From: mark Date: Tue, 31 Mar 2020 21:58:05 -0500 Subject: [PATCH 01/13] start on guide reorg --- src/SUMMARY.md | 26 ++++++++++++++++++-------- src/about-this-guide.md | 21 +++++++++++++-------- src/mir/construction.md | 2 +- 3 files changed, 32 insertions(+), 17 deletions(-) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 366146185..c61eb2d2d 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -33,7 +33,8 @@ - ["Cleanup Crew" ICE-breakers](ice-breaker/cleanup-crew.md) - [LLVM ICE-breakers](ice-breaker/llvm.md) - [Licenses](./licenses.md) -- [Part 2: How rustc works](./part-2-intro.md) + +- [Part 2: High-level Compiler Architecture](./part-2-intro.md) - [High-level overview of the compiler source](./high-level-overview.md) - [The Rustc Driver and Interface](./rustc-driver.md) - [Rustdoc](./rustdoc.md) @@ -46,15 +47,24 @@ - [Profiling Queries](./queries/profiling.md) - [Salsa](./salsa.md) - [Memory Management in Rustc](./memory.md) + +- [Part 3: The Compiler Frontend](./part-3-intro.md) - [Lexing and Parsing](./the-parser.md) - [`#[test]` Implementation](./test-implementation.md) - [Panic Implementation](./panic-implementation.md) - [Macro expansion](./macro-expansion.md) - [Name resolution](./name-resolution.md) + - [AST Validation](./ast-validation.md) - [The HIR (High-level IR)](./hir.md) - [Lowering AST to HIR](./lowering.md) - [Debugging](./hir-debugging.md) + - [The MIR (Mid-level IR)](./mir/index.md) + - [HAIR MIR construction](./mir/construction.md) + - [MIR visitor and traversal](./mir/visitor.md) + - [MIR passes: getting the MIR for a function](./mir/passes.md) - [Closure expansion](./closure.md) + +- [Part 4: The Type System](./part-4-intro.md) - [The `ty` module: representing types](./ty.md) - [Generics and substitutions](./generics.md) - [`TypeFolder` and `TypeFoldable`](./ty-fold.md) @@ -82,12 +92,7 @@ - [Method Lookup](./method-lookup.md) - [Variance](./variance.md) - [Opaque Types](./opaque-types-type-alias-impl-trait.md) - - [The MIR (Mid-level IR)](./mir/index.md) - - [MIR construction](./mir/construction.md) - - [MIR visitor and traversal](./mir/visitor.md) - - [MIR passes: getting the MIR for a function](./mir/passes.md) - - [MIR optimizations](./mir/optimizations.md) - - [Debugging](./mir/debugging.md) + - [Pattern and Exhaustiveness Checking](./pat-exhaustive-checking.md) - [The borrow checker](./borrow_check.md) - [Tracking moves and initialization](./borrow_check/moves_and_initialization.md) - [Move paths](./borrow_check/moves_and_initialization/move_paths.md) @@ -100,9 +105,14 @@ - [Closure constraints](./borrow_check/region_inference/closure_constraints.md) - [Error reporting](./borrow_check/region_inference/error_reporting.md) - [Two-phase-borrows](./borrow_check/two_phase_borrows.md) + - [Parameter Environments](./param_env.md) + +- [Part 5: The Compiler Backend](./part-5-intro.md) + - [The MIR (Mid-level IR)](./mir/index.md) + - [MIR optimizations](./mir/optimizations.md) + - [Debugging](./mir/debugging.md) - [Constant evaluation](./const-eval.md) - [miri const evaluator](./miri.md) - - [Parameter Environments](./param_env.md) - [Compiler Backend](./backend/backend.md) - [Monomorphization](./backend/monomorph.md) - [Lowering MIR](./backend/lowering-mir.md) diff --git a/src/about-this-guide.md b/src/about-this-guide.md index 0882c40c0..681aa7756 100644 --- a/src/about-this-guide.md +++ b/src/about-this-guide.md @@ -2,14 +2,19 @@ This guide is meant to help document how rustc – the Rust compiler – works, as well as to help new contributors get involved in rustc -development. It is not meant to replace code documentation – each -chapter gives only high-level details, the kinds of things that -(ideally) don't change frequently. - -There are three parts to this guide. Part 1 contains information that should -be useful no matter how you are contributing. Part 2 contains information -about how the compiler works. Finally, there are some appendices at the -end with useful reference information. +development. + +There are six parts to this guide: + +1. Contains information that should be useful no matter how you are + contributing, such as procedures for contribution, building the compiler, + etc. +2. Discusses the high-level architecture of the compiler, especially the query + system. +3. Discusses the compiler frontend and internal representations. +4. Discusses the type system. +5. Discusses the compiler backend, code generation, linking, and debug info. +6. Appendices at the end with useful reference information. The guide itself is of course open-source as well, and the sources can be found at the [GitHub repository]. If you find any mistakes in the diff --git a/src/mir/construction.md b/src/mir/construction.md index 52d1f4609..f353eacdf 100644 --- a/src/mir/construction.md +++ b/src/mir/construction.md @@ -1,4 +1,4 @@ -# MIR construction +# HAIR and MIR construction The lowering of [HIR] to [MIR] occurs for the following (probably incomplete) list of items: From 4d68c8a11cbcb1abe4d8fa3752c4ea39845af3eb Mon Sep 17 00:00:00 2001 From: mark Date: Tue, 31 Mar 2020 22:01:39 -0500 Subject: [PATCH 02/13] backend intro --- src/SUMMARY.md | 20 ++++++++++---------- src/{backend/backend.md => part-5-intro.md} | 1 + 2 files changed, 11 insertions(+), 10 deletions(-) rename src/{backend/backend.md => part-5-intro.md} (99%) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index c61eb2d2d..f47902275 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -113,17 +113,17 @@ - [Debugging](./mir/debugging.md) - [Constant evaluation](./const-eval.md) - [miri const evaluator](./miri.md) + - [Monomorphization](./backend/monomorph.md) - [Compiler Backend](./backend/backend.md) - - [Monomorphization](./backend/monomorph.md) - - [Lowering MIR](./backend/lowering-mir.md) - - [Code Generation](./backend/codegen.md) - - [Updating LLVM](./backend/updating-llvm.md) - - [Debugging LLVM](./backend/debugging.md) - - [Backend Agnostic Codegen](./backend/backend-agnostic.md) - - [Implicit Caller Location](./codegen/implicit-caller-location.md) - - [Profile-guided Optimization](./profile-guided-optimization.md) - - [Sanitizers Support](./sanitizers.md) - - [Debugging Support in Rust Compiler](./debugging-support-in-rustc.md) + - [Lowering MIR](./backend/lowering-mir.md) + - [Code Generation](./backend/codegen.md) + - [Updating LLVM](./backend/updating-llvm.md) + - [Debugging LLVM](./backend/debugging.md) + - [Backend Agnostic Codegen](./backend/backend-agnostic.md) + - [Implicit Caller Location](./codegen/implicit-caller-location.md) + - [Profile-guided Optimization](./profile-guided-optimization.md) + - [Sanitizers Support](./sanitizers.md) + - [Debugging Support in Rust Compiler](./debugging-support-in-rustc.md) --- diff --git a/src/backend/backend.md b/src/part-5-intro.md similarity index 99% rename from src/backend/backend.md rename to src/part-5-intro.md index 0eaa72036..2b3cbee5e 100644 --- a/src/backend/backend.md +++ b/src/part-5-intro.md @@ -48,3 +48,4 @@ crate contains code specific to LLVM codegen. At a very high level, the entry point is [`rustc_codegen_ssa::base::codegen_crate`][codegen1]. This function starts the process discussed in the rest of this chapter. + From 7d4a8e001d092977b7bf052ba144b5377e3c80a6 Mon Sep 17 00:00:00 2001 From: mark Date: Tue, 31 Mar 2020 22:11:42 -0500 Subject: [PATCH 03/13] part 2, 3, 4 intros --- src/part-2-intro.md | 21 ++++++++++++--------- src/part-3-intro.md | 7 +++++++ src/part-4-intro.md | 11 +++++++++++ 3 files changed, 30 insertions(+), 9 deletions(-) create mode 100644 src/part-3-intro.md create mode 100644 src/part-4-intro.md diff --git a/src/part-2-intro.md b/src/part-2-intro.md index 1a47f21e5..f3eec8bf7 100644 --- a/src/part-2-intro.md +++ b/src/part-2-intro.md @@ -1,12 +1,15 @@ # Part 2: How rustc works -This part of the guide describes how the compiler works. It goes through -everything from high-level structure of the compiler to how each stage of -compilation works. +The remaining parts of this guide discuss how the compiler works. They go +through everything from high-level structure of the compiler to how each stage +of compilation works. They should be friendly to both readers interested in the +end-to-end process of compilation _and_ readers interested in learning about a +specific system they wish to contribute to. If anything is unclear, feel free +to file an issue on the [rustc-dev-guide +repo](https://github.com/rust-lang/rustc-dev-guide) or contact the compiler +team, as detailed in [this chapter from Part 1](./compiler-team.md). -This section should be friendly to both readers interested in the end-to-end -process of compilation _and_ readers interested in learning about a specific -system they wish to contribute to. If anything is unclear, feel free to file -an issue on the [rustc-dev-guide repo](https://github.com/rust-lang/rustc-dev-guide) -or contact the compiler team, as detailed in [this chapter from Part -1](./compiler-team.md). +In this part, we will specifically look at the high-level architecture of the +compiler. Specifically, will look at the query system, incremental compilation, +and interning. These are three overarching design choices that impact the whole +compiler. diff --git a/src/part-3-intro.md b/src/part-3-intro.md new file mode 100644 index 000000000..89636a60f --- /dev/null +++ b/src/part-3-intro.md @@ -0,0 +1,7 @@ +# Part 3: The Compiler Frontend + +This part of the guide looks at the compiler frontend (with the major exception +of the type system, which is in the next part). This part describes the process +of taking raw source code from the user and transforming it into various forms +that the compiler can work with easily. These are called intermediate +representations. diff --git a/src/part-4-intro.md b/src/part-4-intro.md new file mode 100644 index 000000000..9b0ed08fc --- /dev/null +++ b/src/part-4-intro.md @@ -0,0 +1,11 @@ +# Part 4: The Type System + +This part discusses the many analyses that the compiler uses to check various +properties of the code and to inform later stages. This includes the +representation, inference, and checking of types, the trait system, and the +borrow checker. These analyses do not happen as one big pass or set of +contiguous passes. Rather, they are spread out throughout various parts of the +compilation process and use different intermediate representations. For example, +type checking happens on the HIR, while borrow checking happens on the MIR. +Nonetheless, for the sake of presentation, we will discuss all of these +analyses in this part of the guide. From 1606814399c16f4af5b95b3201f2a20b28db99b3 Mon Sep 17 00:00:00 2001 From: mark Date: Tue, 31 Mar 2020 22:15:32 -0500 Subject: [PATCH 04/13] some missing files --- src/SUMMARY.md | 1 - src/ast-validation.md | 6 ++++++ src/pat-exhaustive-checking.md | 7 +++++++ 3 files changed, 13 insertions(+), 1 deletion(-) create mode 100644 src/ast-validation.md create mode 100644 src/pat-exhaustive-checking.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index f47902275..cf415bebe 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -114,7 +114,6 @@ - [Constant evaluation](./const-eval.md) - [miri const evaluator](./miri.md) - [Monomorphization](./backend/monomorph.md) - - [Compiler Backend](./backend/backend.md) - [Lowering MIR](./backend/lowering-mir.md) - [Code Generation](./backend/codegen.md) - [Updating LLVM](./backend/updating-llvm.md) diff --git a/src/ast-validation.md b/src/ast-validation.md new file mode 100644 index 000000000..64e35bb93 --- /dev/null +++ b/src/ast-validation.md @@ -0,0 +1,6 @@ +# AST Validation + +AST validation is the process of checking various correctness properties about +the AST after macro expansion. + +**TODO**: write this chapter. diff --git a/src/pat-exhaustive-checking.md b/src/pat-exhaustive-checking.md new file mode 100644 index 000000000..b29bb1351 --- /dev/null +++ b/src/pat-exhaustive-checking.md @@ -0,0 +1,7 @@ +# Pattern and Exhaustiveness Checking + +In Rust, pattern matching and bindings have a few very helpful properties. The +compiler will check that bindings are irrefutable when made and that match arms +are exhaustive. + +**TODO**: write this chapter. From 1f7cc532bf68f545b234a8f366b14eb40f942580 Mon Sep 17 00:00:00 2001 From: Who? Me?! Date: Wed, 1 Apr 2020 10:17:36 -0500 Subject: [PATCH 05/13] Fix typos Co-Authored-By: Yuki Okushi Co-Authored-By: Chris Simpkins --- src/SUMMARY.md | 2 +- src/part-2-intro.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index cf415bebe..7a43994fb 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -59,7 +59,7 @@ - [Lowering AST to HIR](./lowering.md) - [Debugging](./hir-debugging.md) - [The MIR (Mid-level IR)](./mir/index.md) - - [HAIR MIR construction](./mir/construction.md) + - [HAIR and MIR construction](./mir/construction.md) - [MIR visitor and traversal](./mir/visitor.md) - [MIR passes: getting the MIR for a function](./mir/passes.md) - [Closure expansion](./closure.md) diff --git a/src/part-2-intro.md b/src/part-2-intro.md index f3eec8bf7..dc774f0c6 100644 --- a/src/part-2-intro.md +++ b/src/part-2-intro.md @@ -6,7 +6,7 @@ of compilation works. They should be friendly to both readers interested in the end-to-end process of compilation _and_ readers interested in learning about a specific system they wish to contribute to. If anything is unclear, feel free to file an issue on the [rustc-dev-guide -repo](https://github.com/rust-lang/rustc-dev-guide) or contact the compiler +repo](https://github.com/rust-lang/rustc-dev-guide/issues) or contact the compiler team, as detailed in [this chapter from Part 1](./compiler-team.md). In this part, we will specifically look at the high-level architecture of the From 72a7022d1e657e0ab8e81aabb36ef02a559af1d9 Mon Sep 17 00:00:00 2001 From: mark Date: Wed, 1 Apr 2020 10:29:10 -0500 Subject: [PATCH 06/13] add links from intro --- src/about-this-guide.md | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/src/about-this-guide.md b/src/about-this-guide.md index 681aa7756..562633a11 100644 --- a/src/about-this-guide.md +++ b/src/about-this-guide.md @@ -6,15 +6,25 @@ development. There are six parts to this guide: -1. Contains information that should be useful no matter how you are - contributing, such as procedures for contribution, building the compiler, - etc. -2. Discusses the high-level architecture of the compiler, especially the query - system. -3. Discusses the compiler frontend and internal representations. -4. Discusses the type system. -5. Discusses the compiler backend, code generation, linking, and debug info. -6. Appendices at the end with useful reference information. +1. [Contributing][p1]: Contains information that should be useful no matter how + you are contributing, such as procedures for contribution, building the + compiler, etc. +2. [High-level Compiler Architecture][p2]: Discusses the high-level + architecture of the compiler, especially the query system. +3. [The Compiler Frontend][p3]: Discusses the compiler frontend and internal + representations. +4. [The Type System][p4]: Discusses the type system. +5. [The Compiler Backend][p5]: Discusses the compiler backend, code generation, + linking, and debug info. +6. [Appendices][app] at the end with useful reference information. There are a + few of these with different information, inluding a glossary. + +[p1]: ./part-1-intro.md +[p2]: ./part-2-intro.md +[p3]: ./part-3-intro.md +[p4]: ./part-4-intro.md +[p5]: ./part-5-intro.md +[app]: ./appendix/stupid-stats.md The guide itself is of course open-source as well, and the sources can be found at the [GitHub repository]. If you find any mistakes in the From 8e3fcaf99bffe695db8374d38872533b5f3256a1 Mon Sep 17 00:00:00 2001 From: mark Date: Fri, 3 Apr 2020 11:13:42 -0500 Subject: [PATCH 07/13] rename compiler source chapter and add placeholder for overview --- src/SUMMARY.md | 3 ++- src/{high-level-overview.md => compiler-src.md} | 0 src/overview.md | 3 +++ 3 files changed, 5 insertions(+), 1 deletion(-) rename src/{high-level-overview.md => compiler-src.md} (100%) create mode 100644 src/overview.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 7a43994fb..fbe9307f7 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -35,10 +35,11 @@ - [Licenses](./licenses.md) - [Part 2: High-level Compiler Architecture](./part-2-intro.md) - - [High-level overview of the compiler source](./high-level-overview.md) - [The Rustc Driver and Interface](./rustc-driver.md) - [Rustdoc](./rustdoc.md) - [Ex: Type checking through `rustc_interface`](./rustc-driver-interacting-with-the-ast.md) + - [Overview of the Compiler](./overview.md) + - [The compiler source code](./compiler-src.md) - [Queries: demand-driven compilation](./query.md) - [The Query Evaluation Model in Detail](./queries/query-evaluation-model-in-detail.md) - [Incremental compilation](./queries/incremental-compilation.md) diff --git a/src/high-level-overview.md b/src/compiler-src.md similarity index 100% rename from src/high-level-overview.md rename to src/compiler-src.md diff --git a/src/overview.md b/src/overview.md new file mode 100644 index 000000000..f5767a3c8 --- /dev/null +++ b/src/overview.md @@ -0,0 +1,3 @@ +# Overview of the Compiler + +See https://github.com/rust-lang/rustc-dev-guide/pull/633 From 047443f32efe992f372e3dfaebeeef4bd2f60cb7 Mon Sep 17 00:00:00 2001 From: mark Date: Fri, 3 Apr 2020 11:15:13 -0500 Subject: [PATCH 08/13] rename parts, mv rustc_driver to part 3, make syntax chapter --- src/SUMMARY.md | 26 ++++++++++++++------------ src/feature-gate-ck.md | 3 +++ 2 files changed, 17 insertions(+), 12 deletions(-) create mode 100644 src/feature-gate-ck.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index fbe9307f7..d6cbe0645 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -35,9 +35,6 @@ - [Licenses](./licenses.md) - [Part 2: High-level Compiler Architecture](./part-2-intro.md) - - [The Rustc Driver and Interface](./rustc-driver.md) - - [Rustdoc](./rustdoc.md) - - [Ex: Type checking through `rustc_interface`](./rustc-driver-interacting-with-the-ast.md) - [Overview of the Compiler](./overview.md) - [The compiler source code](./compiler-src.md) - [Queries: demand-driven compilation](./query.md) @@ -49,13 +46,18 @@ - [Salsa](./salsa.md) - [Memory Management in Rustc](./memory.md) -- [Part 3: The Compiler Frontend](./part-3-intro.md) - - [Lexing and Parsing](./the-parser.md) - - [`#[test]` Implementation](./test-implementation.md) - - [Panic Implementation](./panic-implementation.md) - - [Macro expansion](./macro-expansion.md) - - [Name resolution](./name-resolution.md) - - [AST Validation](./ast-validation.md) +- [Part 3: Source Code Representations](./part-3-intro.md) + - [The Rustc Driver and Interface](./rustc-driver.md) + - [Rustdoc](./rustdoc.md) + - [Ex: Type checking through `rustc_interface`](./rustc-driver-interacting-with-the-ast.md) + - [Syntax and the AST](./syntax-intro.md) + - [Lexing and Parsing](./the-parser.md) + - [`#[test]` Implementation](./test-implementation.md) + - [Panic Implementation](./panic-implementation.md) + - [Macro expansion](./macro-expansion.md) + - [Name resolution](./name-resolution.md) + - [AST Validation](./ast-validation.md) + - [Feature Gate Checking](./feature-gate-ck.md) - [The HIR (High-level IR)](./hir.md) - [Lowering AST to HIR](./lowering.md) - [Debugging](./hir-debugging.md) @@ -65,7 +67,7 @@ - [MIR passes: getting the MIR for a function](./mir/passes.md) - [Closure expansion](./closure.md) -- [Part 4: The Type System](./part-4-intro.md) +- [Part 4: Analysis](./part-4-intro.md) - [The `ty` module: representing types](./ty.md) - [Generics and substitutions](./generics.md) - [`TypeFolder` and `TypeFoldable`](./ty-fold.md) @@ -108,7 +110,7 @@ - [Two-phase-borrows](./borrow_check/two_phase_borrows.md) - [Parameter Environments](./param_env.md) -- [Part 5: The Compiler Backend](./part-5-intro.md) +- [Part 5: From MIR to binaries](./part-5-intro.md) - [The MIR (Mid-level IR)](./mir/index.md) - [MIR optimizations](./mir/optimizations.md) - [Debugging](./mir/debugging.md) diff --git a/src/feature-gate-ck.md b/src/feature-gate-ck.md new file mode 100644 index 000000000..3b3a07a31 --- /dev/null +++ b/src/feature-gate-ck.md @@ -0,0 +1,3 @@ +# Feature Gate Checking + +**TODO**: this chapter From 55d57f0545fd00e5199c87d6413a1686d8beaa1b Mon Sep 17 00:00:00 2001 From: mark Date: Fri, 3 Apr 2020 11:28:26 -0500 Subject: [PATCH 09/13] update part intros --- src/part-3-intro.md | 14 ++++++++------ src/part-4-intro.md | 5 +++-- src/part-5-intro.md | 17 +++++++++-------- 3 files changed, 20 insertions(+), 16 deletions(-) diff --git a/src/part-3-intro.md b/src/part-3-intro.md index 89636a60f..d3fd2a5bf 100644 --- a/src/part-3-intro.md +++ b/src/part-3-intro.md @@ -1,7 +1,9 @@ -# Part 3: The Compiler Frontend +# Part 3: Source Code Representations -This part of the guide looks at the compiler frontend (with the major exception -of the type system, which is in the next part). This part describes the process -of taking raw source code from the user and transforming it into various forms -that the compiler can work with easily. These are called intermediate -representations. +This part of the guide looks at the various ways the compiler transforms and +represents source code. This part describes the process of taking raw source +code from the user and transforming it into various forms that the compiler can +work with easily. These are called intermediate representations. + +This process starts with compiler understanding what the user has asked for: +parsing the command line arguments given and determining what it is to compile. diff --git a/src/part-4-intro.md b/src/part-4-intro.md index 9b0ed08fc..00a74f308 100644 --- a/src/part-4-intro.md +++ b/src/part-4-intro.md @@ -1,7 +1,8 @@ -# Part 4: The Type System +# Part 4: Analysis This part discusses the many analyses that the compiler uses to check various -properties of the code and to inform later stages. This includes the +properties of the code and to inform later stages. Typically, this is what people +mean when they talk about "Rust's type system". This includes the representation, inference, and checking of types, the trait system, and the borrow checker. These analyses do not happen as one big pass or set of contiguous passes. Rather, they are spread out throughout various parts of the diff --git a/src/part-5-intro.md b/src/part-5-intro.md index 2b3cbee5e..b718327c7 100644 --- a/src/part-5-intro.md +++ b/src/part-5-intro.md @@ -1,4 +1,4 @@ -# The Compiler Backend +# From MIR to binaries All of the preceding chapters of this guide have one thing in common: we never generated any executable machine code at all! With this chapter, all of that @@ -7,13 +7,14 @@ changes. It's often useful to think of compilers as being composed of a _frontend_ and a _backend_ (though in rustc, there's not a sharp line between frontend and backend). The _frontend_ is responsible for taking raw source code, checking it -for correctness, and getting it into a format usable by the backend. For rustc, -this format is the MIR. The _backend_ refers to the parts of the compiler that -turn rustc's MIR into actual executable code (e.g. an ELF or EXE binary) that -can run on a processor. All of the previous chapters deal with rustc's -frontend. - -rustc's backend does the following: +for correctness, and getting it into a format from which we can generate code. +For rustc, we typically consider MIR to be this format (though arguably, one +could consider LLVM IR to be this format). The _backend_ refers to the parts +of the compiler that produce actual executable code (e.g. an ELF or EXE binary) +that can run on a processor. All of the previous chapters deal with rustc's +"frontend". + +rustc's "backend" does the following: 0. First, we need to collect the set of things to generate code for. In particular, we need to find out which concrete types to substitute for From 96265e8c5f79540d881804767c20e13e538132fa Mon Sep 17 00:00:00 2001 From: mark Date: Fri, 3 Apr 2020 11:33:38 -0500 Subject: [PATCH 10/13] add syntax ch intro --- src/syntax-intro.md | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 src/syntax-intro.md diff --git a/src/syntax-intro.md b/src/syntax-intro.md new file mode 100644 index 000000000..dd7e2d735 --- /dev/null +++ b/src/syntax-intro.md @@ -0,0 +1,8 @@ +# Syntax and the AST + +Working directly with source code is very inconvenient and error-prone. Thus, +before we do anything else, we convert raw source code into an AST. It turns +out that doing even this involves a lot of work, including lexing, parsing, +macro expansion, name resolution, conditional compilation, feature-gate +checking, and validation of the AST. In this chapter, we take a look at all +of these steps. From 2041af587542d4aab591566968be70275f75e8fa Mon Sep 17 00:00:00 2001 From: mark Date: Sat, 4 Apr 2020 13:57:21 -0500 Subject: [PATCH 11/13] address review comments --- src/part-3-intro.md | 7 +++---- src/part-5-intro.md | 30 +++++++++++++++--------------- 2 files changed, 18 insertions(+), 19 deletions(-) diff --git a/src/part-3-intro.md b/src/part-3-intro.md index d3fd2a5bf..a1ec3ca90 100644 --- a/src/part-3-intro.md +++ b/src/part-3-intro.md @@ -1,9 +1,8 @@ # Part 3: Source Code Representations -This part of the guide looks at the various ways the compiler transforms and -represents source code. This part describes the process of taking raw source -code from the user and transforming it into various forms that the compiler can -work with easily. These are called intermediate representations. +This part describes the process of taking raw source code from the user and +transforming it into various forms that the compiler can work with easily. +These are called intermediate representations. This process starts with compiler understanding what the user has asked for: parsing the command line arguments given and determining what it is to compile. diff --git a/src/part-5-intro.md b/src/part-5-intro.md index b718327c7..41c3eceb2 100644 --- a/src/part-5-intro.md +++ b/src/part-5-intro.md @@ -4,17 +4,17 @@ All of the preceding chapters of this guide have one thing in common: we never generated any executable machine code at all! With this chapter, all of that changes. -It's often useful to think of compilers as being composed of a _frontend_ and a -_backend_ (though in rustc, there's not a sharp line between frontend and -backend). The _frontend_ is responsible for taking raw source code, checking it -for correctness, and getting it into a format from which we can generate code. -For rustc, we typically consider MIR to be this format (though arguably, one -could consider LLVM IR to be this format). The _backend_ refers to the parts -of the compiler that produce actual executable code (e.g. an ELF or EXE binary) -that can run on a processor. All of the previous chapters deal with rustc's -"frontend". - -rustc's "backend" does the following: +So far, we've shown how the compiler can take raw source code in text format +and transform it into MIR. We have also shown how the compiler does various +analyses on the code to detect things like type or lifetime errors. Now, we +will finally take the MIR and produce some executable machine code. + +> NOTE: This part of a compiler is often called the _backend_ the term is a bit +> overloaded because in the compiler source, it usually refers to the "codegen +> backend" (i.e. LLVM or Cranelift). Usually, when you see the word "backend" +> in this part, we are refering to the "codegen backend". + +So what do we need to do? 0. First, we need to collect the set of things to generate code for. In particular, we need to find out which concrete types to substitute for @@ -24,15 +24,15 @@ rustc's "backend" does the following: collecting all the concrete types is called _monomorphization collection_. 1. Next, we need to actually lower the MIR to a codegen IR (usually LLVM IR) for each concrete type we collected. -2. Finally, we need to invoke the codegen backend (e.g. LLVM or Cranelift), - which runs a bunch of optimization passes, generates executable code, and - links together an executable binary. +2. Finally, we need to invoke LLVM or Cranelift, which runs a bunch of + optimization passes, generates executable code, and links together an + executable binary. [codegen1]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/base/fn.codegen_crate.html The code for codegen is actually a bit complex due to a few factors: -- Support for multiple backends (LLVM and Cranelift). We try to share as much +- Support for multiple codegen backends (LLVM and Cranelift). We try to share as much backend code between them as possible, so a lot of it is generic over the codegen implementation. This means that there are often a lot of layers of abstraction. From c74f6956567271cebee6a3b067da85b44ef17050 Mon Sep 17 00:00:00 2001 From: Who? Me?! Date: Sun, 5 Apr 2020 12:52:47 -0500 Subject: [PATCH 12/13] Better overview chapter placeholder Co-Authored-By: Chris Simpkins --- src/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/overview.md b/src/overview.md index f5767a3c8..140d2f352 100644 --- a/src/overview.md +++ b/src/overview.md @@ -1,3 +1,3 @@ # Overview of the Compiler -See https://github.com/rust-lang/rustc-dev-guide/pull/633 +Coming soon! Work is in progress on this chapter. See https://github.com/rust-lang/rustc-dev-guide/pull/633 for the source and the [project README](https://github.com/rust-lang/rustc-dev-guide) for local build instructions. From de10abc78e51842be5dbaa726e9e4287eeb757af Mon Sep 17 00:00:00 2001 From: mark Date: Sun, 5 Apr 2020 18:25:04 -0500 Subject: [PATCH 13/13] fix link --- src/about-this-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/about-this-guide.md b/src/about-this-guide.md index 562633a11..9637c312a 100644 --- a/src/about-this-guide.md +++ b/src/about-this-guide.md @@ -24,7 +24,7 @@ There are six parts to this guide: [p3]: ./part-3-intro.md [p4]: ./part-4-intro.md [p5]: ./part-5-intro.md -[app]: ./appendix/stupid-stats.md +[app]: ./appendix/background.md The guide itself is of course open-source as well, and the sources can be found at the [GitHub repository]. If you find any mistakes in the