Skip to content

Commit de1de35

Browse files
authored
docs(rust): Add comprehensive README.md documentation for all Rust crates (#12706)
1 parent 2ccf725 commit de1de35

File tree

20 files changed

+847
-0
lines changed

20 files changed

+847
-0
lines changed

crates/oxc_allocator/README.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Oxc Allocator
2+
3+
A high-performance memory allocator using bump-based arena allocation for fast AST node creation.
4+
5+
## Overview
6+
7+
Oxc uses a bump-based memory arena for faster AST allocations. This crate provides an [`Allocator`] that manages memory efficiently by allocating objects in a single contiguous memory region, avoiding the overhead of individual heap allocations.
8+
9+
## Key Features
10+
11+
- **Bump allocation**: Fast allocation by simply incrementing a pointer
12+
- **Arena-based memory management**: All allocations live for the same lifetime
13+
- **Zero-copy data structures**: Efficient `Box`, `Vec`, `String`, and `HashMap` implementations
14+
- **Optimal for AST operations**: Perfect for parse-transform-emit workflows
15+
16+
## Architecture
17+
18+
The allocator is designed specifically for AST processing workflows where:
19+
20+
1. A large number of nodes are allocated during parsing
21+
2. All nodes have the same lifetime (tied to the AST)
22+
3. Memory is released all at once when the AST is dropped
23+
24+
This approach is significantly faster than using the system allocator for AST operations.
25+
26+
## Features
27+
28+
- `serialize` - Enables serialization support for `Box` and `Vec` with `serde`
29+
- `from_raw_parts` - Adds unsafe `from_raw_parts` method (not recommended for general use)

crates/oxc_ast/README.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Oxc AST
2+
3+
Abstract Syntax Tree definitions for JavaScript, TypeScript, and JSX.
4+
5+
## Overview
6+
7+
This crate provides comprehensive AST (Abstract Syntax Tree) node definitions that support the full spectrum of JavaScript and TypeScript syntax, including JSX. The AST closely follows ECMAScript specifications while providing ergonomic APIs for manipulation.
8+
9+
## Key Features
10+
11+
- **Complete language support**: JavaScript, TypeScript, JSX, and TSX
12+
- **ECMAScript compliant**: AST structure follows official specifications
13+
- **Memory efficient**: Designed to work with `oxc_allocator` for fast allocation
14+
- **Visitor patterns**: Integrates with `oxc_ast_visit` for traversal
15+
- **Type-safe**: Leverages Rust's type system for correctness
16+
17+
## AST Design Principles
18+
19+
The AST design differs from estree in several important ways:
20+
21+
- **Explicit identifiers**: Uses specific types like `BindingIdentifier`, `IdentifierReference`, and `IdentifierName` instead of generic `Identifier`
22+
- **Precise assignment targets**: `AssignmentExpression.left` uses `AssignmentTarget` instead of generic `Pattern`
23+
- **Literal specialization**: Replaces generic `Literal` with `BooleanLiteral`, `NumericLiteral`, `StringLiteral`, etc.
24+
- **Evaluation order**: Field order follows ECMAScript evaluation order for consistency
25+
26+
## Architecture
27+
28+
The AST is designed for:
29+
30+
- **Parse**: Efficient construction during parsing
31+
- **Transform**: Easy manipulation during transpilation
32+
- **Visit**: Systematic traversal for analysis
33+
- **Codegen**: Clean conversion back to source code
34+
35+
All AST nodes are allocated in an arena (`oxc_allocator`) for optimal performance.

crates/oxc_ast_macros/README.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Oxc AST Macros
2+
3+
Procedural macros for generating AST-related code and ensuring memory layout consistency.
4+
5+
## Overview
6+
7+
This crate provides procedural macros that generate boilerplate code for AST nodes, ensuring consistent memory layout and providing derived traits automatically.
8+
9+
## Key Features
10+
11+
- **`#[ast]` attribute**: Marks types as AST nodes and generates required traits
12+
- **Memory layout control**: Ensures `#[repr(C)]` for predictable memory layout
13+
- **Trait derivation**: Automatically derives common traits like `Debug`, `Clone`, etc.
14+
- **Code generation**: Reduces boilerplate and ensures consistency across AST types
15+
16+
## What the `#[ast]` Macro Does
17+
18+
1. **Adds `#[repr(C)]`**: Ensures predictable memory layout across platforms
19+
2. **Marker for tooling**: Identifies types as AST nodes for code generation tools
20+
3. **Trait derivation**: Automatically implements common traits
21+
4. **Consistency**: Ensures all AST nodes follow the same patterns
22+
23+
## Architecture
24+
25+
This macro system enables:
26+
27+
- **Maintainable AST**: Reduces boilerplate across hundreds of AST types
28+
- **Consistent layout**: Critical for performance and correctness
29+
- **Tool integration**: Allows `oxc_ast_tools` to generate visitor code
30+
- **Type safety**: Ensures all AST nodes have required traits
31+
32+
The macros are designed to be transparent and generate minimal, efficient code.

crates/oxc_ast_visit/README.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Oxc AST Visit
2+
3+
Visitor pattern implementations for traversing and mutating AST nodes.
4+
5+
## Overview
6+
7+
This crate provides visitor traits and implementations for systematically traversing AST nodes. It supports both immutable visitation (`Visit`) and mutable transformation (`VisitMut`).
8+
9+
## Key Features
10+
11+
- **`Visit` trait**: Immutable AST traversal for analysis
12+
- **`VisitMut` trait**: Mutable AST traversal for transformations
13+
- **Generated implementations**: Most visitor code is auto-generated for consistency
14+
- **UTF-8 to UTF-16 conversion**: Special visitors for span conversion
15+
16+
## Architecture
17+
18+
The visitor pattern is designed for:
19+
20+
- **Analysis**: Static analysis, linting, and code inspection
21+
- **Transformation**: Code modification and transpilation
22+
- **Consistency**: Systematic traversal of all AST nodes
23+
- **Performance**: Efficient traversal with minimal overhead
24+
25+
Most visitor implementations are generated by `oxc_ast_tools` to ensure all AST nodes are covered and traversal is consistent.

crates/oxc_cfg/README.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Oxc CFG
2+
3+
Control Flow Graph construction and analysis for JavaScript and TypeScript.
4+
5+
## Overview
6+
7+
This crate provides data structures and algorithms for building and analyzing Control Flow Graphs (CFGs) from AST nodes. CFGs are essential for advanced static analysis, optimization, and understanding program flow.
8+
9+
## Key Features
10+
11+
- **CFG construction**: Build control flow graphs from AST nodes
12+
- **Block-based representation**: Organizes code into basic blocks
13+
- **Graph analysis**: Traverse and analyze control flow patterns
14+
- **DOT export**: Visualize CFGs using Graphviz dot format
15+
- **Visitor integration**: Works with oxc visitor patterns
16+
17+
## Architecture
18+
19+
### Basic Blocks
20+
21+
The CFG organizes code into basic blocks - sequences of instructions with:
22+
23+
- Single entry point (first instruction)
24+
- Single exit point (last instruction)
25+
- No internal branches or jumps
26+
27+
### Graph Structure
28+
29+
- **Nodes**: Basic blocks containing instructions
30+
- **Edges**: Control flow between blocks (conditional, unconditional, exception)
31+
- **Entry/Exit**: Special blocks for function entry and exit points
32+
33+
### Analysis Applications
34+
35+
- **Dead code elimination**: Find unreachable blocks
36+
- **Data flow analysis**: Track variable usage across control paths
37+
- **Loop detection**: Identify back edges and loop structures
38+
- **Path analysis**: Enumerate possible execution paths
39+
40+
The CFG integrates with semantic analysis to provide comprehensive program understanding.

crates/oxc_codegen/README.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Oxc Codegen
2+
3+
High-performance code generation from AST back to JavaScript/TypeScript source code.
4+
5+
## Overview
6+
7+
This crate converts AST nodes back into source code strings, supporting JavaScript, TypeScript, and JSX. It's designed for speed and correctness, producing clean, readable output.
8+
9+
## Key Features
10+
11+
- **Fast code generation**: Optimized for performance with minimal allocations
12+
- **Source map support**: Generate accurate source maps during output
13+
- **Configurable formatting**: Control whitespace, semicolons, and other formatting options
14+
- **Comment preservation**: Maintain comments during code generation
15+
- **Binary expression optimization**: Intelligent parentheses insertion
16+
17+
## Architecture
18+
19+
### Code Generation Pipeline
20+
21+
1. **AST Traversal**: Walk through AST nodes systematically
22+
2. **Token Generation**: Convert nodes to appropriate tokens/strings
23+
3. **Formatting**: Apply whitespace, indentation, and style rules
24+
4. **Source Mapping**: Track original source positions if enabled
25+
26+
### Design Principles
27+
28+
- **Correctness**: Generated code must be functionally equivalent to original
29+
- **Performance**: Minimize string allocations and copying
30+
- **Readability**: Produce clean, well-formatted output
31+
- **Fidelity**: Preserve semantic meaning and behavior
32+
33+
The codegen is adapted from esbuild's approach, optimized for Rust and oxc's AST structure.
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Oxc Data Structures
2+
3+
Common data structures and utilities used across oxc crates.
4+
5+
## Overview
6+
7+
This crate provides specialized data structures and utilities that are used throughout the oxc toolchain. These structures are optimized for the specific needs of compiler and tooling workloads.
8+
9+
## Key Features
10+
11+
- **Code buffer**: Efficient string building with segment tracking
12+
- **Inline strings**: Memory-efficient string storage for short strings
13+
- **Pointer extensions**: Utilities for safe pointer manipulation
14+
- **Slice iterators**: Enhanced iteration capabilities for slices
15+
- **Rope data structure**: Efficient text manipulation for large documents
16+
17+
## Architecture
18+
19+
These data structures are designed with specific compiler requirements in mind:
20+
21+
- **Performance**: Optimized for common patterns in parsing and code generation
22+
- **Memory efficiency**: Minimize allocations and memory overhead
23+
- **Safety**: Provide safe abstractions over potentially unsafe operations
24+
- **Ergonomics**: Easy to use APIs that integrate well with other oxc components
25+
26+
The structures complement Rust's standard library with domain-specific optimizations for JavaScript/TypeScript tooling.

crates/oxc_diagnostics/README.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Oxc Diagnostics
2+
3+
Error reporting and diagnostic utilities for JavaScript and TypeScript tooling.
4+
5+
## Overview
6+
7+
This crate provides comprehensive error handling and diagnostic reporting capabilities. It implements the [miette] diagnostic trait, making it compatible with other Rust diagnostic tooling while providing specialized features for JavaScript/TypeScript errors.
8+
9+
## Key Features
10+
11+
- **Rich diagnostics**: Detailed error messages with source context
12+
- **Source highlighting**: Syntax-highlighted error locations
13+
- **Multiple error support**: Collect and report multiple errors at once
14+
- **Miette integration**: Compatible with the miette diagnostic ecosystem
15+
- **Severity levels**: Support for errors, warnings, and informational messages
16+
17+
## Architecture
18+
19+
### Diagnostic Components
20+
21+
- **Message**: Primary error/warning description
22+
- **Labels**: Highlight specific source locations
23+
- **Help text**: Suggestions for fixing the issue
24+
- **Source context**: Display relevant source code sections
25+
26+
### Error Flow
27+
28+
1. **Creation**: Tools create `OxcDiagnostic` instances for problems
29+
2. **Collection**: `DiagnosticService` aggregates diagnostics
30+
3. **Formatting**: Rich terminal output with colors and context
31+
4. **Reporting**: Display to users in IDE, CLI, or other interfaces
32+
33+
The diagnostic system ensures consistent, high-quality error reporting across all oxc tools.

crates/oxc_estree/README.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Oxc ESTree
2+
3+
ESTree compatibility layer for serialization and interoperability.
4+
5+
## Overview
6+
7+
This crate provides compatibility with the ESTree AST specification, primarily for serialization purposes. It enables oxc AST nodes to be serialized to and from JSON in ESTree format, facilitating interoperability with other JavaScript tools.
8+
9+
## Key Features
10+
11+
- **ESTree compatibility**: Convert oxc AST to/from standard ESTree format
12+
- **Serialization support**: JSON serialization/deserialization via serde
13+
- **Tool interoperability**: Enable integration with ESTree-based tools
14+
- **Optional feature**: Only enabled when `serialize` feature is active
15+
16+
## Usage
17+
18+
## Architecture
19+
20+
### ESTree Specification
21+
22+
ESTree is a community standard for representing JavaScript AST nodes. This crate ensures oxc's AST can be represented in this standard format while maintaining compatibility with the broader JavaScript tooling ecosystem.
23+
24+
### Design Principles
25+
26+
- **Compatibility**: Full compatibility with ESTree specification
27+
- **Optional overhead**: Only included when serialization is needed
28+
- **Type safety**: Maintains Rust's type safety during conversion
29+
- **Performance**: Efficient serialization with minimal overhead
30+
31+
### Use Cases
32+
33+
- **IDE integration**: Language servers communicating via JSON
34+
- **Tool interoperability**: Working with Babel, ESLint, and other ESTree tools
35+
- **Data exchange**: Transferring AST data between different systems
36+
- **Debugging**: Human-readable AST representation
37+
38+
When the `serialize` feature is disabled, this crate provides only a placeholder trait to support derive macros without overhead.
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Oxc Isolated Declarations
2+
3+
TypeScript isolated declarations transformer for generating `.d.ts` files.
4+
5+
## Overview
6+
7+
This crate implements TypeScript's isolated declarations feature, which generates TypeScript declaration files (`.d.ts`) from source code without requiring full type checking. This enables faster builds and better incremental compilation.
8+
9+
## Key Features
10+
11+
- **Fast declaration generation**: Generate `.d.ts` files without full type checking
12+
- **TypeScript 5.5+ compatibility**: Implements the latest isolated declarations specification
13+
- **Incremental builds**: Enables faster TypeScript compilation workflows
14+
- **Comprehensive support**: Handles classes, functions, interfaces, and complex types
15+
16+
## Architecture
17+
18+
### Isolated Declarations Concept
19+
20+
Isolated declarations allow generating TypeScript declaration files without full type inference by requiring that:
21+
22+
- All exported functions have explicit return types
23+
- All exported variables have explicit types
24+
- Type information is locally available
25+
26+
### Implementation Details
27+
28+
- **AST transformation**: Convert implementation AST to declaration AST
29+
- **Type extraction**: Extract and preserve type information
30+
- **Export analysis**: Determine what needs to be included in declarations
31+
- **Error reporting**: Provide helpful diagnostics for missing type annotations
32+
33+
### Benefits
34+
35+
- **Faster builds**: No full type checking required
36+
- **Incremental compilation**: Each file can be processed independently
37+
- **Parallel processing**: Multiple files can be processed simultaneously
38+
- **Simplified tooling**: Easier to integrate into build systems
39+
40+
This implementation follows the TypeScript compiler's approach while leveraging oxc's performance-oriented architecture.

0 commit comments

Comments
 (0)