Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Oxide candidate extractor [0] #16306

Merged
merged 44 commits into from
Mar 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
1bde46d
start of Oxide scanner improvements
RobinMalfait Feb 6, 2025
d9369ea
bump Rust version
RobinMalfait Feb 24, 2025
157ef40
bump Rust dependencies
RobinMalfait Feb 24, 2025
85341d7
apply `cargo clippy`
RobinMalfait Feb 24, 2025
8208f1d
make `move_to` less branchy
RobinMalfait Feb 24, 2025
15aaae6
add `advance()` shorthand
RobinMalfait Feb 24, 2025
529c92c
add throughput helper for benchmarks
RobinMalfait Feb 26, 2025
6dd19f7
add example fixture file
RobinMalfait Feb 26, 2025
a945f51
adjust internal `ChangedContent` use an enum instead
RobinMalfait Feb 26, 2025
8deb2ab
add `Machine`
RobinMalfait Feb 26, 2025
49885ce
add `BracketStack`
RobinMalfait Feb 26, 2025
4a78d4c
add `StringMachine`
RobinMalfait Feb 26, 2025
d9aa816
add `CssVariableMachine`
RobinMalfait Feb 26, 2025
bfe611d
add `ArbitraryValueMachine`
RobinMalfait Feb 26, 2025
38748b4
add `ArbitraryVariableMachine`
RobinMalfait Feb 26, 2025
dc66379
add `ArbitraryPropertyMachine`
RobinMalfait Feb 26, 2025
4763d3f
add `ModifierMachine`
RobinMalfait Feb 26, 2025
4f1f668
add `NamedVariantMachine`
RobinMalfait Feb 26, 2025
68c44fa
add `NamedUtilityMachine`
RobinMalfait Feb 26, 2025
ef366d0
add `UtilityMachine`
RobinMalfait Feb 26, 2025
e61fd30
add `VariantMachine`
RobinMalfait Feb 26, 2025
97eb783
add `CandidateMachine`
RobinMalfait Feb 26, 2025
b1c2be4
add new extractor
RobinMalfait Feb 26, 2025
b7027f9
remove old parser
RobinMalfait Feb 26, 2025
bdf2e81
wire up new Extractor
RobinMalfait Feb 26, 2025
f911de9
adjust test from the scanner
RobinMalfait Feb 26, 2025
cc11fb5
update changelog
RobinMalfait Feb 26, 2025
60e5f67
add `advance_twice` method
RobinMalfait Feb 26, 2025
3ab7a52
use `cursor.advance_twice()`
RobinMalfait Feb 26, 2025
6bbf003
simplify ArbitraryValueMachine
RobinMalfait Feb 26, 2025
21fe019
support emoji in CSS Variables
RobinMalfait Feb 26, 2025
f083c9d
remove `cursor.rewind_by`
RobinMalfait Feb 26, 2025
36905b5
Merge branch 'main' into feat/improve-oxide-scanner
RobinMalfait Feb 27, 2025
8cf78f0
add dedicated pre-processors for certain file extensions
RobinMalfait Feb 27, 2025
869e557
remove tests when we already have dedicated tests for them
RobinMalfait Feb 27, 2025
b00d370
add more angular binding attribute tests
RobinMalfait Feb 27, 2025
a11c471
adjust tests with better boundaries
RobinMalfait Feb 27, 2025
b0968b6
add a few more CSS Variable tests
RobinMalfait Feb 27, 2025
3d8c41a
improve comments and fix typos
RobinMalfait Feb 27, 2025
9564525
drop `Class::AlphaUpper`
RobinMalfait Feb 27, 2025
fb235a2
use a set_range!
RobinMalfait Feb 27, 2025
df51fba
Merge branch 'main' into feat/improve-oxide-scanner
RobinMalfait Feb 27, 2025
072d8d2
Merge branch 'main' into feat/improve-oxide-scanner
RobinMalfait Mar 2, 2025
995948d
Merge branch 'main' into feat/improve-oxide-scanner
RobinMalfait Mar 5, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- _Experimental_: Add `user-valid` and `user-invalid` variants ([#12370](https://github.com/tailwindlabs/tailwindcss/pull/12370))
- _Experimental_: Add `wrap-anywhere`, `wrap-break-word`, and `wrap-normal` utilities ([#12128](https://github.com/tailwindlabs/tailwindcss/pull/12128))
- Add `col-<number>` and `row-<number>` utilities for `grid-column` and `grid-row` ([#15183](https://github.com/tailwindlabs/tailwindcss/pull/15183))
- Add new candidate extractor ([#16306](https://github.com/tailwindlabs/tailwindcss/pull/16306))

### Fixed

Expand Down
26 changes: 13 additions & 13 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions crates/node/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ crate-type = ["cdylib"]

[dependencies]
# Default enable napi4 feature, see https://nodejs.org/api/n-api.html#node-api-version-matrix
napi = { version = "2.16.11", default-features = false, features = ["napi4"] }
napi-derive = "2.16.12"
napi = { version = "2.16.16", default-features = false, features = ["napi4"] }
napi-derive = "2.16.13"
tailwindcss-oxide = { path = "../oxide" }
rayon = "1.5.3"
rayon = "1.10.0"

[build-dependencies]
napi-build = "2.0.1"
napi-build = "2.1.4"
19 changes: 15 additions & 4 deletions crates/node/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,23 @@ pub struct GlobEntry {
pub pattern: String,
}

impl From<ChangedContent> for tailwindcss_oxide::ChangedContent {
impl From<ChangedContent> for tailwindcss_oxide::ChangedContent<'_> {
fn from(changed_content: ChangedContent) -> Self {
Self {
file: changed_content.file.map(Into::into),
content: changed_content.content,
if let Some(file) = changed_content.file {
return tailwindcss_oxide::ChangedContent::File(
file.into(),
changed_content.extension.into(),
);
}

if let Some(contents) = changed_content.content {
return tailwindcss_oxide::ChangedContent::Content(
contents,
changed_content.extension.into(),
);
}

unreachable!()
}
}

Expand Down
5 changes: 3 additions & 2 deletions crates/oxide/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ version = "0.1.0"
edition = "2021"

[dependencies]
bstr = "1.10.0"
bstr = "1.11.3"
globwalk = "0.9.1"
log = "0.4.22"
rayon = "1.10.0"
fxhash = { package = "rustc-hash", version = "2.0.0" }
fxhash = { package = "rustc-hash", version = "2.1.1" }
crossbeam = "0.8.4"
tracing = { version = "0.1.40", features = [] }
tracing-subscriber = { version = "0.3.18", features = ["env-filter"] }
Expand All @@ -20,3 +20,4 @@ fast-glob = "0.4.3"

[dev-dependencies]
tempfile = "3.13.0"

54 changes: 27 additions & 27 deletions crates/oxide/src/cursor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,34 @@ impl<'a> Cursor<'a> {
cursor
}

pub fn rewind_by(&mut self, amount: usize) {
self.move_to(self.pos.saturating_sub(amount));
}

pub fn advance_by(&mut self, amount: usize) {
self.move_to(self.pos.saturating_add(amount));
}

#[inline(always)]
pub fn advance(&mut self) {
self.pos += 1;

self.prev = self.curr;
self.curr = self.next;
self.next = *self
.input
.get(self.pos.saturating_add(1))
.unwrap_or(&0x00u8);
}

#[inline(always)]
pub fn advance_twice(&mut self) {
self.pos += 2;

self.prev = self.next;
self.curr = *self.input.get(self.pos).unwrap_or(&0x00u8);
self.next = *self
.input
.get(self.pos.saturating_add(1))
.unwrap_or(&0x00u8);
}

pub fn move_to(&mut self, pos: usize) {
let len = self.input.len();
let pos = pos.clamp(0, len);
Expand All @@ -57,13 +77,9 @@ impl<'a> Cursor<'a> {
self.at_start = pos == 0;
self.at_end = pos + 1 >= len;

self.prev = if pos > 0 { self.input[pos - 1] } else { 0x00 };
self.curr = if pos < len { self.input[pos] } else { 0x00 };
self.next = if pos + 1 < len {
self.input[pos + 1]
} else {
0x00
};
self.prev = *self.input.get(pos.wrapping_sub(1)).unwrap_or(&0x00u8);
self.curr = *self.input.get(pos).unwrap_or(&0x00u8);
self.next = *self.input.get(pos.saturating_add(1)).unwrap_or(&0x00u8);
}
}

Expand Down Expand Up @@ -139,21 +155,5 @@ mod test {
assert_eq!(cursor.prev, b'd');
assert_eq!(cursor.curr, 0x00);
assert_eq!(cursor.next, 0x00);

cursor.rewind_by(1);
assert_eq!(cursor.pos, 10);
assert!(!cursor.at_start);
assert!(cursor.at_end);
assert_eq!(cursor.prev, b'l');
assert_eq!(cursor.curr, b'd');
assert_eq!(cursor.next, 0x00);

cursor.rewind_by(10);
assert_eq!(cursor.pos, 0);
assert!(cursor.at_start);
assert!(!cursor.at_end);
assert_eq!(cursor.prev, 0x00);
assert_eq!(cursor.curr, b'h');
assert_eq!(cursor.next, b'e');
}
}
Loading