Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Oxide candidate extractor [0] #16306

Merged
merged 44 commits into from
Mar 5, 2025
Merged
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
1bde46d
start of Oxide scanner improvements
RobinMalfait Feb 6, 2025
d9369ea
bump Rust version
RobinMalfait Feb 24, 2025
157ef40
bump Rust dependencies
RobinMalfait Feb 24, 2025
85341d7
apply `cargo clippy`
RobinMalfait Feb 24, 2025
8208f1d
make `move_to` less branchy
RobinMalfait Feb 24, 2025
15aaae6
add `advance()` shorthand
RobinMalfait Feb 24, 2025
529c92c
add throughput helper for benchmarks
RobinMalfait Feb 26, 2025
6dd19f7
add example fixture file
RobinMalfait Feb 26, 2025
a945f51
adjust internal `ChangedContent` use an enum instead
RobinMalfait Feb 26, 2025
8deb2ab
add `Machine`
RobinMalfait Feb 26, 2025
49885ce
add `BracketStack`
RobinMalfait Feb 26, 2025
4a78d4c
add `StringMachine`
RobinMalfait Feb 26, 2025
d9aa816
add `CssVariableMachine`
RobinMalfait Feb 26, 2025
bfe611d
add `ArbitraryValueMachine`
RobinMalfait Feb 26, 2025
38748b4
add `ArbitraryVariableMachine`
RobinMalfait Feb 26, 2025
dc66379
add `ArbitraryPropertyMachine`
RobinMalfait Feb 26, 2025
4763d3f
add `ModifierMachine`
RobinMalfait Feb 26, 2025
4f1f668
add `NamedVariantMachine`
RobinMalfait Feb 26, 2025
68c44fa
add `NamedUtilityMachine`
RobinMalfait Feb 26, 2025
ef366d0
add `UtilityMachine`
RobinMalfait Feb 26, 2025
e61fd30
add `VariantMachine`
RobinMalfait Feb 26, 2025
97eb783
add `CandidateMachine`
RobinMalfait Feb 26, 2025
b1c2be4
add new extractor
RobinMalfait Feb 26, 2025
b7027f9
remove old parser
RobinMalfait Feb 26, 2025
bdf2e81
wire up new Extractor
RobinMalfait Feb 26, 2025
f911de9
adjust test from the scanner
RobinMalfait Feb 26, 2025
cc11fb5
update changelog
RobinMalfait Feb 26, 2025
60e5f67
add `advance_twice` method
RobinMalfait Feb 26, 2025
3ab7a52
use `cursor.advance_twice()`
RobinMalfait Feb 26, 2025
6bbf003
simplify ArbitraryValueMachine
RobinMalfait Feb 26, 2025
21fe019
support emoji in CSS Variables
RobinMalfait Feb 26, 2025
f083c9d
remove `cursor.rewind_by`
RobinMalfait Feb 26, 2025
36905b5
Merge branch 'main' into feat/improve-oxide-scanner
RobinMalfait Feb 27, 2025
8cf78f0
add dedicated pre-processors for certain file extensions
RobinMalfait Feb 27, 2025
869e557
remove tests when we already have dedicated tests for them
RobinMalfait Feb 27, 2025
b00d370
add more angular binding attribute tests
RobinMalfait Feb 27, 2025
a11c471
adjust tests with better boundaries
RobinMalfait Feb 27, 2025
b0968b6
add a few more CSS Variable tests
RobinMalfait Feb 27, 2025
3d8c41a
improve comments and fix typos
RobinMalfait Feb 27, 2025
9564525
drop `Class::AlphaUpper`
RobinMalfait Feb 27, 2025
fb235a2
use a set_range!
RobinMalfait Feb 27, 2025
df51fba
Merge branch 'main' into feat/improve-oxide-scanner
RobinMalfait Feb 27, 2025
072d8d2
Merge branch 'main' into feat/improve-oxide-scanner
RobinMalfait Mar 2, 2025
995948d
Merge branch 'main' into feat/improve-oxide-scanner
RobinMalfait Mar 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion crates/oxide/src/extractor/css_variable_machine.rs
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,13 @@ const CLASS_TABLE: [Class; 256] = {
set_range!(Class::AllowedCharacter, b'A'..=b'Z');
set_range!(Class::AllowedCharacter, b'0'..=b'9');

// non-ASCII (such as Emoji): https://drafts.csswg.org/css-syntax-3/#non-ascii-ident-code-point
let mut i = 0x80;
while i <= 0xff {
table[i as usize] = Class::AllowedCharacter;
i += 1;
}

set!(Class::End, b'\0');

table
Expand Down Expand Up @@ -167,6 +174,9 @@ mod tests {
"calc(var(--first) + var(--second))",
vec!["--first", "--second"],
),
// Variables with... emojis
("--😀", vec!["--😀"]),
("--😀-😁", vec!["--😀-😁"]),
// Escaped character in the middle, skips the next character
(r#"--spacing-1\/2"#, vec![r#"--spacing-1\/2"#]),
// Escaped whitespace is not allowed
Expand Down Expand Up @@ -223,7 +233,13 @@ mod tests {
] {
let input = wrapper.replace("{}", input);

assert_eq!(CssVariableMachine::test_extract_all(&input), expected);
let actual = CssVariableMachine::test_extract_all(&input);

if actual != expected {
dbg!(&input, &actual, &expected);
}

assert_eq!(actual, expected);
}
}
}
Expand Down