Improve candidate extraction when candidates contain . characters (#17113)

This PR fixes an issue where some classes weren't properly extracted due
to some incorrect assumptions in the pre processors.

Templating languages such as Haml, Slim and Pug allow you to write
classes in a shorter way that are not properly contained inside of
strings. E.g.:
```slim
p.flex.px-2
```

These candidates are not properly extracted because there are no
bounding characters like quotes. To solve this, we pre-process it and
replace `.` with ` ` characters. This results in something like:
```
p flex px-2
```

However, this has some challenges on its own. Candidates like `px-2.5`
cannot be written in this shorthand form, instead they need to be in
strings. Now we _cannot_ replace the `.` because otherwise we would
change `px-2.5` to `px-2 5` which is wrong.

The next problem is that we need to know when they are in a "string".
This has another set of problems because these templating languages
allow you to write normal text that will eventually be the contents of
the HTML tags.

```haml
.text-red-500.text-3xl
  | This text can't should be red
                 ^ Wait, is this the start of a string now???
```

In this example, if we consider the `'` the start of a string, when it's
clearly not, how would we know it's for _sure_ not a string?

This ended up as a bit of a rabbit hole, but we came up with another
approach entirely if we think about the original problem we want to
solve which is when do we change `.` to ` ` characters.

One of the rules in our current extractor is that a `.` has to be
between 2 numbers. Which works great in a scenario like: `px-2.5`.
However, if you look at Haml or Slim syntax, this is also allowed:

```slim
p.bg-red-500.2xl:flex
           ^^^ Uh oh...
```

In this scenario, a `.` is surrounded by numbers so we shouldn't replace
it with a space. But as you can see, we clearly do... so we need another
heuristic in this case.

Luckily, one of the rules in Tailwind CSS is that a utility cannot start
with a number, but a variant _can_. This means that if we see a scenario
like `<digit>.<digit>` then we can just check if the value after the `.`
is a valid variant or not.

In this case it is a valid variant so we _do_ want to replace the `.`
with a ` ` even though we do have the `<digit>.<digit>` format.

🥴

# Test plan

1. Added additional tests.
2. Existing tests still pass

---------

Co-authored-by: Philipp Spiess <hello@philippspiess.com>

Author: RobinMalfait

CHANGELOG.md

@@ -23,7 +23,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Ensure classes between `>` and `<` are properly extracted ([#17094](https://github.com/tailwindlabs/tailwindcss/pull/17094))
 - Treat starting single quote as verbatim text in Slim ([#17085](https://github.com/tailwindlabs/tailwindcss/pull/17085))
 - Ensure `.node` and `.wasm` files are not scanned for utilities ([#17123](https://github.com/tailwindlabs/tailwindcss/pull/17123))
-- Improve performance when scanning `JSON` files ([#17125](https://github.com/tailwindlabs/tailwindcss/pull/17125))
+- Improve performance when scanning JSON files ([#17125](https://github.com/tailwindlabs/tailwindcss/pull/17125))
+- Fix extracting candidates containing dots in Haml, Pug, and Slim pre processors ([#17113](https://github.com/tailwindlabs/tailwindcss/pull/17113))
 - Don't create invalid CSS when encountering a link wrapped in square brackets ([#17129](https://github.com/tailwindlabs/tailwindcss/pull/17129))
 
 ## [4.0.12] - 2025-03-07

crates/oxide/src/extractor/pre_processors/haml.rs

@@ -1,6 +1,8 @@
 use crate::cursor;
 use crate::extractor::bracket_stack::BracketStack;
+use crate::extractor::machine::{Machine, MachineState};
 use crate::extractor::pre_processors::pre_processor::PreProcessor;
+use crate::extractor::variant_machine::VariantMachine;
 
 #[derive(Debug, Default)]
 pub struct Haml;
@@ -14,29 +16,48 @@ impl PreProcessor for Haml {
 
         while cursor.pos < len {
             match cursor.curr {
-                // Consume strings as-is
-                b'\'' | b'"' => {
-                    let len = cursor.input.len();
-                    let end_char = cursor.curr;
-
-                    cursor.advance();
-
-                    while cursor.pos < len {
-                        match cursor.curr {
-                            // Escaped character, skip ahead to the next character
-                            b'\\' => cursor.advance_twice(),
-
-                            // End of the string
-                            b'\'' | b'"' if cursor.curr == end_char => break,
+                // Only replace `.` with a space if it's not surrounded by numbers. E.g.:
+                //
+                // ```diff
+                // - .flex.items-center
+                // +  flex items-center
+                // ```
+                //
+                // But with numbers, it's allowed:
+                //
+                // ```diff
+                // - px-2.5
+                // + px-2.5
+                // ```
+                b'.' => {
+                    // Don't replace dots with spaces when inside of any type of brackets, because
+                    // this could be part of arbitrary values. E.g.: `bg-[url(https://example.com)]`
+                    //                                                                       ^
+                    if !bracket_stack.is_empty() {
+                        cursor.advance();
+                        continue;
+                    }
 
-                            // Everything else is valid
-                            _ => cursor.advance(),
-                        };
+                    // If the dot is surrounded by digits, we want to keep it. E.g.: `px-2.5`
+                    // EXCEPT if it's followed by a valid variant that happens to start with a
+                    // digit.
+                    // E.g.: `bg-red-500.2xl:flex`
+                    //                 ^^^
+                    if cursor.prev.is_ascii_digit() && cursor.next.is_ascii_digit() {
+                        let mut next_cursor = cursor.clone();
+                        next_cursor.advance();
+
+                        let mut variant_machine = VariantMachine::default();
+                        if let MachineState::Done(_) = variant_machine.next(&mut next_cursor) {
+                            result[cursor.pos] = b' ';
+                        }
+                    } else {
+                        result[cursor.pos] = b' ';
                     }
                 }
 
                 // Replace following characters with spaces if they are not inside of brackets
-                b'.' | b'#' | b'=' if bracket_stack.is_empty() => {
+                b'#' | b'=' if bracket_stack.is_empty() => {
                     result[cursor.pos] = b' ';
                 }
 
@@ -48,7 +69,7 @@ impl PreProcessor for Haml {
                     bracket_stack.push(cursor.curr);
                 }
 
-                b')' | b']' | b'}' => {
+                b')' | b']' | b'}' if !bracket_stack.is_empty() => {
                     bracket_stack.pop(cursor.curr);
 
                     // Replace closing bracket with a space
@@ -116,8 +137,56 @@ mod tests {
                 ".text-lime-500.xl:text-emerald-500#root",
                 " text-lime-500 xl:text-emerald-500 root",
             ),
+            // Dots in strings in HTML attributes stay as-is
+            (r#"<div id="px-2.5"></div>"#, r#"<div id "px-2.5"></div>"#),
         ] {
             Haml::test(input, expected);
         }
     }
+
+    #[test]
+    fn test_strings_only_occur_when_nested() {
+        let input = r#"
+            %p.mt-2.text-xl
+              The quote in the next word, can't be the start of a string
+
+            %h3.mt-24.text-center.text-4xl.font-bold.italic
+              The classes above should be extracted
+        "#;
+
+        Haml::test_extract_contains(
+            input,
+            vec![
+                // First paragraph
+                "mt-2",
+                "text-xl",
+                // second paragraph
+                "mt-24",
+                "text-center",
+                "text-4xl",
+                "font-bold",
+                "italic",
+            ],
+        );
+    }
+
+    // https://github.com/tailwindlabs/tailwindcss/pull/17051#issuecomment-2711181352
+    #[test]
+    fn test_haml_full_file() {
+        let processed = Haml.process(include_bytes!("./test-fixtures/haml/src-1.haml"));
+        let actual = std::str::from_utf8(&processed).unwrap();
+        let expected = include_str!("./test-fixtures/haml/dst-1.haml");
+
+        assert_eq!(actual, expected);
+    }
+
+    #[test]
+    fn test_arbitrary_code_followed_by_classes() {
+        let input = r#"
+            %p
+              = i < 3
+              .flex.items-center
+        "#;
+        Haml::test_extract_contains(input, vec!["flex", "items-center"]);
+    }
 }

crates/oxide/src/extractor/pre_processors/pug.rs

@@ -1,6 +1,8 @@
 use crate::cursor;
 use crate::extractor::bracket_stack::BracketStack;
+use crate::extractor::machine::{Machine, MachineState};
 use crate::extractor::pre_processors::pre_processor::PreProcessor;
+use crate::extractor::variant_machine::VariantMachine;
 
 #[derive(Debug, Default)]
 pub struct Pug;
@@ -14,32 +16,46 @@ impl PreProcessor for Pug {
 
         while cursor.pos < len {
             match cursor.curr {
-                // Consume strings as-is
-                b'\'' | b'"' => {
-                    let len = cursor.input.len();
-                    let end_char = cursor.curr;
-
-                    cursor.advance();
-
-                    while cursor.pos < len {
-                        match cursor.curr {
-                            // Escaped character, skip ahead to the next character
-                            b'\\' => cursor.advance_twice(),
+                // Only replace `.` with a space if it's not surrounded by numbers. E.g.:
+                //
+                // ```diff
+                // - .flex.items-center
+                // +  flex items-center
+                // ```
+                //
+                // But with numbers, it's allowed:
+                //
+                // ```diff
+                // - px-2.5
+                // + px-2.5
+                // ```
+                b'.' => {
+                    // Don't replace dots with spaces when inside of any type of brackets, because
+                    // this could be part of arbitrary values. E.g.: `bg-[url(https://example.com)]`
+                    //                                                                       ^
+                    if !bracket_stack.is_empty() {
+                        cursor.advance();
+                        continue;
+                    }
 
-                            // End of the string
-                            b'\'' | b'"' if cursor.curr == end_char => break,
+                    // If the dot is surrounded by digits, we want to keep it. E.g.: `px-2.5`
+                    // EXCEPT if it's followed by a valid variant that happens to start with a
+                    // digit.
+                    // E.g.: `bg-red-500.2xl:flex`
+                    //                 ^^^
+                    if cursor.prev.is_ascii_digit() && cursor.next.is_ascii_digit() {
+                        let mut next_cursor = cursor.clone();
+                        next_cursor.advance();
 
-                            // Everything else is valid
-                            _ => cursor.advance(),
-                        };
+                        let mut variant_machine = VariantMachine::default();
+                        if let MachineState::Done(_) = variant_machine.next(&mut next_cursor) {
+                            result[cursor.pos] = b' ';
+                        }
+                    } else {
+                        result[cursor.pos] = b' ';
                     }
                 }
 
-                // Replace dots with spaces
-                b'.' if bracket_stack.is_empty() => {
-                    result[cursor.pos] = b' ';
-                }
-
                 b'(' | b'[' | b'{' => {
                     bracket_stack.push(cursor.curr);
                 }
@@ -77,8 +93,45 @@ mod tests {
                 "bg-[url(https://example.com/?q=[1,2])]",
                 "bg-[url(https://example.com/?q=[1,2])]",
             ),
+            // Classes in HTML attributes
+            (r#"<div id="px-2.5"></div>"#, r#"<div id="px-2.5"></div>"#),
         ] {
             Pug::test(input, expected);
         }
     }
+
+    #[test]
+    fn test_strings_only_occur_when_nested() {
+        let input = r#"
+            p.mt-2.text-xl
+              div The quote in the next word, can't be the start of a string
+
+            h3.mt-24.text-center.text-4xl.font-bold.italic
+              div The classes above should be extracted
+        "#;
+
+        Pug::test_extract_contains(
+            input,
+            vec![
+                // First paragraph
+                "mt-2",
+                "text-xl",
+                // second paragraph
+                "mt-24",
+                "text-center",
+                "text-4xl",
+                "font-bold",
+                "italic",
+            ],
+        );
+    }
+
+    #[test]
+    fn test_arbitrary_code_followed_by_classes() {
+        let input = r#"
+            - i < 3
+              .flex.items-center
+        "#;
+        Pug::test_extract_contains(input, vec!["flex", "items-center"]);
+    }
 }