Skip to content

Commit

Permalink
ci: regression tests for double suspense/double resource fetch (#3103)
Browse files Browse the repository at this point in the history 
* test: first cut of the instrumented suspense_tests

- Based on initial concepts developed for reproducing #2937 and others,
  streamlined and instrumented for e2e testing and refined for inclusion
  as a standalone module to be plugged into some other App.

* First cut of the fixtures and tests

* Actually make it work properly

- Instead of using examples, just feed it the table because examples
  will rerun the whole scenario from scratch, which isn't what we are
  trying to test here.
- Provide a basic example with item listing to show how this will work.

* Use ticketing system to disambiguate CSR calls

- Keep all SSR calls on ticket 0 as a means to disambiguate them from
  CSR calls.  For the mean time the focus of tests isn't on that
  behavior but this may be modified when suitable.

* Update the baseline fixtures

- Given the new understanding, the scenerios all being the baseline
  tests they are now moved into one file.
- Have the checks against all calls at once for better diff output,
  and reword the new scenerios into more idomatic gherkin.
- Streamline the steps and provide additional ones that will help with
  feature definitions.

* Translate the reproduction steps into Gherkin

* Comment out logging to avoid output interference.

* Be able to reset CSR counters everywhere

- Done by providing a button directly on the top level component with
  the navigation footer.  This will be useful for the next test.

* Test showing difference between hydrate and CSR

- Specifically, under hydrated load, resources that shouldn't need
  refetching gets refetched, while CSR does not show this issue.
  • Loading branch information
@metatoaster
metatoaster authored Oct 25, 2024
1 parent d306a15 commit 0e73d18
Show file tree
Hide file tree
Showing 11 changed files with 1,129 additions and 14 deletions.
6 changes: 5 additions & 1 deletion examples/suspense_tests/Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ crate-type = ["cdylib", "rlib"]
actix-files = { version = "0.6.6", optional = true }
actix-web = { version = "4.8", optional = true, features = ["macros"] }
console_error_panic_hook = "0.1.7"
js-sys = { version = "0.3.70", optional = true }
leptos = { path = "../../leptos" }
leptos_actix = { path = "../../integrations/actix", optional = true }
leptos_router = { path = "../../router" }
Expand All @@ -19,7 +20,10 @@ serde = "1.0"
tokio = { version = "1.39", features = ["time", "rt"], optional = true }

[features]
hydrate = ["leptos/hydrate"]
hydrate = [
"dep:js-sys",
"leptos/hydrate",
]
ssr = [
"dep:actix-files",
"dep:actix-web",
Expand Down
94 changes: 94 additions & 0 deletions examples/suspense_tests/e2e/features/check_instrumented.feature
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
@check_instrumented
Feature: Instrumented Counters showing the expected values

Scenario: I can fresh CSR instrumented counters
Given I see the app
When I access the instrumented counters via CSR
Then I see the following counters under section
| Suspend Calls | |
| item_listing | 0 |
| item_overview | 0 |
| item_inspect | 0 |
And the following counters under section
| Server Calls (CSR) | |
| list_items | 0 |
| get_item | 0 |
| inspect_item_root | 0 |
| inspect_item_field | 0 |

Scenario: I should see counter going up after viewing Item Listing
Given I see the app
When I select the following links
| Instrumented |
| Item Listing |
| Counters |
Then I see the following counters under section
| Suspend Calls | |
| item_listing | 1 |
| item_overview | 0 |
| item_inspect | 0 |
And the following counters under section
| Server Calls (CSR) | |
| list_items | 1 |
| get_item | 0 |
| inspect_item_root | 0 |
| inspect_item_field | 0 |

# the reload has happened in Item Listing, it follows a suspend
# will be called as hydration happens.
Scenario: Refreshing Item Listing should have only suspend counters
Given I see the app
When I access the instrumented counters via SSR
And I select the component Item Listing
And I reload the page
And I select the component Counters
Then I see the following counters under section
| Suspend Calls | |
| item_listing | 1 |
| item_overview | 0 |
| item_inspect | 0 |
And the following counters under section
| Server Calls (CSR) | |
| list_items | 0 |
| get_item | 0 |
| inspect_item_root | 0 |
| inspect_item_field | 0 |

Scenario: Reset CSR Counters work as expected.
Given I see the app
When I access the instrumented counters via SSR
And I select the component Item Listing
And I click on Reset CSR Counters
And I select the component Counters
Then I see the following counters under section
| Suspend Calls | |
| item_listing | 0 |
| item_overview | 0 |
| item_inspect | 0 |
And the following counters under section
| Server Calls (CSR) | |
| list_items | 0 |
| get_item | 0 |
| inspect_item_root | 0 |
| inspect_item_field | 0 |

Scenario: Standard usage of the instruments traversing down
Given I see the app
When I select the following links
| Instrumented |
| Item Listing |
| Item 2 |
| Inspect path3 |
| Inspect path3/field1 |
And I access the instrumented counters via CSR
Then I see the following counters under section
| Suspend Calls | |
| item_listing | 1 |
| item_overview | 1 |
| item_inspect | 2 |
And the following counters under section
| Server Calls (CSR) | |
| list_items | 1 |
| get_item | 1 |
| inspect_item_root | 1 |
| inspect_item_field | 1 |
195 changes: 195 additions & 0 deletions examples/suspense_tests/e2e/features/check_instrumented_suspense_resource.feature
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
@check_instrumented_suspense_resource
Feature: Using instrumented counters for real
Check that the suspend/suspense and the underlying resources are
called with the expected number of times for CSR rendering.

Background:

Given I see the app
And I select the mode Instrumented

Scenario: Emulate steps 1 to 5 of issue #2961
Given I select the link Target 3##
And I refresh the page
When I select the following links
| Item Listing |
| Target 4## |
And I go check the Counters
Then I see the following counters under section
| Suspend Calls | |
| item_listing | 1 |
| item_overview | 2 |
| item_inspect | 0 |
And the following counters under section
| Server Calls (CSR) | |
| list_items | 0 |
| get_item | 2 |
| inspect_item_root | 0 |
| inspect_item_field | 0 |

Scenario: Emulate step 6 of issue #2961
Given I select the link Target 41#
And I refresh the page
When I select the following links
| Target 4## |
| Target 42# |
And I go check the Counters
Then I see the following counters under section
| Suspend Calls | |
| item_listing | 0 |
| item_overview | 1 |
| item_inspect | 2 |
And the following counters under section
| Server Calls (CSR) | |
| list_items | 0 |
| get_item | 1 |
| inspect_item_root | 2 |
| inspect_item_field | 0 |

Scenario: Emulate step 7 of issue #2961
Given I select the link Target 42#
And I refresh the page
When I select the following links
| Target 4## |
| Target 42# |
| Target 41# |
And I go check the Counters
Then I see the following counters under section
| Suspend Calls | |
| item_listing | 0 |
| item_overview | 1 |
| item_inspect | 3 |
And the following counters under section
| Server Calls (CSR) | |
| list_items | 0 |
| get_item | 1 |
| inspect_item_root | 3 |
| inspect_item_field | 0 |

Scenario: Emulate step 8, "not trigger double fetch".
Given I select the link Target 3##
And I refresh the page
When I select the following links
| Item Listing |
| Target 4## |
| Target 41# |
And I go check the Counters
Then I see the following counters under section
| Suspend Calls | |
| item_listing | 1 |
| item_overview | 2 |
| item_inspect | 1 |
And the following counters under section
| Server Calls (CSR) | |
| list_items | 0 |
| get_item | 2 |
| inspect_item_root | 1 |
| inspect_item_field | 0 |

Scenario: Like above, for the "double fetch" which shouldn't happen
Given I select the link Target 3##
And I refresh the page
When I select the following links
| Item Listing |
| Target 4## |
| Target 41# |
| Target 3## |
And I go check the Counters
Then I see the following counters under section
| Suspend Calls | |
| item_listing | 1 |
| item_overview | 3 |
| item_inspect | 1 |
And the following counters under section
| Server Calls (CSR) | |
| list_items | 0 |
| get_item | 3 |
| inspect_item_root | 1 |
| inspect_item_field | 0 |

Scenario: Like above, but using 4## instead
Given I select the link Target 3##
And I refresh the page
When I select the following links
| Item Listing |
| Target 4## |
| Target 41# |
| Target 4## |
And I go check the Counters
Then I see the following counters under section
| Suspend Calls | |
| item_listing | 1 |
| item_overview | 3 |
| item_inspect | 1 |
And the following counters under section
| Server Calls (CSR) | |
| list_items | 0 |
| get_item | 2 |
| inspect_item_root | 1 |
| inspect_item_field | 0 |

# Currently, get_item is invoked with `3` as the argument upon
# selection of `Item Listing` despite that `Item Listing` doesn't
# need `get_item` calls. Seems like it may be due to the system
# still reacting to the unmounting of the component that needed
# view that generated the original `Item 3` (hydrated from SSR).
# Tests above may also have this type of behavior, but is somewhat
# masked because the direction of going down and then back up, but
# if this behavior changes for the better (avoiding this spurious
# resource fetch) then the above tests may need updating to reflect
# the corrected behavior. Note the difference with the fully CSR
# scenario after this one
Scenario: Emulate part of step 8 of issue #2961
Given I select the link Target 3##
And I refresh the page
When I select the link Item Listing
And I go check the Counters
Then I see the following counters under section
| Server Calls (CSR) | |
| list_items | 0 |
| get_item | 1 |
| inspect_item_root | 0 |
| inspect_item_field | 0 |

# Instead of refreshing the page like above, CSR counters is reset
# instead to keep the starting counter conditions identical.
Scenario: Emulate above, instead of refresh page, reset csr counters
Given I select the link Target 3##
And I click on Reset CSR Counters
When I select the link Item Listing
And I go check the Counters
Then I see the following counters under section
| Server Calls (CSR) | |
| list_items | 0 |
| get_item | 0 |
| inspect_item_root | 0 |
| inspect_item_field | 0 |

# Again, the following two sets demostrates resources making stale
# and redundant requests when hydrated, and not do so when under
# CSR.
Scenario: Start with hydration from Target 41# and go up
Given I select the link Target 41#
And I refresh the page
When I select the link Target 4##
And I select the link Item Listing
And I go check the Counters
Then I see the following counters under section
| Server Calls (CSR) | |
| list_items | 0 |
| get_item | 1 |
| inspect_item_root | 1 |
| inspect_item_field | 0 |

Scenario: Start with hydration from Target 41# and go up
Given I select the link Target 41#
And I click on Reset CSR Counters
When I select the link Target 4##
And I select the link Item Listing
And I go check the Counters
Then I see the following counters under section
| Server Calls (CSR) | |
| list_items | 0 |
| get_item | 0 |
| inspect_item_root | 0 |
| inspect_item_field | 0 |
16 changes: 16 additions & 0 deletions examples/suspense_tests/e2e/tests/fixtures/action.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,19 @@ pub async fn click_second_button(client: &Client) -> Result<()> {

Ok(())
}

pub async fn click_reset_counters_button(client: &Client) -> Result<()> {
let reset_counter = find::reset_counter(client).await?;

reset_counter.click().await?;

Ok(())
}

pub async fn click_reset_csr_counters_button(client: &Client) -> Result<()> {
let reset_counter = find::reset_csr_counter(client).await?;

reset_counter.click().await?;

Ok(())
}
18 changes: 18 additions & 0 deletions examples/suspense_tests/e2e/tests/fixtures/check.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,21 @@ pub async fn second_count_is(client: &Client, expected: u32) -> Result<()> {

Ok(())
}

pub async fn instrumented_counts(
client: &Client,
expected: &[(&str, u32)],
) -> Result<()> {
let mut actual = Vec::<(&str, u32)>::new();

for (selector, _) in expected.iter() {
actual.push((
selector,
find::instrumented_count(client, selector).await?,
))
}

assert_eq!(actual, expected);

Ok(())
}
37 changes: 37 additions & 0 deletions examples/suspense_tests/e2e/tests/fixtures/find.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,43 @@ pub async fn second_button(client: &Client) -> Result<Element> {
Ok(counter_button)
}

pub async fn instrumented_count(
client: &Client,
selector: &str,
) -> Result<u32> {
let element = client
.wait()
.for_element(Locator::Id(selector))
.await
.expect(format!("Element #{selector} not found.")
.as_str());
let text = element.text().await?;
let count = text.parse::<u32>()
.expect(format!("Element #{selector} does not contain a number.")
.as_str());
Ok(count)
}

pub async fn reset_counter(client: &Client) -> Result<Element> {
let reset_button = client
.wait()
.for_element(Locator::Id("reset-counters"))
.await
.expect("Reset counter input not found");

Ok(reset_button)
}

pub async fn reset_csr_counter(client: &Client) -> Result<Element> {
let reset_button = client
.wait()
.for_element(Locator::Id("reset-csr-counters"))
.await
.expect("Reset CSR counter input not found");

Ok(reset_button)
}

async fn component_message(client: &Client, id: &str) -> Result<String> {
let element =
client.wait().for_element(Locator::Id(id)).await.expect(
Expand Down
Loading

0 comments on commit 0e73d18

Please sign in to comment.