Skip to content
This repository has been archived by the owner on Sep 11, 2024. It is now read-only.

DRAFT: Add e2e tests for the module system #11396

Closed
wants to merge 1 commit into from

Conversation

dhenneke
Copy link
Contributor

@dhenneke dhenneke commented Aug 10, 2023

This PR implements a proposal for testing the Module System with Cypress. It needs to be viewed together with element-hq/element-web#25951.

The changes:

  1. We create a module in the cypress/example_module (tbd) folder. It has a minimal setup with a package.json, but we don't expect that anyone calls yarn install we expect it to be used directly as a TypeScript file, so Webpack is advised to run babel for it (-> change in element-web).
  2. We add a ELEMENT_BUILD_CONFIG variable that can change the location of the build_config.yaml file that is used in the build process. A yarn start:e2e script is adding the example_module created in 1. (-> change in element-web)
  3. Cypress runs and we use flags in the localStorage to "enable" the module. (1) This takes case that the module doesn't interfere with any other tests and (2) it allows us to put the tests into the existing Cypress setup without the need for a different build of Element to test the modules. Individual parts of the module could also be enabled/disabled via localStorage flags in certain tests (this PR).

I added two example tests for the translations and the RoomViewLifecycle.PreviewRoomNotLoggedIn lifecycle to show how a test could look like.

@germain-gg we discussed this before, and I wanted to propose this so we have a base for our discussions. Do you think this setup would work for us? I would be happy to receive feedback, and other ideas on how to make the module system testable.

Checklist

  • Tests written for new code (and old code if feasible)
  • Linter and other CI checks pass
  • Sign-off given on the changes (see CONTRIBUTING.md)

Type: task
Notes: none


This change is marked as an internal change (Task), so will not be included in the changelog.

Signed-off-by: Dominik Henneke <dominik.henneke@nordeck.net>
@github-actions github-actions bot added Z-Community-PR Issue is solved by a community member's PR T-Task Refactoring, enabling or disabling functionality, other engineering tasks labels Aug 10, 2023
@Johennes Johennes added A-Timesheet-1 Log any time spent on this into the A-Timesheet-1 project and removed Z-Community-PR Issue is solved by a community member's PR labels Aug 16, 2023
Copy link
Member

@t3chguy t3chguy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is exciting, end to end testing and improving modules is very important. We think, though, that this approach isn't flexible enough and leads to testing only that Example modules work which isn't in the nature of end to end tests. It doesn't allow for testing actual modules like https://github.com/vector-im/element-web-ilag-module - do you think you could pivot and instead make it test that? Additionally having cypress always testing a module-enabled build means we're no longer testing "vanilla" element which is crucial to maintain, modules may have other implications (performance or otherwise) down the line even if they do not intend to. We need to be testing the primary vanilla experience first and foremost

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A-Timesheet-1 Log any time spent on this into the A-Timesheet-1 project T-Task Refactoring, enabling or disabling functionality, other engineering tasks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants