Create a manual UI sanity check runbook

[albertteoh]

Requirement - what kind of business use case are you trying to solve?

• Relates to When to release 1.22 jaeger#2734 (comment)
• Ensure a bug-free UI release by documenting a consistent set of manual sanity checks on the Jaeger UI.
• Motivated by a lack of confidence in existing unit test coverage when applying dependabot version bumps.
• Doubles as a UI feature document for new users, particularly for features that are less apparent.
• Potential input in the future for automated UI tests with a framework like Selenium if the investment in time is worthwhile and deemed to not be overly brittle. Subject to further consideration and discussion.

Problem - what in Jaeger blocks you from solving the requirement?

No such document exists.

Proposal - what do you suggest to solve the problem or improve the existing situation?

Create a SANITY_TEST.md file consisting of BDD-style test cases to be executed prior to releasing a new version until we have enough confidence in unit test coverage.

Perhaps start with core features and expand to more scenarios where appropriate, for example:

- Scenario 1: Generating traces

**given** the user has started Jaeger all-in-one
**and** is on the Jaeger UI homepage (e.g. http://localhost:16686/)
**when** the user refreshes the page the page once
**then** the jaeger-query service should appear as an item in the Service drop-down

- Scenario 2: Finding traces

**given** Scenario 1
**when** the user selects the jaeger-query Service
**and** sets a Lookback of "1 hour"
**and** clicks the "Find Traces" button
**then** at least one trace should appear in the trace results summary

Any open questions to address

• Will this be too tedious to expect maintainers to go through?
• We will need to consider the maintenance cost of such a document for when the UI behaviour changes. Should we expect contributors to update this doc along with their PR + unit tests? Can the sanity checks be written to minimize changes required due to UI changes?
• Should we, instead, invest more effort into increasing unit test coverage? Do we have the resources to do so?

[jpkrohling]

Don't you think this would be a great Outreachy internship idea? The first phase is to document what to test, the second phase could be to automate the testing. In a previous life, I used quite a lot of Selenium with Saucelabs. Perhaps there's a version of it free for open source projects?

[albertteoh]

Don't you think this would be a great Outreachy internship idea?

Yup, I think this is a great idea. I'm new to the Outreachy internship program (I saw some mention of it in our Gitter chat). With some Googling, it looks like this would be a good place to start reading about it? https://www.outreachy.org/

I guess we'd need to submit this as a possible project to outreachy for approval then elect a candidate to work on it?

Perhaps there's a version of it free for open source projects?

While exploring, I managed to get a Selenium-based test using, funnily enough, SeleniumBase running with Github actions, so I think it should be available for open source projects.

[jpkrohling]

With some Googling, it looks like this would be a good place to start reading about it? https://www.outreachy.org/

That's correct! Here's a draft project proposal. Let me know if you'd like to be a mentor or co-mentor.

---

UI testing for Jaeger

Mentor: Juraci Paixão Kröhling (@jpkrohling)
Status: Pending

One of the key parts of Jaeger is its UI. Unfortunately, we have little to no automation when it comes to ensuring the UI is working properly before we perform a new release.

Your role in this project is to help us improve the situation by first determining what to test, writing BDD-style tests that can be executed manually by the maintainers before a release, and automating the tests using tools like Cucumber, Watir, or Selenium.

For this project, you’ll need to be proficient with Javascript, and you’ll leave this internship with a great understanding of Quality Assurance practices, especially when applied to testing user interfaces, as well as intermediary-level knowledge about distributed tracing and Jaeger.

[meenal06]

Hi @jpkrohling , I have been planning on the tests and have prepared a plan of approach for testing:

• I have been thinking to use cypress tool due to its performance, ease of maintainers and wide community support.
• I have seen used Jaeger myself and I think rather than unit tests we should focus on integration tests for each functionality and e2e tests from any tutorial/session through cypress.
• Since cypress code is human readable there wont be much need for comments except for complex execution parts. For maintainability purpose, since Jaeger is open-source we can opt for cypress open-source plan where we would have a dedicated dashboard as well as video records and various analysis about our tests on a dashboard provided by the cypress.
• We can then add cypress github action to our ci for asserting each PR for unit-tests for automating the run for tests

I will be sure to mention each step in detail with my proposal.

[jpkrohling]

I don't know anything about Cypress, would you mind joining our weekly meeting and do a quick presentation about it? A quick demo should be sufficient to start a discussion.

[meenal06]

Sure, I would be happy to! Where can I find the invite?

…

On Tue, 6 Apr, 2021, 17:13 Juraci Paixão Kröhling, ***@***.***> wrote: I don't know anything about Cypress, would you mind joining our weekly meeting and do a quick presentation about it? A quick demo should be sufficient to start a discussion. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#708 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AS2Y6XV7WOBZSXQAYH4AUX3THLXVPANCNFSM4YCGDT5A> .

[jpkrohling]

You can find more information here: https://www.jaegertracing.io/get-in-touch/

Which time would suit you better, this week's or next week's?

[meenal06]

You can find more information here: https://www.jaegertracing.io/get-in-touch/

Which time would suit you better, this week's or next week's?

How about next week? I can also setup a basic test for jaeger-ui to show as a demo for next week with cypress.

[jpkrohling]

Sure, sounds good to me! I just added it to the agenda.