Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

e2e: Updated report #16223

Merged
merged 1 commit into from
Jun 8, 2023
Merged

e2e: Updated report #16223

merged 1 commit into from
Jun 8, 2023

Conversation

yevh-berdnyk
Copy link
Contributor

@yevh-berdnyk yevh-berdnyk commented Jun 8, 2023

... and couple of small fixes

@yevh-berdnyk yevh-berdnyk requested a review from churik as a code owner June 8, 2023 05:29
@yevh-berdnyk yevh-berdnyk self-assigned this Jun 8, 2023
@status-im-auto
Copy link
Member

status-im-auto commented Jun 8, 2023

Jenkins Builds

Click to see older builds (12)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 3704997 #1 2023-06-08 05:35:32 ~5 min android-e2e 🤖apk 📲
✔️ 3704997 #1 2023-06-08 05:36:05 ~6 min android 🤖apk 📲
✔️ 3704997 #1 2023-06-08 05:36:28 ~6 min ios 📱ipa 📲
✔️ 3704997 #1 2023-06-08 05:38:29 ~8 min tests 📄log
✔️ e86891f #2 2023-06-08 07:08:46 ~5 min android 🤖apk 📲
✔️ e86891f #2 2023-06-08 07:08:55 ~5 min android-e2e 🤖apk 📲
✔️ e86891f #2 2023-06-08 07:09:45 ~6 min ios 📱ipa 📲
✔️ e86891f #2 2023-06-08 07:11:22 ~7 min tests 📄log
✔️ da0a30b #3 2023-06-08 09:37:06 ~6 min android-e2e 🤖apk 📲
✔️ da0a30b #3 2023-06-08 09:39:39 ~9 min ios 📱ipa 📲
✔️ da0a30b #3 2023-06-08 09:45:11 ~14 min tests 📄log
✔️ da0a30b #3 2023-06-08 09:48:09 ~17 min android 🤖apk 📲
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 4c12e3f #5 2023-06-08 12:24:25 ~6 min ios 📱ipa 📲
✔️ 4c12e3f #5 2023-06-08 12:29:48 ~11 min android 🤖apk 📲
✔️ 4c12e3f #5 2023-06-08 12:30:51 ~12 min android-e2e 🤖apk 📲
✔️ 4c12e3f #5 2023-06-08 12:32:40 ~14 min tests 📄log
✔️ 032c254 #7 2023-06-08 12:53:36 ~5 min android 🤖apk 📲
✔️ 032c254 #7 2023-06-08 12:53:53 ~5 min android-e2e 🤖apk 📲
✔️ 032c254 #7 2023-06-08 12:55:17 ~7 min tests 📄log
✔️ 032c254 #7 2023-06-08 12:59:39 ~11 min ios 📱ipa 📲

@yevh-berdnyk yevh-berdnyk force-pushed the tests/report-update branch from 3704997 to e86891f Compare June 8, 2023 07:03
@status-im-auto
Copy link
Member

55% of end-end tests have passed

Total executed tests: 33
Failed tests: 15
Passed tests: 18
IDs of failed tests: 702732,702783,702841,702733,702855,702844,702731,703194,702838,702957,702958,703086,702859,702840,702894 

Failed tests (15)

Click to expand
  • Rerun failed tests

  • Class TestActivityMultipleDevicePR:

    1. test_activity_center_mentions, id: 702957
    Test setup failed: base_test_case.py:355: in setup_method
        driver.execute_script("sauce:context=Started %s" % method.__name__)
    /home/jenkins/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py:634: in execute_script
        return self.execute(command, {
    /home/jenkins/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py:321: in execute
        self.error_handler.check_response(response)
    /home/jenkins/.local/lib/python3.10/site-packages/appium/webdriver/errorhandler.py:31: in check_response
        raise wde
    /home/jenkins/.local/lib/python3.10/site-packages/appium/webdriver/errorhandler.py:26: in check_response
        super().check_response(response)
    /home/jenkins/.local/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py:208: in check_response
        raise exception_class(value)
     ERROR The test with session id 71e5105c458b42739f773978c43e5cb0 has already finished, and can't receive further commands.
    E   You can learn more at https://app.eu-central-1.saucelabs.com/tests/71e5105c458b42739f773978c43e5cb0
    E   For help, please check https://docs.saucelabs.com/dev/error-messages
    



    2. test_activity_center_admin_notification_accept_swipe, id: 702958

    Test setup failed: base_test_case.py:355: in setup_method
        driver.execute_script("sauce:context=Started %s" % method.__name__)
    /home/jenkins/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py:634: in execute_script
        return self.execute(command, {
    /home/jenkins/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py:321: in execute
        self.error_handler.check_response(response)
    /home/jenkins/.local/lib/python3.10/site-packages/appium/webdriver/errorhandler.py:31: in check_response
        raise wde
    /home/jenkins/.local/lib/python3.10/site-packages/appium/webdriver/errorhandler.py:26: in check_response
        super().check_response(response)
    /home/jenkins/.local/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py:208: in check_response
        raise exception_class(value)
     ERROR The test with session id 71e5105c458b42739f773978c43e5cb0 has already finished, and can't receive further commands.
    E   You can learn more at https://app.eu-central-1.saucelabs.com/tests/71e5105c458b42739f773978c43e5cb0
    E   For help, please check https://docs.saucelabs.com/dev/error-messages
    



    Class TestGroupChatMultipleDeviceMergedNewUI:

    1. test_group_chat_pin_messages, id: 702732

    Test is not run, e2e blocker  
    

    [[reason: [NOTRUN] Pin feature is in development]]

    Class TestOneToOneChatMultipleSharedDevicesNewUi:

    1. test_1_1_chat_is_shown_message_sent_delivered_from_offline, id: 702783

    # STEP: Device1 goes back online and checks that 1-1 chat will be fetched
    Device 1: Looking for a message by text: test message

    critical/chats/test_1_1_public_chats.py:1242: in test_1_1_chat_is_shown_message_sent_delivered_from_offline
        self.errors.verify_no_errors()
    base_test_case.py:182: in verify_no_errors
        pytest.fail('\n '.join([self.errors.pop(0) for _ in range(len(self.errors))]))
     Message status was not delivered after back up online, it is "Sending"! 
    

    [[Issue with messages not being sent for a long time]]

    Device sessions

    2. test_1_1_chat_text_message_delete_push_disappear, id: 702733

    Device 2: Find Text by xpath: //*[starts-with(@text,'message to delete for me')]/ancestor::android.view.ViewGroup[@content-desc='chat-item']
    Device 2: Tap on found: Text

    critical/chats/test_1_1_public_chats.py:1166: in test_1_1_chat_text_message_delete_push_disappear
        self.chat_2.chat_element_by_text(message_to_delete_for_me).wait_for_status_to_be("Delivered")
    ../views/chat_view.py:228: in wait_for_status_to_be
        raise TimeoutException("Message status was not changed to %s, it's %s" % (expected_status, current_status))
     Message status was not changed to Delivered, it's Sending
    



    Device sessions

    3. test_1_1_chat_edit_message, id: 702855

    Device 2: Find Text by xpath: //*[starts-with(@text,'Message before edit 1-1')]/ancestor::android.view.ViewGroup[@content-desc='chat-item']
    Device 2: Tap on found: Text

    critical/chats/test_1_1_public_chats.py:1140: in test_1_1_chat_edit_message
        self.chat_2.chat_element_by_text(message_before_edit_1_1).wait_for_status_to_be("Delivered")
    ../views/chat_view.py:228: in wait_for_status_to_be
        raise TimeoutException("Message status was not changed to %s, it's %s" % (expected_status, current_status))
     Message status was not changed to Delivered, it's Sending
    



    Device sessions

    4. test_1_1_chat_pin_messages, id: 702731

    Test is not run, e2e blocker  
    

    [[reason: [NOTRUN] Pin feature is in development]]

    Class TestCommunityMultipleDeviceMerged:

    1. test_community_unread_messages_badge, id: 702841

    Device 1: Find Button by accessibility id: jump-to

    critical/test_public_chat_browsing.py:668: in test_community_unread_messages_badge
        self.channel_1.jump_to_communities_home()
    ../views/base_view.py:605: in jump_to_communities_home
        self.jump_to_button.click()
    ../views/base_element.py:91: in click
        self.find_element().click()
    ../views/base_element.py:80: in find_element
        raise NoSuchElementException(
     Device 1: Button by accessibility id: `jump-to` is not found on the screen
    



    Device sessions

    2. test_community_links_with_previews_github_youtube_twitter_gif_send_enable, id: 702844

    Device 2: Tap on found: SendMessageButton
    Device 1: Getting preview message for link: #11707

    critical/test_public_chat_browsing.py:649: in test_community_links_with_previews_github_youtube_twitter_gif_send_enable
        self.channel_1.get_preview_message_by_text(url).wait_for_element(60)
    ../views/base_element.py:117: in wait_for_element
        raise TimeoutException(
     Device `1`: `PreviewMessage` by` xpath`: `//*[starts-with(@text,'https://github.com/status-im/status-mobile/pull/11707')]/ancestor::android.view.ViewGroup[@content-desc='chat-item']` is not found on the screen after wait_for_element; 
     RemoteDisconnected
    



    Device sessions

    3. test_community_several_images_send_reply, id: 703194

    # STEP: Send several images in 1-1 chat from Gallery
    Device 1: Find Button by accessibility id: open-images-button

    critical/test_public_chat_browsing.py:555: in test_community_several_images_send_reply
        self.channel_1.send_images_with_description(image_description, [0, 1])
    ../views/chat_view.py:1166: in send_images_with_description
        self.show_images_button.click()
    ../views/base_element.py:91: in click
        self.find_element().click()
    ../views/base_element.py:80: in find_element
        raise NoSuchElementException(
     Device 1: Button by accessibility id: `open-images-button` is not found on the screen
    



    Device sessions

    4. test_community_message_send_check_timestamps_sender_username, id: 702838

    Device 1: Find Text by xpath: //*[starts-with(@text,'hello')]/ancestor::android.view.ViewGroup[@content-desc='chat-item']
    Device 1: Tap on found: Text

    critical/test_public_chat_browsing.py:458: in test_community_message_send_check_timestamps_sender_username
        self.channel_1.chat_element_by_text(message).wait_for_status_to_be('Delivered', timeout=120)
    ../views/chat_view.py:228: in wait_for_status_to_be
        raise TimeoutException("Message status was not changed to %s, it's %s" % (expected_status, current_status))
     Message status was not changed to Delivered, it's Sending
    



    Device sessions

    5. test_community_mark_all_messages_as_read, id: 703086

    Device 1: Find Button by accessibility id: jump-to

    critical/test_public_chat_browsing.py:764: in test_community_mark_all_messages_as_read
        self.channel_1.jump_to_communities_home()
    ../views/base_view.py:605: in jump_to_communities_home
        self.jump_to_button.click()
    ../views/base_element.py:91: in click
        self.find_element().click()
    ../views/base_element.py:80: in find_element
        raise NoSuchElementException(
     Device 1: Button by accessibility id: `jump-to` is not found on the screen
    



    Device sessions

    6. test_community_one_image_send_reply, id: 702859

    Device 1: Tap on found: Text
    Device 1: Find Text by xpath: //*[starts-with(@text,'description')]/ancestor::android.view.ViewGroup[@content-desc='chat-item']

    critical/test_public_chat_browsing.py:505: in test_community_one_image_send_reply
        self.channel_1.chat_element_by_text(image_description).wait_for_status_to_be('Delivered', timeout=120)
    ../views/chat_view.py:223: in wait_for_status_to_be
        current_status = self.status
    ../views/chat_view.py:213: in status
        Text(self.driver, xpath=self.locator).click()
    ../views/base_element.py:91: in click
        self.find_element().click()
    ../views/base_element.py:80: in find_element
        raise NoSuchElementException(
     Device 1: Text by xpath: `//*[starts-with(@text,'description')]/ancestor::android.view.ViewGroup[@content-desc='chat-item']` is not found on the screen
    



    Device sessions

    7. test_community_emoji_send_copy_paste_reply, id: 702840

    Device 1: Sending message 'emoji'

    critical/test_public_chat_browsing.py:579: in test_community_emoji_send_copy_paste_reply
        self.channel_1.send_message(emoji_message)
    ../views/chat_view.py:958: in send_message
        self.chat_message_input.wait_for_element(wait_chat_input_sec)
    ../views/base_element.py:117: in wait_for_element
        raise TimeoutException(
     Device `1`: `ChatMessageInput` by` accessibility id`: `chat-message-input` is not found on the screen after wait_for_element
    



    Device sessions

    8. test_community_contact_block_unblock_offline, id: 702894

    Device 1: Find Button by accessibility id: jump-to

    critical/test_public_chat_browsing.py:687: in test_community_contact_block_unblock_offline
        [home.jump_to_card_by_text('# %s' % self.channel_name) for home in self.homes]
    critical/test_public_chat_browsing.py:687: in <listcomp>
        [home.jump_to_card_by_text('# %s' % self.channel_name) for home in self.homes]
    ../views/base_view.py:609: in jump_to_card_by_text
        self.jump_to_button.click()
    ../views/base_element.py:91: in click
        self.find_element().click()
    ../views/base_element.py:80: in find_element
        raise NoSuchElementException(
     Device 1: Button by accessibility id: `jump-to` is not found on the screen
    



    Device sessions

    Passed tests (18)

    Click to expand

    Class TestCommunityOneDeviceMerged:

    1. test_community_copy_and_paste_message_in_chat_input, id: 702742
    Device sessions

    2. test_community_navigate_to_channel_when_relaunch, id: 702846
    Device sessions

    3. test_restore_multiaccount_with_waku_backup_remove_switch, id: 703133
    Device sessions

    Class TestGroupChatMultipleDeviceMergedNewUI:

    1. test_group_chat_join_send_text_messages_push, id: 702807
    Device sessions

    2. test_group_chat_offline_pn, id: 702808
    Device sessions

    Class TestCommunityMultipleDeviceMerged:

    1. test_community_message_edit, id: 702843
    Device sessions

    2. test_community_message_delete, id: 702839
    Device sessions

    3. test_community_mentions_push_notification, id: 702786
    Device sessions

    4. test_community_leave, id: 702845
    Device sessions

    Class TestOneToOneChatMultipleSharedDevicesNewUi:

    1. test_1_1_chat_non_latin_messages_stack_update_profile_photo, id: 702745
    Device sessions

    2. test_1_1_chat_message_reaction, id: 702730
    Device sessions

    3. test_1_1_chat_delete_via_long_press_relogin, id: 702784
    Device sessions

    4. test_1_1_chat_push_emoji, id: 702813
    Device sessions

    5. test_1_1_chat_emoji_send_reply_and_open_link, id: 702782
    Device sessions

    Class TestActivityMultipleDevicePR:

    1. test_navigation_jump_to, id: 702936
    Device sessions

    2. test_activity_center_reply_read_unread_delete_filter_swipe, id: 702947
    Device sessions

    Class TestActivityCenterContactRequestMultipleDevicePR:

    1. test_activity_center_contact_request_accept_swipe_mark_all_as_read, id: 702851
    Device sessions

    2. test_activity_center_contact_request_decline, id: 702850
    Device sessions

    @yevh-berdnyk yevh-berdnyk force-pushed the tests/report-update branch from e86891f to da0a30b Compare June 8, 2023 09:29
    @status-im-auto
    Copy link
    Member

    70% of end-end tests have passed

    Total executed tests: 33
    Failed tests: 10
    Passed tests: 23
    
    IDs of failed tests: 703194,702732,702783,702894,702733,702859,702786,702731,702838,702958 
    

    Failed tests (10)

    Click to expand
  • Rerun failed tests

  • Class TestGroupChatMultipleDeviceMergedNewUI:

    1. test_group_chat_pin_messages, id: 702732
    Test is not run, e2e blocker  
    

    [[reason: [NOTRUN] Pin feature is in development]]

    Class TestCommunityMultipleDeviceMerged:

    1. test_community_several_images_send_reply, id: 703194

    Device 1: Find RepliedMessageText by xpath: //*[starts-with(@text,'reply to gallery')]/preceding::android.widget.TextView[@content-desc='quoted-message']
    Device 1: RepliedMessageText is gallery

    critical/test_public_chat_browsing.py:571: in test_community_several_images_send_reply
        self.errors.verify_no_errors()
    base_test_case.py:182: in verify_no_errors
        pytest.fail('\n '.join([self.errors.pop(0) for _ in range(len(self.errors))]))
     Reply message was not received by the sender
    



    Device sessions

    2. test_community_contact_block_unblock_offline, id: 702894

    Device 2: Text is Delivered
    Device 1: Looking for a message by text: Hurray! unblocked

    critical/test_public_chat_browsing.py:759: in test_community_contact_block_unblock_offline
        self.errors.verify_no_errors()
    base_test_case.py:182: in verify_no_errors
        pytest.fail('\n '.join([self.errors.pop(0) for _ in range(len(self.errors))]))
     Messages from blocked user is not cleared in public chat
    



    Device sessions

    3. test_community_one_image_send_reply, id: 702859

    Device 1: Find RepliedMessageText by xpath: //*[starts-with(@text,'reply to image')]/preceding::android.widget.TextView[@content-desc='quoted-message']
    Device 1: RepliedMessageText is description

    critical/test_public_chat_browsing.py:548: in test_community_one_image_send_reply
        self.errors.verify_no_errors()
    base_test_case.py:182: in verify_no_errors
        pytest.fail('\n '.join([self.errors.pop(0) for _ in range(len(self.errors))]))
     Reply message was not received by the sender
    



    Device sessions

    4. test_community_mentions_push_notification, id: 702786

    # STEP: Admin gets push notification with the mention and tap it
    Device 1: Getting PN by 'user_1'

    critical/test_public_chat_browsing.py:850: in test_community_mentions_push_notification
        self.errors.verify_no_errors()
    base_test_case.py:182: in verify_no_errors
        pytest.fail('\n '.join([self.errors.pop(0) for _ in range(len(self.errors))]))
     Push notification with the mention was not received by admin
    



    Device sessions

    5. test_community_message_send_check_timestamps_sender_username, id: 702838

    Device 2: Verifying that 'new message' is under today
    Device 2: Looking for a message by text: new message

    critical/test_public_chat_browsing.py:458: in test_community_message_send_check_timestamps_sender_username
        self.channel_2.verify_message_is_under_today_text(new_message, self.errors, 60)
    ../views/chat_view.py:948: in verify_message_is_under_today_text
        message_element.wait_for_visibility_of_element(timeout)
    ../views/base_element.py:135: in wait_for_visibility_of_element
        raise TimeoutException(
     Device 2: ChatElementByText by xpath:`//*[starts-with(@text,'new message')]/ancestor::android.view.ViewGroup[@content-desc='chat-item']` is not found on the screen after wait_for_visibility_of_element
    



    Device sessions

    Class TestOneToOneChatMultipleSharedDevicesNewUi:

    1. test_1_1_chat_is_shown_message_sent_delivered_from_offline, id: 702783

    # STEP: Device1 goes back online and checks that 1-1 chat will be fetched
    Device 1: Looking for a message by text: test message

    critical/chats/test_1_1_public_chats.py:1250: in test_1_1_chat_is_shown_message_sent_delivered_from_offline
        self.errors.verify_no_errors()
    base_test_case.py:182: in verify_no_errors
        pytest.fail('\n '.join([self.errors.pop(0) for _ in range(len(self.errors))]))
     Message status was not delivered after back up online, it is "Sending"! 
    

    [[Issue with messages not being sent for a long time]]

    Device sessions

    2. test_1_1_chat_text_message_delete_push_disappear, id: 702733

    Device 2: Tap on found: Button
    Device 1: Getting PN by 'DELETE ME'

    critical/chats/test_1_1_public_chats.py:1207: in test_1_1_chat_text_message_delete_push_disappear
        self.errors.verify_no_errors()
    base_test_case.py:182: in verify_no_errors
        pytest.fail('\n '.join([self.errors.pop(0) for _ in range(len(self.errors))]))
     Message status was not changed to 'Delivered' after 60 s, https://github.com/status-im/status-mobile/issues/15385
    



    Device sessions

    3. test_1_1_chat_pin_messages, id: 702731

    Test is not run, e2e blocker  
    

    [[reason: [NOTRUN] Pin feature is in development]]

    Class TestActivityMultipleDevicePR:

    1. test_activity_center_admin_notification_accept_swipe, id: 702958

    Device 2: Click until ChatMessageInput by accessibility id: chat-message-input will be presented
    Device 2: Find Button by xpath: //*[contains(@text, "View")]

    medium/test_activity_center.py:305: in test_activity_center_admin_notification_accept_swipe
        self.chat_2.element_by_text_part('View').click()
    ../views/base_element.py:91: in click
        self.find_element().click()
    ../views/base_element.py:80: in find_element
        raise NoSuchElementException(
     Device 2: Button by xpath: `//*[contains(@text, "View")]` is not found on the screen
    



    Device sessions

    Passed tests (23)

    Click to expand

    Class TestGroupChatMultipleDeviceMergedNewUI:

    1. test_group_chat_join_send_text_messages_push, id: 702807
    Device sessions

    2. test_group_chat_offline_pn, id: 702808
    Device sessions

    Class TestCommunityMultipleDeviceMerged:

    1. test_community_message_delete, id: 702839
    Device sessions

    2. test_community_leave, id: 702845
    Device sessions

    3. test_community_message_edit, id: 702843
    Device sessions

    4. test_community_mark_all_messages_as_read, id: 703086
    Device sessions

    5. test_community_links_with_previews_github_youtube_twitter_gif_send_enable, id: 702844
    Device sessions

    6. test_community_unread_messages_badge, id: 702841
    Device sessions

    7. test_community_emoji_send_copy_paste_reply, id: 702840
    Device sessions

    Class TestCommunityOneDeviceMerged:

    1. test_restore_multiaccount_with_waku_backup_remove_switch, id: 703133
    Device sessions

    2. test_community_navigate_to_channel_when_relaunch, id: 702846
    Device sessions

    3. test_community_copy_and_paste_message_in_chat_input, id: 702742
    Device sessions

    Class TestOneToOneChatMultipleSharedDevicesNewUi:

    1. test_1_1_chat_push_emoji, id: 702813
    Device sessions

    2. test_1_1_chat_message_reaction, id: 702730
    Device sessions

    3. test_1_1_chat_edit_message, id: 702855
    Device sessions

    4. test_1_1_chat_emoji_send_reply_and_open_link, id: 702782
    Device sessions

    5. test_1_1_chat_delete_via_long_press_relogin, id: 702784
    Device sessions

    6. test_1_1_chat_non_latin_messages_stack_update_profile_photo, id: 702745
    Device sessions

    Class TestActivityCenterContactRequestMultipleDevicePR:

    1. test_activity_center_contact_request_accept_swipe_mark_all_as_read, id: 702851
    Device sessions

    2. test_activity_center_contact_request_decline, id: 702850
    Device sessions

    Class TestActivityMultipleDevicePR:

    1. test_activity_center_reply_read_unread_delete_filter_swipe, id: 702947
    Device sessions

    2. test_navigation_jump_to, id: 702936
    Device sessions

    3. test_activity_center_mentions, id: 702957
    Device sessions

    @yevh-berdnyk yevh-berdnyk force-pushed the tests/report-update branch 3 times, most recently from 4c12e3f to 7325bf1 Compare June 8, 2023 12:47
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    Projects
    No open projects
    Archived in project
    Development

    Successfully merging this pull request may close these issues.

    3 participants