Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test_TC_LVL_6_1.yaml has disabled test step #19869

Closed
woody-apple opened this issue Jun 23, 2022 · 2 comments · Fixed by #20030
Closed

Test_TC_LVL_6_1.yaml has disabled test step #19869

woody-apple opened this issue Jun 23, 2022 · 2 comments · Fixed by #20030
Assignees
@raju-apple
Labels
update tests V1.0

Comments

@woody-apple
Copy link
Contributor

woody-apple commented Jun 23, 2022 •
edited
Loading

Problem

This YAML step is failing per example here: https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true

2022-06-23 03:24:50.401 ERROR   03:24:47.707 - TEST OUT  : [1655954687707] [26076:104009] CHIP: [EM] Received message of type 0x5 with protocolId (0, 1) and MessageCounter:171556752 on exchange 32088i
[2662](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2663)
2022-06-23 03:24:50.401 ERROR   03:24:47.707 - TEST OUT  : [1655954687707] [26076:104009] CHIP: [EM] Found matching exchange: 32088i, Delegate: 0x7b3c000177f0
[2663](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2664)
2022-06-23 03:24:50.401 ERROR   03:24:47.707 - TEST OUT  : [1655954687707] [26076:104009] CHIP: [EM] Rxd Ack; Removing MessageCounter:213441215 from Retrans Table on exchange 32088i
[2664](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2665)
2022-06-23 03:24:50.401 ERROR   03:24:47.707 - TEST OUT  : [1655954687707] [26076:104009] CHIP: [EM] Removed CHIP MessageCounter:213441215 from RetransTable on exchange 32088i
[2665](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2666)
2022-06-23 03:24:50.401 ERROR   03:24:47.707 - TEST OUT  : [1655954687707] [26076:104009] CHIP: [DMG] ReportDataMessage =
[2666](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2667)
2022-06-23 03:24:50.401 ERROR   03:24:47.707 - TEST OUT  : [1655954687707] [26076:104009] CHIP: [DMG] {
[2667](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2668)
2022-06-23 03:24:50.401 ERROR   03:24:47.707 - TEST OUT  : [1655954687707] [26076:104009] CHIP: [DMG] 	AttributeReportIBs =
[2668](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2669)
2022-06-23 03:24:50.401 ERROR   03:24:47.707 - TEST OUT  : [1655954687707] [26076:104009] CHIP: [DMG] 	[
[2669](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2670)
2022-06-23 03:24:50.401 ERROR   03:24:47.707 - TEST OUT  : [1655954687707] [26076:104009] CHIP: [DMG] 		AttributeReportIB =
[2670](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2671)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687707] [26076:104009] CHIP: [DMG] 		{
[2671](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2672)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687707] [26076:104009] CHIP: [DMG] 			AttributeDataIB =
[2672](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2673)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687707] [26076:104009] CHIP: [DMG] 			{
[2673](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2674)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687707] [26076:104009] CHIP: [DMG] 				DataVersion = 0x61a4d190,
[2674](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2675)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 				AttributePathIB =
[2675](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2676)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 				{
[2676](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2677)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 					Endpoint = 0x1,
[2677](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2678)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 					Cluster = 0x8,
[2678](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2679)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 					Attribute = 0x0000_0000,
[2679](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2680)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 				}
[2680](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2681)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 					
[2681](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2682)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 				Data = 26, 
[2682](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2683)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 			},
[2683](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2684)
2022-06-23 03:24:50.402 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 			
[2684](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2685)
2022-06-23 03:24:50.403 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 		},
[2685](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2686)
2022-06-23 03:24:50.403 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 		
[2686](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2687)
2022-06-23 03:24:50.403 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 	],
[2687](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2688)
2022-06-23 03:24:50.403 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 	
[2688](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2689)
2022-06-23 03:24:50.403 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 	SuppressResponse = true, 
[2689](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2690)
2022-06-23 03:24:50.403 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] 	InteractionModelRevision = 1
[2690](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2691)
2022-06-23 03:24:50.403 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [DMG] }
[2691](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2692)
2022-06-23 03:24:50.403 ERROR   03:24:47.708 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [TOO]  ***** Test Failure: currentLevel value mismatch: expected 25 but got 26
[2692](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2693)
2022-06-23 03:24:50.403 ERROR   03:24:47.708 - TEST OUT  : 
[2693](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2694)
2022-06-23 03:24:50.403 ERROR   03:24:47.709 - TEST OUT  : [1655954687708] [26076:104009] CHIP: [EM] Sending Standalone Ack for MessageCounter:171556752 on exchange 32088i
[2694](https://github.com/project-chip/connectedhomeip/runs/7015795479?check_suite_focus=true#step:11:2695)

The YAML is:

    - label: "Reads CurrentLevel attribute from DUT"
      PICS: PICS_SKIP_SAMPLE_APP
      command: "readAttribute"
      attribute: "current level"
      response:
          value: 50
          constraints:
              notValue: CurrentLevelValue

Proposed Solution

Fix
woody-apple added a commit that referenced this issue Jun 23, 2022
@woody-apple
Disabling test
87cfe3f 
Filed #19869
@kvikrambhat
Copy link
Contributor

@raju-apple @woody-apple The test steps that is in question here is marked by the PICS "PICS_SKIP_SAMPLE_APP" , so this steps should not even run on the CI as this PIC was meant to skip steps that are currently not supported by the sample apps.

@manjunath-grl
Copy link
Contributor

We are observing inconsistent values for the CurrentLevel attribute after the wait time, instead of checking for specific value we have updated the script to check for a range with 5% tolerance

@andy31415 andy31415 mentioned this issue Jun 28, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@raju-apple raju-apple

Labels
update tests V1.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use 5% tolerance when checking CurrentLevel attribute in TC-LVL-6.1 manjunath-grl/connectedhomeip
4 participants
@kvikrambhat @woody-apple @raju-apple @manjunath-grl