Address various DEM testing issues #35067
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Review changes with SemanticDiff. Analyzed 10 of 16 files. Overall, the semantic diff is 37% smaller than the GitHub diff.
|
|
PR #35067: Size comparison from b823a29 to a71bb17 Full report (10 builds for nrfconnect, nxp, qpg, stm32, tizen)
|
… as CI tests can run slower
|
PR #35067: Size comparison from b823a29 to 0c66ca8 Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
…sary and renumber test cases
|
PR #35067: Size comparison from b823a29 to 5bdddd1 Full report (22 builds for bl602, bl702, bl702l, nrfconnect, nxp, qpg, stm32, tizen)
|
|
PR #35067: Size comparison from b823a29 to 1f89e7c Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
|
PR #35067: Size comparison from b823a29 to 8f63c49 Increases above 0.2%:
Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
…eature_map() call
|
PR #35067: Size comparison from 1d1319b to 65af1b3 Full report (41 builds for bl602, bl702, bl702l, cyw30739, linux, nrfconnect, nxp, psoc6, qpg, stm32, tizen)
|
|
PR #35067: Size comparison from 1d1319b to 17edc0c Increases above 0.2%:
Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
|
I've reviewed Test cases 2.2 - 2.9 and ensured that they align with the test plan PR (https://github.com/CHIP-Specifications/chip-test-plans/pull/4543) fixing up minor issues. I think they all look good now - Thanks P! I just need to look at 2.10 (Q Quality tests) to tweak this test case so it covers both Forecast and PowerAdjustment attributes. |
… tidying up and fails due to powerAdjustmentCapability not being updated as expected.
|
PR #35067: Size comparison from 18a1863 to 58e6bf3 Full report (7 builds for nxp, qpg, stm32, tizen)
|
…buteChangeCallback when reason was changing.
|
PR #35067: Size comparison from 18a1863 to 81175d3 Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
|
PR #35067: Size comparison from 18a1863 to 50085be Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
doru91
approved these changes
Aug 29, 2024
|
PR #35067: Size comparison from 18a1863 to ce885c7 Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
jamesharrow
approved these changes
Aug 29, 2024
hasty
approved these changes
Aug 29, 2024
|
PR #35067: Size comparison from cfdbba9 to cd820e7 Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
bzbarsky-apple
approved these changes
Aug 29, 2024
shgutte
pushed a commit
to shgutte/connectedhomeip
that referenced
this pull request
Sep 10, 2024
* Add checks for correct DEM feature map for test * Add checks for a nominalPower having a value and get slotIndex check correct * Allow a little tolerance checking the duration returned in the event as CI tests can run slower * Restyled by clang-format * Restyled by autopep8 * Add check DEM feature map step + add event registration step if necessary and renumber test cases * Fix issue project-chip#4415 Fails randomly on CI when checking pause duration * Test an illegal slot index == number of slots in a forecast * Add a test in PFR when ModifyForecastRequest does not specify NominalPower * Set ClusterRevision to 4 * First versionof DEM 1-1 * Regenerate files * Get DEM 1.1 test passing * Update test step definitions * Get featureSet option correct * Restyled by autopep8 * First cut of a Q quality test * Regenerate files following a re-bootstrap * Fix typo in comment * Cancel subscriptions at the end of the test * Add a cancel subscription method * Ensure MatterReportingAttributeChangeCallback is called after forecast is changed * Restyled by autopep8 * Restyled by isort * Fix ruff detected errors * Restyled by autopep8 * Fix the cancelling of subscriptions * Used .name instead of needing a feature string mapping. * Removed TC_DEM_1_1 now that the test plan has removed this test case. * Corrected Featuremap -> FeatureMap in test steps * Aligned verification step 2 on TC_DEM_2.3 to actual code and syncd test plan. * Aligned verification step 2 on TC_DEM_2.4 to actual code and syncd test plan. * Added in missing text in step 18 in TC_DEM_2.4 * TC_DEM_2.5 minor update to step 2 and aligning to updated test plan. * TC_DEM_2.5 - slight change to call to name args in validate_feature_map() call * TC_DEM_2.6,2.7,2.8 step 2 alignment with tweaked test plan * TC_DEM_2.6/2.7/2.8 - slight change to call to name args in validate_feature_map() call * Corrected log message in TC_DEM_2.9 step 4b * Updated TC_DEM_2.10 (Q Quality) Basic structure is there - steps need tidying up and fails due to powerAdjustmentCapability not being updated as expected. * Fixed bug where PowerAdjustReason wasn't calling MatterReportingAttributeChangeCallback when reason was changing. * Restyled by clang-format * Restyled by autopep8 * Restyled by isort * Autogen'd the TestSteps from latest test plan * Changed featureSet to 0x7b to enable PA, PFR, FA (plus other) features --------- Co-authored-by: Restyled.io <commits@restyled.io> Co-authored-by: James Harrow <james.harrow@gmail.com> Co-authored-by: Hasty Granbery <hasty@granbery.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses various DEM testing issues:
Fixes [Test Failed] DEM_TC_2_6 Device crashes. #34954
Fixes [Test Failed] [TC_DEM_2_7] fails on ESP32 #34960
Fixes [Test Failed] [TC_DEM_2.8] Step 5 - Invalidcommand returned instead on ConstraintError #34962
Fixes [CERT-TEST-FAILURE] - TC-DEM-1.1 - ClusterRevision value has to be updated for Device Energy Management Cluster #34297
https://github.com/CHIP-Specifications/chip-test-plans/issues/4415
https://github.com/CHIP-Specifications/chip-test-plans/issues/4482
Adds a new TC_DEM_2.10.py script for checking Q Quality and that MatterReporting callback is called when expected (found and fixed some issues here!)
This should be cross-reviewed against the DEM Test Plan change PR:
https://github.com/CHIP-Specifications/chip-test-plans/pull/4543