Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added limit to FIM csv export #7182

Merged

Conversation

JuanGarriuz
Copy link
Member

@JuanGarriuz JuanGarriuz commented Nov 29, 2024

Description

This PR adds:

  • A setting to set the maximum number of rows to be included in csv reports
  • A warning icon explaining that that will only be visible if there are more results than the rows limit.
  • The table's sorting as a filter for exporting csv, to include the results in the reports as they are shown in the table

Issues Resolved

Evidence

imagen

Peek 12-12-2024 11-38

Setting the limit to 1439
imagen

Test

Pre requisites

  • Bring up an environment with a real manager and an agent
  • Add a high number of files in the agent for the FIM.
    • By default it reads the directory /etc. You can create the following file inside /etc
      for i in {1..12000}; do 
          echo "TEST" >> "file$i.txt" 
        done 
      
    And execute it with bash <filename>. After that, restart the agent and wait for all the files to be read (it can take some time).
  • Navigate to File Integrity Monitoring/Inventory and select the agent.

Test

  • Verify that there are more than 10.000 files and the warning message appears.

  • Click on Export formatted and verify that the exported file only contains 10.000 results.

  • Change the sorting in the table, click on Export formatted and verify that the exported file is sorted like the table.

  • Apply some search term to reduce the number of files to less than 10.000 and verify that the warning is not shown.

  • Click on Export formatted and verify that the exported report contains the same results shown in the table.

  • Navigate to Dashboard management > App Settings and change the reports.csv.maxRows setting to a different number

  • Repeat the previous steps considering the new set value

Check List

  • All tests pass
    • yarn test:jest
  • New functionality includes testing.
  • New functionality has been documented.
  • Update CHANGELOG.md
  • Commits are signed per the DCO using --signoff

@JuanGarriuz JuanGarriuz changed the title Added info message about many files on FIM Added info message about too many files on FIM Nov 29, 2024
@JuanGarriuz JuanGarriuz linked an issue Nov 29, 2024 that may be closed by this pull request
@JuanGarriuz JuanGarriuz marked this pull request as ready for review November 29, 2024 07:53
chantal-kelm

This comment was marked as outdated.

guidomodarelli

This comment was marked as outdated.

guidomodarelli

This comment was marked as outdated.

@chantal-kelm chantal-kelm self-requested a review December 3, 2024 13:48
@Tostti Tostti changed the title Added info message about too many files on FIM Added limit to FIM csv export Dec 5, 2024
@guidomodarelli guidomodarelli self-requested a review December 5, 2024 17:51
guidomodarelli

This comment was marked as resolved.

Copy link
Member

@asteriscos asteriscos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The exported file seems to have columns not selected in the table.

image

image

The configuration seems to be working as expected.

image

image

image

chantal-kelm
chantal-kelm previously approved these changes Dec 12, 2024
Copy link
Member

@chantal-kelm chantal-kelm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CR: ✅
TEST: ✅

Captura de pantalla 2024-12-12 a la(s) 3 58 20 p  m Captura de pantalla 2024-12-12 a la(s) 4 43 15 p  m Captura de pantalla 2024-12-12 a la(s) 5 28 48 p  m

Copy link
Contributor

@guidomodarelli guidomodarelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Legend:
⚫ 🠆 The test hasn't started yet.
🟢 🠆 All checks passed.
🟡 🠆 There is at least one expected fail or skipped test and no failures.
🔴 🠆 There is at least one failed check.
⚪ 🠆 Doesn't apply.
🔧 🠆 Request changes.

CR 🔧
TEST 🟢

  • Verify that there are more than 10.000 files and the warning message appears. 🟢

    image

  • Click on Export formatted and verify that the exported file only contains 10.000 results. 🟢

    image

  • Change the sorting in the table, click on Export formatted and verify that the exported file is sorted like the table. 🟢

    image

  • Apply some search term to reduce the number of files to less than 10.000 and verify that the warning is not shown. 🟢

    image

  • Click on Export formatted and verify that the exported report contains the same results shown in the table. 🟢

    image

  • Navigate to Dashboard management > App Settings and change the reports.csv.maxRows setting to a different number.

    image

  • Verify that there are more than 1.000 files and the warning message appears. 🟢

    image

  • Click on Export formatted and verify that the exported file only contains 1.000 results. 🟢

    image

  • Change the sorting in the table, click on Export formatted and verify that the exported file is sorted like the table. 🟢

    image
    image

  • Apply some search term to reduce the number of files to less than 1.000 and verify that the warning is not shown. 🟢

    image

  • Click on Export formatted and verify that the exported report contains the same results shown in the table. 🟢

    image

guidomodarelli
guidomodarelli previously approved these changes Dec 13, 2024
Copy link
Contributor

@guidomodarelli guidomodarelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Legend:
⚫ 🠆 The test hasn't started yet.
🟢 🠆 All checks passed.
🟡 🠆 There is at least one expected fail or skipped test and no failures.
🔴 🠆 There is at least one failed check.
⚪ 🠆 Doesn't apply.
🔧 🠆 Request changes.

CR 🟢

guidomodarelli
guidomodarelli previously approved these changes Dec 13, 2024
Copy link
Contributor

@guidomodarelli guidomodarelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Legend:
⚫ 🠆 The test hasn't started yet.
🟢 🠆 All checks passed.
🟡 🠆 There is at least one expected fail or skipped test and no failures.
🔴 🠆 There is at least one failed check.
⚪ 🠆 Doesn't apply.
🔧 🠆 Request changes.

CR 🟢

asteriscos
asteriscos previously approved these changes Dec 13, 2024
Copy link
Member

@asteriscos asteriscos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CR: 🟢
Test: 🟢
image

@JuanGarriuz
Copy link
Member Author

JuanGarriuz commented Dec 13, 2024

Test 🟢

Verify that there are more than 10.000 files and the warning message appears.

image

Click on Export formatted and verify that the exported file only contains 10.000 results.
[First row is the header]

image

image

Change the sorting in the table, click on Export formatted and verify that the exported file is sorted like the table.

image
image

Apply some search term to reduce the number of files to less than 10.000 and verify that the warning is not shown.

image

Click on Export formatted and verify that the exported report contains the same results shown in the table.

image

Navigate to Dashboard management > App Settings and change the reports.csv.maxRows setting to a different number

image
image
image
image

CR 🟢

Copy link
Contributor

Wazuh Core plugin code coverage (Jest) test % values
Statements 45.3% ( 400 / 883 )
Branches 41.09% ( 157 / 382 )
Functions 43.31% ( 136 / 314 )
Lines 45.48% ( 398 / 875 )

Copy link
Contributor

Wazuh Check Updates plugin code coverage (Jest) test % values
Statements 76.44% ( 172 / 225 )
Branches 58.65% ( 61 / 104 )
Functions 61.7% ( 29 / 47 )
Lines 76.44% ( 172 / 225 )

Copy link
Contributor

Main plugin code coverage (Jest) test % values
Statements 15.27% ( 4549 / 29779 )
Branches 10.08% ( 1946 / 19292 )
Functions 15.18% ( 1091 / 7184 )
Lines 15.49% ( 4438 / 28649 )

@asteriscos asteriscos merged commit c4fff72 into 4.10.2 Dec 13, 2024
5 of 6 checks passed
@asteriscos asteriscos deleted the enhacement/7068-add-alert-message-about-to-many-files-on-fim branch December 13, 2024 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Timeout error when exporting CSV in FIM
6 participants