Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bumping Jinja2 to version 3.1.1 for CVE #487

Merged
merged 1 commit into from
Mar 22, 2024

Conversation

beaioun
Copy link
Collaborator

@beaioun beaioun commented Mar 22, 2024

Description

This change updates the setup file for Jinja2 from version 2.11.3 to 3.1.3 to resolve the CVE in the older version.

Issues Resolved

#481

Testing

  • New functionality includes testing

Test locally and through Github Actions
The Jinja2 3.1.3 was tested to be compatible with OSB


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: beaioun <mshi@ucsd.edu>
Copy link
Collaborator

@IanHoang IanHoang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were you able to run a quick test with this in --test-mode to verify that the changes are okay?

@beaioun
Copy link
Collaborator Author

beaioun commented Mar 22, 2024

Were you able to run a quick test with this in --test-mode to verify that the changes are okay?

@IanHoang Yes and here is the result output for the test run:

Using cached Jinja2-3.1.3-py3-none-any.whl (133 kB)
Installing collected packages: Jinja2, opensearch-benchmark
  Attempting uninstall: Jinja2
    Found existing installation: Jinja2 2.11.3
    Uninstalling Jinja2-2.11.3:
      Successfully uninstalled Jinja2-2.11.3
  Attempting uninstall: opensearch-benchmark
    Found existing installation: opensearch-benchmark 1.4.0
    Uninstalling opensearch-benchmark-1.4.0:
      Successfully uninstalled opensearch-benchmark-1.4.0
  Running setup.py develop for opensearch-benchmark
Successfully installed Jinja2-3.1.3 opensearch-benchmark-1.4.0
will@LAPTOP-60H1M2PO:/mnt/c/Wills_Working_Directory/Development/opensearch-benchmark$ opensearch-benchmark execute-test --pipeline=benchmark-only --workload=percolator --target-host=https://localhost:9200 --client-options=basic_auth_user:admin,basic_auth_password:admin,verify_certs:false --test-mode

   ____                  _____                      __       ____                  __                         __  
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<   
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|  
    /_/

[INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds.
[INFO] Executing test with workload [percolator], test_procedure [append-no-conflicts] and provision_config_instance ['external'] with version [2.11.1].

[WARNING] merges_total_time is 31 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] indexing_total_time is 1871 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] refresh_total_time is 533 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
Running delete-index                                                           [100% done]
Running create-index                                                           [100% done]
Running check-cluster-health                                                   [100% done]
Running index                                                                  [100% done]
Running refresh-after-index                                                    [100% done]
Running force-merge                                                            [100% done]
Running refresh-after-force-merge                                              [100% done]
Running wait-until-merges-finish                                               [100% done]
Running percolator_with_content_president_bush                                 [100% done]
Running percolator_with_content_saddam_hussein                                 [100% done]
Running percolator_with_content_hurricane_katrina                              [100% done]
Running percolator_with_content_google                                         [100% done]
Running percolator_no_score_with_content_google                                [100% done]
Running percolator_with_highlighting                                           [100% done]
Running percolator_with_content_ignore_me                                      [100% done]
Running percolator_no_score_with_content_ignore_me                             [100% done]

------------------------------------------------------
    _______             __   _____
   / ____(_)___  ____ _/ /  / ___/_________  ________
  / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
 / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
/_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
------------------------------------------------------

|                                                         Metric |                                       Task |       Value |   Unit |
|---------------------------------------------------------------:|-------------------------------------------:|------------:|-------:|
|                     Cumulative indexing time of primary shards |                                            |   0.0134667 |    min |
|             Min cumulative indexing time across primary shards |                                            |           0 |    min |
|          Median cumulative indexing time across primary shards |                                            |  0.00126667 |    min |
|             Max cumulative indexing time across primary shards |                                            |  0.00373333 |    min |
|            Cumulative indexing throttle time of primary shards |                                            |           0 |    min |
|    Min cumulative indexing throttle time across primary shards |                                            |           0 |    min |
| Median cumulative indexing throttle time across primary shards |                                            |           0 |    min |
|    Max cumulative indexing throttle time across primary shards |                                            |           0 |    min |
|                        Cumulative merge time of primary shards |                                            |      0.0015 |    min |
|                       Cumulative merge count of primary shards |                                            |           3 |        |
|                Min cumulative merge time across primary shards |                                            |           0 |    min |
|             Median cumulative merge time across primary shards |                                            |           0 |    min |
|                Max cumulative merge time across primary shards |                                            |      0.0015 |    min |
|               Cumulative merge throttle time of primary shards |                                            |           0 |    min |
|       Min cumulative merge throttle time across primary shards |                                            |           0 |    min |
|    Median cumulative merge throttle time across primary shards |                                            |           0 |    min |
|       Max cumulative merge throttle time across primary shards |                                            |           0 |    min |
|                      Cumulative refresh time of primary shards |                                            |   0.0109833 |    min |
|                     Cumulative refresh count of primary shards |                                            |         134 |        |
|              Min cumulative refresh time across primary shards |                                            |           0 |    min |
|           Median cumulative refresh time across primary shards |                                            | 0.000333333 |    min |
|              Max cumulative refresh time across primary shards |                                            |     0.00755 |    min |
|                        Cumulative flush time of primary shards |                                            |           0 |    min |
|                       Cumulative flush count of primary shards |                                            |           0 |        |
|                Min cumulative flush time across primary shards |                                            |           0 |    min |
|             Median cumulative flush time across primary shards |                                            |           0 |    min |
|                Max cumulative flush time across primary shards |                                            |           0 |    min |
|                                        Total Young Gen GC time |                                            |           0 |      s |
|                                       Total Young Gen GC count |                                            |           0 |        |
|                                          Total Old Gen GC time |                                            |           0 |      s |
|                                         Total Old Gen GC count |                                            |           0 |        |
|                                                     Store size |                                            | 0.000676217 |     GB |
|                                                  Translog size |                                            | 7.20406e-05 |     GB |
|                                         Heap used for segments |                                            |           0 |     MB |
|                                       Heap used for doc values |                                            |           0 |     MB |
|                                            Heap used for terms |                                            |           0 |     MB |
|                                            Heap used for norms |                                            |           0 |     MB |
|                                           Heap used for points |                                            |           0 |     MB |
|                                    Heap used for stored fields |                                            |           0 |     MB |
|                                                  Segment count |                                            |          39 |        |
|                                                 Min Throughput |                                      index |     7543.18 | docs/s |
|                                                Mean Throughput |                                      index |     7543.18 | docs/s |
|                                              Median Throughput |                                      index |     7543.18 | docs/s |
|                                                 Max Throughput |                                      index |     7543.18 | docs/s |
|                                        50th percentile latency |                                      index |     100.622 |     ms |
|                                       100th percentile latency |                                      index |      111.13 |     ms |
|                                   50th percentile service time |                                      index |     100.622 |     ms |
|                                  100th percentile service time |                                      index |      111.13 |     ms |
|                                                     error rate |                                      index |           0 |      % |
|                                                 Min Throughput |                   wait-until-merges-finish |       54.88 |  ops/s |
|                                                Mean Throughput |                   wait-until-merges-finish |       54.88 |  ops/s |
|                                              Median Throughput |                   wait-until-merges-finish |       54.88 |  ops/s |
|                                                 Max Throughput |                   wait-until-merges-finish |       54.88 |  ops/s |
|                                       100th percentile latency |                   wait-until-merges-finish |     16.8764 |     ms |
|                                  100th percentile service time |                   wait-until-merges-finish |     16.8764 |     ms |
|                                                     error rate |                   wait-until-merges-finish |           0 |      % |
|                                                 Min Throughput |     percolator_with_content_president_bush |       48.49 |  ops/s |
|                                                Mean Throughput |     percolator_with_content_president_bush |       48.49 |  ops/s |
|                                              Median Throughput |     percolator_with_content_president_bush |       48.49 |  ops/s |
|                                                 Max Throughput |     percolator_with_content_president_bush |       48.49 |  ops/s |
|                                       100th percentile latency |     percolator_with_content_president_bush |     29.9591 |     ms |
|                                  100th percentile service time |     percolator_with_content_president_bush |     9.08803 |     ms |
|                                                     error rate |     percolator_with_content_president_bush |           0 |      % |
|                                                 Min Throughput |     percolator_with_content_saddam_hussein |       49.37 |  ops/s |
|                                                Mean Throughput |     percolator_with_content_saddam_hussein |       49.37 |  ops/s |
|                                              Median Throughput |     percolator_with_content_saddam_hussein |       49.37 |  ops/s |
|                                                 Max Throughput |     percolator_with_content_saddam_hussein |       49.37 |  ops/s |
|                                       100th percentile latency |     percolator_with_content_saddam_hussein |     32.6356 |     ms |
|                                  100th percentile service time |     percolator_with_content_saddam_hussein |      12.146 |     ms |
|                                                     error rate |     percolator_with_content_saddam_hussein |           0 |      % |
|                                                 Min Throughput |  percolator_with_content_hurricane_katrina |       60.67 |  ops/s |
|                                                Mean Throughput |  percolator_with_content_hurricane_katrina |       60.67 |  ops/s |
|                                              Median Throughput |  percolator_with_content_hurricane_katrina |       60.67 |  ops/s |
|                                                 Max Throughput |  percolator_with_content_hurricane_katrina |       60.67 |  ops/s |
|                                       100th percentile latency |  percolator_with_content_hurricane_katrina |     25.1797 |     ms |
|                                  100th percentile service time |  percolator_with_content_hurricane_katrina |     8.46809 |     ms |
|                                                     error rate |  percolator_with_content_hurricane_katrina |           0 |      % |
|                                                 Min Throughput |             percolator_with_content_google |       38.24 |  ops/s |
|                                                Mean Throughput |             percolator_with_content_google |       38.24 |  ops/s |
|                                              Median Throughput |             percolator_with_content_google |       38.24 |  ops/s |
|                                                 Max Throughput |             percolator_with_content_google |       38.24 |  ops/s |
|                                       100th percentile latency |             percolator_with_content_google |     40.6798 |     ms |
|                                  100th percentile service time |             percolator_with_content_google |     14.2949 |     ms |
|                                                     error rate |             percolator_with_content_google |           0 |      % |
|                                                 Min Throughput |    percolator_no_score_with_content_google |       37.07 |  ops/s |
|                                                Mean Throughput |    percolator_no_score_with_content_google |       37.07 |  ops/s |
|                                              Median Throughput |    percolator_no_score_with_content_google |       37.07 |  ops/s |
|                                                 Max Throughput |    percolator_no_score_with_content_google |       37.07 |  ops/s |
|                                       100th percentile latency |    percolator_no_score_with_content_google |      44.089 |     ms |
|                                  100th percentile service time |    percolator_no_score_with_content_google |      16.837 |     ms |
|                                                     error rate |    percolator_no_score_with_content_google |           0 |      % |
|                                                 Min Throughput |               percolator_with_highlighting |       58.31 |  ops/s |
|                                                Mean Throughput |               percolator_with_highlighting |       58.31 |  ops/s |
|                                              Median Throughput |               percolator_with_highlighting |       58.31 |  ops/s |
|                                                 Max Throughput |               percolator_with_highlighting |       58.31 |  ops/s |
|                                       100th percentile latency |               percolator_with_highlighting |     26.9303 |     ms |
|                                  100th percentile service time |               percolator_with_highlighting |      9.5425 |     ms |
|                                                     error rate |               percolator_with_highlighting |           0 |      % |
|                                                 Min Throughput |          percolator_with_content_ignore_me |       22.39 |  ops/s |
|                                                Mean Throughput |          percolator_with_content_ignore_me |       22.39 |  ops/s |
|                                              Median Throughput |          percolator_with_content_ignore_me |       22.39 |  ops/s |
|                                                 Max Throughput |          percolator_with_content_ignore_me |       22.39 |  ops/s |
|                                       100th percentile latency |          percolator_with_content_ignore_me |     69.6343 |     ms |
|                                  100th percentile service time |          percolator_with_content_ignore_me |     24.7344 |     ms |
|                                                     error rate |          percolator_with_content_ignore_me |           0 |      % |
|                                                 Min Throughput | percolator_no_score_with_content_ignore_me |       43.97 |  ops/s |
|                                                Mean Throughput | percolator_no_score_with_content_ignore_me |       43.97 |  ops/s |
|                                              Median Throughput | percolator_no_score_with_content_ignore_me |       43.97 |  ops/s |
|                                                 Max Throughput | percolator_no_score_with_content_ignore_me |       43.97 |  ops/s |
|                                       100th percentile latency | percolator_no_score_with_content_ignore_me |     38.8761 |     ms |
|                                  100th percentile service time | percolator_no_score_with_content_ignore_me |     15.8837 |     ms |
|                                                     error rate | percolator_no_score_with_content_ignore_me |           0 |      % |


--------------------------------
[INFO] SUCCESS (took 14 seconds)
--------------------------------

So far no issues, let me know if there is something I missed that needs to be tested out.

Copy link
Collaborator

@IanHoang IanHoang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@IanHoang IanHoang added 1.0 Backport to patch version branch 1.X Backport to minor version branch labels Mar 22, 2024
@IanHoang IanHoang merged commit 415c1d7 into opensearch-project:main Mar 22, 2024
8 checks passed
@IanHoang IanHoang added the Mend: dependency security vulnerability Security vulnerability detected by Mend label Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1.X Backport to minor version branch 1.0 Backport to patch version branch Mend: dependency security vulnerability Security vulnerability detected by Mend
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants