chore(iast): optimize iast by migrating should_iast_patch function to c [backport 2.21]

[avara1986]

Backport #12774 to 2.21.

TL;DR: Before this PR, IAST startup time was x5,16 lower than an application without IAST startup time
After this PR, IAST startup time was x2,79

Description

In this PR, we aim to improve the startup times of an application when IAST (Interactive Application Security Testing) is enabled. The following improvements have been made:

Performance improvements

• These changes aim to streamline the codebase, improve the functionality of the IAST module, and remove outdated or redundant code.When reading the source code of a module for transformation with AST, it is now done in binary mode, which is 30% faste as shown in this gist: https://gist.github.com/avara1986/8e31a23978b5eecd9dc8f2f3135060b0. The execution times are:

  • open("r"): 0.9285 secs
  • open("rb"): 0.6362 secs

• The function should_iast_patch has been migrated to C. By comparing the original function with the new one using the following timeit command, is 40% faster.

  # refactor
  python -m timeit -n 5000000 -r 5 -u usec -s "from ddtrace.appsec._iast._ast import 
  iastpatch""iastpatch.should_iast_patch('mimodulo.submodulo')" 
  # main
  python -m timeit -n 5000000 -r 5 -u usec -s "from ddtrace.appsec._iast._ast.ast_patching import _should_iast_patch" 
  "_should_iast_patch('package.mypackage')"
  

• I wanted to migrate exec to PyEval_EvalCode but the results are very revealing, exec is approximately 17% faster. The results of this gist https://gist.github.com/avara1986/ae2a65dca1b90afa762af67db680bc2b

  • exec: mean time of 0.083023 seconds
  • PyEval_EvalCode: mean time of 0.099550 seconds

By combining all these changes, the microbenchmarks for startup times show that the baseline takes 2.46 seconds, while this branch takes 1.24 seconds, making it 1.99x faster.

Performance analysis

The _should_iast_patch logs are under asm_config._iast_debug because Debug mode has a significant performance impact, making the function between 1.14x and 3.08x slower

Benchmark Results
--------------------------------------------------------------------------------
Module Name                    Debug Time (s)       No Debug Time (s)    Ratio     
--------------------------------------------------------------------------------
os                                    0.017021            0.005522       3.08x
requests                              0.023237            0.020418       1.14x
ddtrace.appsec._iast                  0.024511            0.012280       2.00x
non.existent.module                   0.023403            0.011170       2.10x

Branch deployed in rel-env and no leaks found:

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

(cherry picked from commit 9159b81)

[github-actions]

CODEOWNERS have been resolved as:

ddtrace/appsec/_iast/_ast/CMakeLists.txt                                @DataDog/asm-python
ddtrace/appsec/_iast/_ast/iastpatch.c                                   @DataDog/asm-python
ddtrace/appsec/_iast/_ast/iastpatch.pyi                                 @DataDog/asm-python
ddtrace/appsec/_iast/_stacktrace.pyi                                    @DataDog/asm-python
ddtrace/appsec/_iast/__init__.py                                        @DataDog/asm-python
ddtrace/appsec/_iast/_ast/ast_patching.py                               @DataDog/asm-python
ddtrace/appsec/_iast/_ast/visitor.py                                    @DataDog/asm-python
ddtrace/appsec/_iast/_loader.py                                         @DataDog/asm-python
ddtrace/appsec/_iast/_logs.py                                           @DataDog/asm-python
ddtrace/appsec/_iast/_taint_tracking/Constants.h                        @DataDog/asm-python
setup.py                                                                @DataDog/python-guild
tests/appsec/iast/_ast/test_ast_patching.py                             @DataDog/asm-python
tests/appsec/iast/_ast/test_ast_patching_type_hints.py                  @DataDog/asm-python
tests/appsec/iast/test_env_var.py                                       @DataDog/asm-python
tests/appsec/iast/test_loader.py                                        @DataDog/asm-python
tests/appsec/iast_packages/test_packages.py                             @DataDog/asm-python

[datadog-dd-trace-py-rkomorn]

Datadog Report

Branch report: backport-12774-to-2.21
Commit report: 64b3300
Test service: dd-trace-py

✅ 0 Failed, 130 Passed, 1468 Skipped, 2m 0.93s Total Time

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-03-24 14:24:13

Comparing candidate commit 64b3300 in PR branch backport-12774-to-2.21 with baseline commit 7ef0289 in branch 2.21.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 418 metrics, 2 unstable metrics.