chore: update hatch version and remove hatch-containers #10373

taegyunkim · 2024-08-24T00:31:01Z

Initially, started this PR to not use pip install to install hatch. hatch docs doesn't recommend installing hatch via pip, as it could modify the Python environment in which you choose to install.

I addressed some of them but not all.

Removing hatch-containers, which was used to build docs on local mac. It can now build docs natively by installing enchant, also running hatch run docs:build in scripts/ddtest works too.

Checklist

PR author has checked that all the criteria below are met
The PR description includes an overview of the change
The PR description articulates the motivation for the change
The change includes tests OR the PR description describes a testing strategy
The PR description notes risks associated with the change, if any
Newly-added code is easy to change
The change follows the library release note guidelines
The change includes or references documentation updates if necessary
Backport labels are set (if applicable)

Reviewer Checklist

Reviewer has checked that all the criteria below are met
Title is accurate
All changes are related to the pull request's stated goal
Avoids breaking API changes
Testing strategy adequately addresses listed risks
Newly-added code is easy to change
Release note makes sense to a user of the library
If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
Backport labels are set in a manner that is consistent with the release branch maintenance policy

github-actions · 2024-08-24T00:31:30Z

CODEOWNERS have been resolved as:

.circleci/config.templ.yml                                              @DataDog/python-guild @DataDog/apm-core-python
.github/workflows/generate-package-versions.yml                         @DataDog/python-guild @DataDog/apm-core-python
.github/workflows/unit_tests.yml                                        @DataDog/python-guild @DataDog/apm-core-python
.readthedocs.yml                                                        @DataDog/python-guild @DataDog/apm-core-python
docker/Dockerfile                                                       @DataDog/apm-core-python
docs/contributing-testing.rst                                           @DataDog/python-guild
hatch.toml                                                              @DataDog/python-guild
scripts/ddtest                                                          @DataDog/apm-core-python
scripts/docs/build.sh                                                   @DataDog/python-guild
scripts/docs/install.sh                                                 @DataDog/python-guild

datadog-dd-trace-py-rkomorn · 2024-08-24T01:00:19Z

Datadog Report

Branch report: taegyunkim/ddtest-hatch-no
Commit report: 73de7c3
Test service: dd-trace-py

✅ 0 Failed, 141066 Passed, 1673 Skipped, 9h 11m 40.15s Total duration (1m 26.04s time saved)
❄️ 4 New Flaky

New Flaky Tests (4)

test_iter_events_multi_thread - test_memalloc.py - Last Failure
Expand for error
```
 assert 815 >= 1000
```

test_cmdi - test_pygoat.py - Last Failure

Expand for error

 HTTPConnectionPool(host='0.0.0.0', port=9126): Max retries exceeded with url: /test/session/traces?test_session_token=pygoat_test (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09957c92a0>: Failed to establish a new connection: [Errno 111] Connection refused'))

test_sqli - test_pygoat.py - Last Failure

Expand for error

 HTTPConnectionPool(host='0.0.0.0', port=9126): Max retries exceeded with url: /test/session/start?test_session_token=pygoat_test (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0991e788b0>: Failed to establish a new connection: [Errno 111] Connection refused'))

test_ssrf2 - test_pygoat.py - Last Failure

Expand for error

 HTTPConnectionPool(host='0.0.0.0', port=9126): Max retries exceeded with url: /test/session/start?test_session_token=pygoat_test (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0991e4a560>: Failed to establish a new connection: [Errno 111] Connection refused'))

pr-commenter · 2024-08-24T01:22:06Z

Benchmarks

Benchmark execution time: 2024-08-26 21:37:10

Comparing candidate commit 73de7c3 in PR branch taegyunkim/ddtest-hatch-no with baseline commit 769579f in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 353 metrics, 47 unstable metrics.

taegyunkim · 2024-08-26T13:54:23Z

This keeps failing on dd_coverage, something more fundamental is wrong looking at discussion on our slack regarding testrunner image

…1.2.0?

erikayasuda

Just some clarifying questions, otherwise LGTM

.circleci/config.templ.yml

docker/Dockerfile

#10373 modified docker/Dockerfile but didn't trigger conftests circle ci test and merged. After the merge on main, conftests started to fail on main. 1. Fix it by specifying the python version on `hatch.toml`. This actually looks like a bug on hatch and for now setting the version to be 3.10 as circle ci docker image we use for conftests is based on python 3.10, and hatch seems to pick up the version when it's available. If not set, hatch installs 3.12 and results in this weird error, [failing to install ddtrace as it can't find clang compiler. ](https://app.circleci.com/pipelines/github/DataDog/dd-trace-py/70063/workflows/21549ae8-682f-4538-bd5a-b1b1077e5043/jobs/4254100) 2. Also updates suitespec generator gen_circleci_config.py calls `fnmatch.filter(changed_files, pattern)` to get the list of files matching pattern and look below. ``` ❯ python Python 3.12.5 (main, Aug 20 2024, 16:07:54) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import fnmatch >>> fnmatch.filter(["docker/Dockerfile"], "docker") [] >>> fnmatch.filter(["docker/Dockerfile"], "docker*") ['docker/Dockerfile'] ``` ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

1. Upgrade testrunner image to the one generated by #10373 2. testrunner now runs python 3.11.9 (upgraded from 3.11.3) and gevent isn't compatible with python 3.11.9, python/cpython#117983, so update the test to run when version < 3.11.9 3. Regenerate riot file for appsec_integrations using python 3.12, as outdated pydantic doesn't work with 3.12.4, see pydantic/pydantic#9637. This also resulted in updating pytest version and `setup_module` is used instead of `setup`. 4. Add `CMAKE_BUILD_PARALLEL_LEVEL` to a few places ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

taegyunkim requested a review from a team as a code owner August 24, 2024 00:31

taegyunkim requested a review from erikayasuda August 24, 2024 00:31

taegyunkim added the changelog/no-changelog A changelog entry is not required for this PR. label Aug 24, 2024

taegyunkim enabled auto-merge (squash) August 24, 2024 00:32

taegyunkim requested a review from a team as a code owner August 24, 2024 01:10

taegyunkim changed the title ~~chore: don't install hatch via pip in scripts/ddtest~~ chore: don't install hatch via pip Aug 24, 2024

romainkomorn-exdatadog approved these changes Aug 26, 2024

View reviewed changes

taegyunkim closed this Aug 26, 2024

auto-merge was automatically disabled August 26, 2024 13:54
Pull request was closed

taegyunkim reopened this Aug 26, 2024

taegyunkim changed the title ~~chore: don't install hatch via pip~~ [do not merge] chore: don't install hatch via pip Aug 26, 2024

taegyunkim force-pushed the taegyunkim/ddtest-hatch-no branch from c6c223d to 0d4e88f Compare August 26, 2024 14:43

taegyunkim changed the title ~~[do not merge] chore: don't install hatch via pip~~ chore: update hatch version and remove hatch-containers Aug 26, 2024

taegyunkim added 13 commits August 26, 2024 17:01

chore: don't install hatch via pip in scripts/ddtest

2db3bcf

setup hatch using binary instead of pip install

00a71b9

let there be sudo

7693bb8

don't install hatch-containers as this seems to be installing hatch>=…

95ae385

…1.2.0?

don't install hatch via pip

b6a5884

no sudo

ca36233

version up

9a48419

build_docs is using testrunner which already has hatch

fc4c896

use latest hatch

022ed50

revert images

25ca498

comment

8ad635d

try installing hatch

eae88d2

fix version

6926fda

taegyunkim added 12 commits August 26, 2024 17:01

print

46ea66b

print path and python versions

9973ada

do we need this?

c57a0ad

print

48fd187

it was indeed needed

8bb3c70

oh i hate python

4e972ea

update

d8e43cd

can build docs on mac too

70d0743

update

05228ff

not circle ci check

55b56ec

comment

eb7a14b

check it right

73de7c3

taegyunkim force-pushed the taegyunkim/ddtest-hatch-no branch from 0037f4b to 73de7c3 Compare August 26, 2024 21:01

erikayasuda approved these changes Aug 26, 2024

View reviewed changes

.circleci/config.templ.yml Show resolved Hide resolved

docker/Dockerfile Show resolved Hide resolved

taegyunkim merged commit ecc5741 into main Aug 27, 2024
206 of 207 checks passed

taegyunkim deleted the taegyunkim/ddtest-hatch-no branch August 27, 2024 00:11

taegyunkim mentioned this pull request Aug 27, 2024

chore: fix broken conftests circle ci #10394

Merged

2 tasks

taegyunkim mentioned this pull request Aug 28, 2024

chore: upgrade testrunner image #10405

Merged

2 tasks

taegyunkim mentioned this pull request Sep 5, 2024

chore(iast): optimize modulo aspect more (3/3) #10514

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: update hatch version and remove hatch-containers #10373

chore: update hatch version and remove hatch-containers #10373

taegyunkim commented Aug 24, 2024 •

edited by erikayasuda

Loading

github-actions bot commented Aug 24, 2024 •

edited

Loading

datadog-dd-trace-py-rkomorn bot commented Aug 24, 2024 •

edited

Loading

pr-commenter bot commented Aug 24, 2024 •

edited

Loading

taegyunkim commented Aug 26, 2024 •

edited

Loading

erikayasuda left a comment

chore: update hatch version and remove hatch-containers #10373

chore: update hatch version and remove hatch-containers #10373

Conversation

taegyunkim commented Aug 24, 2024 • edited by erikayasuda Loading

Checklist

Reviewer Checklist

github-actions bot commented Aug 24, 2024 • edited Loading

datadog-dd-trace-py-rkomorn bot commented Aug 24, 2024 • edited Loading

Datadog Report

New Flaky Tests (4)

pr-commenter bot commented Aug 24, 2024 • edited Loading

Benchmarks

taegyunkim commented Aug 26, 2024 • edited Loading

erikayasuda left a comment

Choose a reason for hiding this comment

taegyunkim commented Aug 24, 2024 •

edited by erikayasuda

Loading

github-actions bot commented Aug 24, 2024 •

edited

Loading

datadog-dd-trace-py-rkomorn bot commented Aug 24, 2024 •

edited

Loading

pr-commenter bot commented Aug 24, 2024 •

edited

Loading

taegyunkim commented Aug 26, 2024 •

edited

Loading