Skip to content

Commit

Permalink
Limit concurrent downloads by UV (#39810)
Browse files Browse the repository at this point in the history
In some circumstances, the number of parallell downloads run by
UV might hit local networking or proxy limits. This PR limits the
number of concurrent downloads to number of processors that the
docker container has to avoid those problems.

(cherry picked from commit 5aee696)
  • Loading branch information
potiuk authored and ephraimbuddy committed Jun 4, 2024
1 parent e968e04 commit bca0e5e
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions scripts/in_container/_in_container_utils.sh
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,8 @@ function in_container_get_packaging_tool() {
fi
export UPGRADE_EAGERLY="--upgrade --resolution highest"
export UPGRADE_IF_NEEDED="--upgrade"
UV_CONCURRENT_DOWNLOADS=$(nproc --all)
export UV_CONCURRENT_DOWNLOADS
else
echo
echo "${COLOR_BLUE}Using 'pip' to install Airflow${COLOR_RESET}"
Expand Down

0 comments on commit bca0e5e

Please sign in to comment.