Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop sentis upgrade #5979

Merged
merged 36 commits into from
Oct 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
ae4a5bc
Commiting changes.
miguelalonsojr Apr 5, 2023
718ea24
Initial barracuda 4 upgrade.
miguelalonsojr Apr 9, 2023
7d4ed74
Play mode tests passing.
miguelalonsojr Apr 12, 2023
76b2e29
Edit mode tests passing.
miguelalonsojr Apr 13, 2023
3b0ebbe
Training fixes.
miguelalonsojr Apr 16, 2023
aa5256c
Fixed performance issue with stacking sensor.
miguelalonsojr Apr 16, 2023
39eaa39
Fixed failing tests and issue with stacking sensor.
miguelalonsojr Apr 16, 2023
b5b6264
Updated examples for barracuda 4 upgrade.
miguelalonsojr Apr 17, 2023
621756f
Fixed issue with attention ONNX export w.r.t. dimensions.
miguelalonsojr Apr 19, 2023
82e45d6
Fixed issue with Buffer Sensor and Recurrent In/Out.
miguelalonsojr Apr 20, 2023
1c322a3
Retrained old policies and updated with ONNX policies. Deprecated old…
miguelalonsojr Apr 20, 2023
7aca88a
Saving work.
miguelalonsojr Jul 26, 2023
a829b72
Saving work.
miguelalonsojr Aug 14, 2023
6a6237c
Updating to Sentis 1.1.1-exp.2
miguelalonsojr Sep 11, 2023
9d34079
Fixed more errors with Sentis upgrade.
miguelalonsojr Sep 12, 2023
6b8e7a3
Fixed tensor allocation issue in TensorUtils.ResizeTensor. Inference …
miguelalonsojr Sep 12, 2023
56e7630
Fixed broken Sentis model links for some example environments.
miguelalonsojr Sep 12, 2023
343cc5f
Fixed some broken edit mode tests.
miguelalonsojr Sep 12, 2023
a98520e
Fixed some failing tests.
miguelalonsojr Sep 12, 2023
c6f8303
Fixing bugs with GPU inference on Sentis.
miguelalonsojr Oct 2, 2023
01dd77f
Updated packages lock and onnx meta files.
miguelalonsojr Oct 2, 2023
c8f46d0
Refactoring all Barracuda related naming to Sentis.
miguelalonsojr Oct 2, 2023
b4122df
Python max version bump.
miguelalonsojr Oct 2, 2023
17ee61f
Precommit fixes.
miguelalonsojr Oct 2, 2023
1fcf380
Pinned tensorboard version
miguelalonsojr Oct 2, 2023
44d80de
Revert tensorboard version.
miguelalonsojr Oct 2, 2023
565568c
Fixed rpc tests.
miguelalonsojr Oct 2, 2023
d49357c
Fixed failing python tests.
miguelalonsojr Oct 3, 2023
d40d4d2
Fixed some more failing tests. Added six as an explicit dependency du…
miguelalonsojr Oct 3, 2023
fff6d98
gha fix.
miguelalonsojr Oct 3, 2023
2196124
Updated environment registry for Sentis.
miguelalonsojr Oct 3, 2023
e6a7c0b
Fixed texture sensor test.
miguelalonsojr Oct 3, 2023
42ab41f
Develop python 3.10 (#5981)
miguelalonsojr Oct 4, 2023
6427cdf
Develop torch 1.13.1 (#5982)
miguelalonsojr Oct 5, 2023
4e39553
Updated protobufs. (#5983)
miguelalonsojr Oct 5, 2023
ddcc42a
Updated training init tests to remove inference test temporarily. (#5…
miguelalonsojr Oct 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/colab.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.9.x
python-version: 3.10.12
- uses: actions/setup-dotnet@v1
with:
dotnet-version: '6.0.x'
Expand Down
8 changes: 2 additions & 6 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,14 +37,10 @@ jobs:
# If one test in the matrix fails we still want to run the others.
fail-fast: false
matrix:
python-version: [3.8.x, 3.9.x, 3.10.x]
python-version: [3.10.x]
include:
- python-version: 3.8.x
pip_constraints: test_constraints_min_version.txt
- python-version: 3.9.x
pip_constraints: test_constraints_mid_version.txt
- python-version: 3.10.x
pip_constraints: test_constraints_max_version.txt
pip_constraints: test_constraints_version.txt
steps:
- uses: actions/checkout@v2
- name: Set up Python
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: 3.8.x
python-version: 3.10.x
- uses: actions/setup-ruby@v1
env:
ImageOS: ubuntu20
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publish_docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ jobs:
uses: actions/checkout@v2
with:
token: ${{ secrets.PUBLIC_GH_TOKEN }}
- name: Setup Python 3.8
- name: Setup Python 3.10
uses: actions/setup-python@v2
with:
python-version: 3.8
python-version: 3.10
- name: Publish docs
run: |
pip install mkdocs
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publish_pypi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@ jobs:

steps:
- uses: actions/checkout@main
- name: Set up Python 3.8
- name: Set up Python 3.10
uses: actions/setup-python@v2
with:
python-version: 3.8
python-version: 3.10
- name: Install dependencies
run: pip install setuptools wheel twine --user
- name: verify git tag vs. version
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publish_pypi_python_api.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@ jobs:

steps:
- uses: actions/checkout@v2
- name: Set up Python 3.8
- name: Set up Python 3.10
uses: actions/setup-python@v2
with:
python-version: 3.8
python-version: 3.10
- name: Install dependencies
run: pip install setuptools wheel twine --user
- name: verify git tag vs. version
Expand Down
10 changes: 3 additions & 7 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,14 +37,10 @@ jobs:
# If one test in the matrix fails we still want to run the others.
fail-fast: false
matrix:
python-version: [3.8.x, 3.9.x, 3.10.x]
python-version: [3.10.12]
include:
- python-version: 3.8.x
pip_constraints: test_constraints_min_version.txt
- python-version: 3.9.x
pip_constraints: test_constraints_mid_version.txt
- python-version: 3.10.x
pip_constraints: test_constraints_max_version.txt
- python-version: 3.10.12
pip_constraints: test_constraints_version.txt
steps:
- uses: actions/checkout@v2
- name: Set up Python
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -135,4 +135,4 @@ repos:
language: python
entry: ./utils/generate_markdown_docs.py --package_dirs ml-agents-envs ml-agents
pass_filenames: false
additional_dependencies: [pyyaml, pydoc-markdown==3.10.1]
additional_dependencies: [pyyaml==5.3, pydoc-markdown==3.10.1]
2 changes: 1 addition & 1 deletion .yamato/com.unity.ml-agents-coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ test_coverage_{{ package.name }}_{{ platform.name }}_{{ editor.version }}:
- npm install upm-ci-utils@1.27.0 -g --registry https://artifactory.prd.cds.internal.unity3d.com/artifactory/api/npm/upm-npm
- upm-ci project test -u {{ editor.version }} --type project-tests --project-path {{ editor.testProject }} --package-filter {{ package.name }} {{ coverageOptions }} --extra-utr-arg "reruncount=2"
- |
conda activate python3.8
conda activate python3.10
python3 ml-agents/tests/yamato/check_coverage_percent.py upm-ci~/test-results/ {{ package.minCoveragePct }}
artifacts:
logs:
Expand Down
2 changes: 1 addition & 1 deletion .yamato/com.unity.ml-agents-pack.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ pack:
commands:
- |
eval "$($HOME/anaconda/bin/conda shell.bash hook)"
conda activate python3.8
conda activate python3.10
python3 -m pip install unity-downloader-cli --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple --upgrade
unity-downloader-cli -u 2022.3 -c editor --wait --fast
./.Editor/Unity -projectPath Project -batchMode -executeMethod Unity.MLAgents.SampleExporter.ExportCuratedSamples -logFile -
Expand Down
2 changes: 1 addition & 1 deletion .yamato/com.unity.ml-agents-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ test_{{ package.name }}_{{ platform.name }}_trunk:
commands:
- |
{% if platform.name == "linux" %}
conda activate python3.8
conda activate python3.10
{% endif %}
python -m pip install unity-downloader-cli --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple --upgrade
unity-downloader-cli -u trunk -c editor --wait --fast
Expand Down
2 changes: 1 addition & 1 deletion .yamato/compressed-sensor-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ test_compressed_obs_{{ editor.version }}:
commands:
- |
eval "$($HOME/anaconda/bin/conda shell.bash hook)"
conda activate python3.8
conda activate python3.10
python -m pip install pyyaml --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
python -u -m ml-agents.tests.yamato.setup_venv
python ml-agents/tests/yamato/scripts/run_compressed_sensor.py --env=artifacts/testPlayer-TestGridCompressed
Expand Down
2 changes: 1 addition & 1 deletion .yamato/gym-interface-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ test_gym_interface_{{ editor.version }}:
commands:
- |
eval "$($HOME/anaconda/bin/conda shell.bash hook)"
conda activate python3.8
conda activate python3.10
python -m pip install wheel --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
python -m pip install pyyaml --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
python -u -m ml-agents.tests.yamato.setup_venv
Expand Down
4 changes: 2 additions & 2 deletions .yamato/protobuf-generation-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@ test_linux_protobuf_generation:
- |
sudo apt-get update && sudo apt-get install -y nuget
eval "$($HOME/anaconda/bin/conda shell.bash hook)"
conda activate python3.8
conda activate python3.10
nuget install Grpc.Tools -Version $GRPC_VERSION -OutputDirectory protobuf-definitions/
python3 -m pip install --upgrade pip --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
python3 -m pip install grpcio==1.28.1 grpcio-tools==1.13.0 protobuf==3.11.3 six==1.14.0 mypy-protobuf==1.16.0 --progress-bar=off --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
python3 -m pip install grpcio==1.48.2 grpcio-tools==1.48.2 protobuf==3.19.6 six==1.16.0 mypy-protobuf==1.16.0 --progress-bar=off --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
pushd protobuf-definitions
chmod +x Grpc.Tools.$GRPC_VERSION/tools/linux_x64/protoc
chmod +x Grpc.Tools.$GRPC_VERSION/tools/linux_x64/grpc_csharp_plugin
Expand Down
2 changes: 1 addition & 1 deletion .yamato/pytest-gpu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ pytest_gpu:
commands:
- |
eval "$($HOME/anaconda/bin/conda shell.bash hook)"
conda activate python3.8
conda activate python3.10
python3 -m pip install pyyaml --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
python3 -u -m ml-agents.tests.yamato.setup_venv
python3 -m pip install --progress-bar=off -r test_requirements.txt --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
Expand Down
2 changes: 1 addition & 1 deletion .yamato/python-ll-api-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ test_linux_ll_api_{{ editor.version }}:
commands:
- |
eval "$($HOME/anaconda/bin/conda shell.bash hook)"
conda activate python3.8
conda activate python3.10
python -m pip install pyyaml --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
python -u -m ml-agents.tests.yamato.setup_venv
python ml-agents/tests/yamato/scripts/run_llapi.py
Expand Down
2 changes: 1 addition & 1 deletion .yamato/standalone-build-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ test_linux_standalone_{{ editor.version }}:
commands:
- |
eval "$($HOME/anaconda/bin/conda shell.bash hook)"
conda activate python3.8
conda activate python3.10
python3 -m pip install pyyaml --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
python3 -m pip install unity-downloader-cli --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple --upgrade
unity-downloader-cli -u {{ editor.version }} -c editor --wait --fast
Expand Down
2 changes: 1 addition & 1 deletion .yamato/standalone-build-webgl-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ test_webgl_standalone_{{ editor_version }}:
commands:
- |
eval "$($HOME/anaconda/bin/conda shell.bash hook)"
conda activate python3.8
conda activate python3.10
python -m pip install pyyaml --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
python -m pip install unity-downloader-cli --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple --upgrade
unity-downloader-cli -u {{ editor_version }} -c editor -c WebGL --wait --fast
Expand Down
2 changes: 1 addition & 1 deletion .yamato/training-int-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ test_linux_training_int_{{ editor.version }}:
commands:
- |
eval "$($HOME/anaconda/bin/conda shell.bash hook)"
conda activate python3.8
conda activate python3.10
python -m pip install pyyaml --index-url https://artifactory.prd.it.unity3d.com/artifactory/api/pypi/pypi/simple
python -u -m ml-agents.tests.yamato.training_int_tests
dependencies:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"references": [
"Unity.ML-Agents.Editor",
"Unity.ML-Agents",
"Unity.Barracuda",
"Unity.Sentis",
"Unity.ML-Agents.CommunicatorObjects",
"Unity.PerformanceTesting"
],
Expand Down
24 changes: 12 additions & 12 deletions DevProject/Packages/packages-lock.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,6 @@
},
"url": "https://artifactory.prd.cds.internal.unity3d.com/artifactory/api/npm/upm-candidates"
},
"com.unity.barracuda": {
"version": "3.0.0",
"depth": 1,
"source": "registry",
"dependencies": {
"com.unity.burst": "1.6.0",
"com.unity.modules.jsonserialize": "1.0.0",
"com.unity.modules.imageconversion": "1.0.0"
},
"url": "https://artifactory.prd.cds.internal.unity3d.com/artifactory/api/npm/upm-candidates"
},
"com.unity.burst": {
"version": "1.8.7",
"depth": 2,
Expand Down Expand Up @@ -66,7 +55,7 @@
"depth": 0,
"source": "local",
"dependencies": {
"com.unity.barracuda": "3.0.0",
"com.unity.sentis": "1.2.0-exp.2",
"com.unity.modules.imageconversion": "1.0.0",
"com.unity.modules.jsonserialize": "1.0.0",
"com.unity.modules.physics": "1.0.0"
Expand Down Expand Up @@ -115,6 +104,17 @@
},
"url": "https://artifactory.prd.cds.internal.unity3d.com/artifactory/api/npm/upm-candidates"
},
"com.unity.sentis": {
"version": "1.2.0-exp.2",
"depth": 1,
"source": "registry",
"dependencies": {
"com.unity.burst": "1.8.4",
"com.unity.modules.jsonserialize": "1.0.0",
"com.unity.modules.imageconversion": "1.0.0"
},
"url": "https://artifactory.prd.cds.internal.unity3d.com/artifactory/api/npm/upm-candidates"
},
"com.unity.settings-manager": {
"version": "2.0.1",
"depth": 1,
Expand Down
2 changes: 1 addition & 1 deletion DevProject/ProjectSettings/EditorBuildSettings.asset
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@ EditorBuildSettings:
path: Assets/ML-Agents/Scripts/Tests/Runtime/AcademyTest/AcademyStepperTestScene.unity
guid: 9bafc50b1e55b43b2b1ae9620f1f8311
m_configObjects:
com.unity.ml-agents.settings: {fileID: 11400000, guid: b412e5eb0b23f4bca86655ada9a633d4,
com.unity.ml-agents.settings: {fileID: 11400000, guid: e4ad406c716274fdfb58baafffa12dcd,
type: 2}
2 changes: 2 additions & 0 deletions Project/Assets/ML-Agents/Editor/DisableBurstFromMenu.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
#if UNITY_CLOUD_BUILD
using UnityEditor;

// TODO do we still need this?

public class DisableBurstFromMenu
{
/// This method is needed to disable Burst compilation on windows for our cloudbuild tests.
Expand Down
Loading
Loading