Skip to content

Commit

Permalink
Merge branch 'isl-org:main' into master
Browse files Browse the repository at this point in the history
  • Loading branch information
NobuoTsukamoto authored Dec 22, 2024
2 parents ef76d42 + 34d8039 commit c5bfce7
Show file tree
Hide file tree
Showing 1,851 changed files with 23,056 additions and 11,002 deletions.
2 changes: 1 addition & 1 deletion .clang-format
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ BasedOnStyle: Google
IndentWidth: 4
ColumnLimit: 80
UseTab: Never
Standard: c++14
Standard: c++17
ContinuationIndentWidth: 8
AccessModifierOffset: -4
BinPackParameters: false
Expand Down
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/0-build-issue-report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ body:
options:
- label: "I have searched for [similar issues](https://github.com/isl-org/Open3D/issues)."
required: true
- label: "For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip)."
- label: "For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip)."
required: true
- label: "I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch)."
- label: "I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch)."
required: true

- type: textarea
Expand Down
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/1-bug-report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ body:
options:
- label: "I have searched for [similar issues](https://github.com/isl-org/Open3D/issues)."
required: true
- label: "For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip)."
- label: "For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip)."
required: true
- label: "I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch)."
- label: "I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch)."
required: true

- type: textarea
Expand Down
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/2-questions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ body:
options:
- label: "I have searched for [similar issues](https://github.com/isl-org/Open3D/issues)."
required: true
- label: "For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip)."
- label: "For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip)."
required: true
- label: "I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch)."
- label: "I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch)."
required: true

- type: textarea
Expand Down
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/3-feature-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ body:
options:
- label: "I have searched for [similar issues](https://github.com/isl-org/Open3D/issues)."
required: true
- label: "For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip)."
- label: "For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip)."
required: true
- label: "I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch)."
- label: "I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch)."
required: true

- type: textarea
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
contact_links:
- name: Link to Open3D Documentation and Tutorials
url: http://www.open3d.org/docs/latest/
url: https://www.open3d.org/docs/latest/
about: "Read the documentation and tutorials for Open3D."
- name: Link to Open3D Discord Server
url: https://discord.gg/D35BGvn
Expand Down
22 changes: 11 additions & 11 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ actions updates. On macOS and Windows, using Ubuntu reduces CI cost.

- `.github/workflows/documentation.yml`: Github Actions workflow file to
create and deploy documentation. Documentation is created for every branch
as a CI test, but deployed only for `master`.
as a CI test, but deployed only for `main`.
- `util/ci_utils.sh:build_docs()`: Called by GitHub Actions to build documentation.
- `unpack_docs.sh`: Called by the documentation server to deploy the docs into
the website.
Expand Down Expand Up @@ -43,9 +43,9 @@ actions updates. On macOS and Windows, using Ubuntu reduces CI cost.
gsutil lifecycle set gcs.lifecycle.json gs://open3d-docs/
```

Objects will be stored in the bucket for one week. Currently, the
documentation server fetches the latest docs from `master` branch every hour.
If the documentation server fails to fetch the docs matching the `master`
Objects will be stored in the bucket for 30 days. Currently, the
documentation server fetches the latest docs from `main` branch every hour.
If the documentation server fails to fetch the docs matching the `main`
commit id, the last successfully fetched docs will be displayed.
3. Create service account
```bash
Expand Down Expand Up @@ -163,14 +163,14 @@ used for running CI.
- ARM64 cache (limit 1.5GB) is stored on Google cloud bucket
(`open3d-ci-cache` in the `isl-buckets` project). The bucket is world
readable, but needs the `open3d-ci-sa` service account for writing. Every
ARM64 build downloads the cache contents before build. Only `master` branch
ARM64 build downloads the cache contents before build. Only `main` branch
builds use `gsutil rsync` to update the cache in GCS. Cache transfer only
takes a few minutes, but reduces ARM64 CI time to about 1:15 hours.

## Development wheels and binary archives for user testing

`master` branch Python wheels and binary archives are uploaded to a world
readable GCS bucket in `open3d-releases-master/{python-wheels,devel}` for users
`main` branch Python wheels and binary archives are uploaded to a world
readable GCS bucket in `open3d-releases/{python-wheels,devel}` for users
to try out development wheels.

### Google Cloud storage
Expand All @@ -185,10 +185,10 @@ bucket with:
- One month (30 days) object lifecycle

```bash
gsutil mb -p open3d-dev -c STANDARD -l US -b on gs://open3d-releases-master
gsutil acl ch -u AllUsers:R gs://open3d-releases-master
gsutil lifecycle set gcs.lifecycle.json gs:/open3d-releases-master
gsutil mb -p open3d-dev -c STANDARD -l US -b on gs://open3d-releases
gsutil acl ch -u AllUsers:R gs://open3d-releases
gsutil lifecycle set gcs.lifecycle.json gs:/open3d-releases
gsutil iam ch \
serviceAccount:open3d-ci-sa-gpu@open3d-dev.iam.gserviceaccount.com:objectAdmin \
gs://open3d-releases-master
gs://open3d-releases
```
11 changes: 4 additions & 7 deletions .github/workflows/clean-gcloud-profiles.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,11 @@
# happens, run this workflow manually to clean up the login profiles.

name: Clean GCloud Profiles
permissions:
contents: read

on:
workflow_dispatch:
# push:
# branches:
# - master
# pull_request:
# types: [opened, reopened, synchronize]

env:
GCE_GPU_CI_SA: ${{ secrets.GCE_GPU_CI_SA }}
Expand All @@ -36,12 +33,12 @@ jobs:
fail-fast: false
steps:
- name: GCloud CLI auth
uses: 'google-github-actions/auth@v1'
uses: 'google-github-actions/auth@v2'
with:
project_id: ${{ secrets.GCE_PROJECT }}
credentials_json: '${{ secrets.GCE_SA_KEY_GPU_CI }}'
- name: GCloud CLI setup
uses: google-github-actions/setup-gcloud@v1
uses: google-github-actions/setup-gcloud@v2
with:
version: ${{ env.GCE_CLI_GHA_VERSION }}
project_id: ${{ secrets.GCE_PROJECT }}
Expand Down
83 changes: 28 additions & 55 deletions .github/workflows/documentation.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
name: Documentation
permissions: {}

on:
workflow_dispatch:
Expand All @@ -9,41 +10,39 @@ on:
default: 'ON'
push:
branches:
- master
- main
pull_request:
types: [opened, reopened, synchronize]

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

env:
GCE_CLI_GHA_VERSION: '416.0.0' # Fixed to avoid dependency on API changes

jobs:
headless-docs:
# Build headless and docs
headless-docs: # Build headless and docs
permissions:
contents: write # Artifact upload and release upload
runs-on: ubuntu-latest # Warn about build issues in new versions
env:
OPEN3D_ML_ROOT: ${{ github.workspace }}/Open3D-ML
DEVELOPER_BUILD: ${{ github.event.inputs.developer_build || 'ON' }}
steps:
- name: Checkout Open3D source code
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Maximize build space
run: |
source util/ci_utils.sh
maximize_ubuntu_github_actions_build_space
- name: Checkout Open3D-ML source code
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
repository: isl-org/Open3D-ML
path: ${{ env.OPEN3D_ML_ROOT }}

- name: Setup cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
# Ref: https://github.com/apache/incubator-mxnet/pull/18459/files
path: ~/.ccache
Expand All @@ -56,18 +55,18 @@ jobs:
restore-keys: |
${{ runner.os }}-ccache
- name: Set up Python version
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: '3.7'
python-version: '3.11'

- name: Install dependencies
env:
DEBIAN_FRONTEND: noninteractive
run: |
# the build system of the main repo expects a master branch. make sure
# master exists
# the build system of the main repo expects a main branch. make sure
# main exists
pushd "${OPEN3D_ML_ROOT}"
git checkout -b master || true
git checkout -b main || true
popd
source util/ci_utils.sh
install_docs_dependencies "${OPEN3D_ML_ROOT}"
Expand All @@ -78,51 +77,25 @@ jobs:
ccache -s
source util/ci_utils.sh
build_docs "$DEVELOPER_BUILD"
# PWD: Open3D/docs
ccache -s
tar_file="open3d-${GITHUB_SHA}-docs.tar.gz"
rm -rf "${GITHUB_WORKSPACE}/${tar_file}"
# Docs in docs/_out/html
tar -C _out -cvzf "${GITHUB_WORKSPACE}/${tar_file}" html
- name: Upload docs
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: open3d_docs
path: docs/_out/html
name: open3d-${{ github.sha }}-docs.tar.gz
path: open3d-${{ github.sha }}-docs.tar.gz
if-no-files-found: error
compression-level: 0 # no compression

- name: GCloud CLI auth
if: ${{ github.ref == 'refs/heads/master' }}
uses: 'google-github-actions/auth@v1'
with:
project_id: ${{ secrets.GCE_PROJECT }}
credentials_json: '${{ secrets.GCE_SA_KEY_GPU_CI }}'
- name: GCloud CLI setup
if: ${{ github.ref == 'refs/heads/master' }}
uses: google-github-actions/setup-gcloud@v1
with:
version: ${{ env.GCE_CLI_GHA_VERSION }}
project_id: ${{ secrets.GCE_PROJECT }}

- name: Deploy docs
if: ${{ github.ref == 'refs/heads/master' }}
run: |
# Compress and upload the docs, only for master branch
docs_out_dir="docs/_out" # Docs in ${docs_out_dir}/html
tar_file="${{ github.sha }}_ready.tar.gz"
rm -rf ${tar_file}
tar -C ${docs_out_dir} -czvf ${tar_file} html
gsutil cp ${tar_file} gs://open3d-docs/${tar_file}
echo "Docs archive uploaded to:"
echo "https://storage.googleapis.com/open3d-docs/${tar_file}"
- name: Check wheels and ready documentation archive
if: ${{ github.ref == 'refs/heads/master' }}
- name: Update devel release
if: ${{ github.ref == 'refs/heads/main' }}
env:
GH_TOKEN: ${{ github.token }}
run: |
if [ $(gsutil ls gs://open3d-docs/${{ github.sha }}_ready* | wc -l)\
-eq 4 ]; then
echo "All wheels and docs available. Making docs ready."
# Remove all marker files: Note _ at end of pattern.
gsutil rm gs://open3d-docs/${{ github.sha }}_ready_*
# Rename docs archive:
gsutil mv gs://open3d-docs/${{ github.sha }}_ready.tar.gz \
gs://open3d-docs/${{ github.sha }}.tar.gz
else
echo "All wheels / docs not available yet."
fi
gh release upload main-devel open3d-${{ github.sha }}-docs.tar.gz --clobber
gh release view main-devel
6 changes: 5 additions & 1 deletion .github/workflows/gcs.lifecycle.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@
"type": "Delete"
},
"condition": {
"age": 7
"age": 30,
"matchesPrefix": [
"devel/",
"python-wheels/"
]
}
}
]
Expand Down
Loading

0 comments on commit c5bfce7

Please sign in to comment.