Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

torch20 opt #89

Closed
wants to merge 355 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
355 commits
Select commit Hold shift + click to select a range
1a5b9a6
fix deleted method
jgrss Apr 21, 2024
8e5be6a
format
jgrss Apr 21, 2024
e0d422e
format
jgrss Apr 21, 2024
0ab0a2a
deprecated scipy
jgrss Apr 21, 2024
21d9c5d
add None check
jgrss Apr 21, 2024
f3aa3bd
update Dataset
jgrss Apr 21, 2024
38823d0
add registers for collate
jgrss Apr 21, 2024
6f66547
minor formatting
jgrss Apr 21, 2024
20de135
remove model tensor reshaping
jgrss Apr 21, 2024
014f381
rename args
jgrss Apr 21, 2024
f7ae086
network name changes
jgrss Apr 21, 2024
360308f
name changes
jgrss Apr 21, 2024
159f46b
add keyword arg
jgrss Apr 21, 2024
573c2b2
move normalization method
jgrss Apr 21, 2024
b3ea91c
update tests
jgrss Apr 21, 2024
8aabd8a
convert to tensors
jgrss Apr 21, 2024
710edf9
update README
jgrss Apr 21, 2024
07020bc
add isort
Apr 23, 2024
83bbc97
update dependencies
Apr 23, 2024
8f020db
upgrade version
Apr 23, 2024
083673b
format
Apr 23, 2024
09c9c6e
format
Apr 23, 2024
2e1cb6e
format
Apr 23, 2024
6474465
remove file
Apr 23, 2024
592f4c2
rename file
Apr 23, 2024
13a88e8
format
Apr 23, 2024
e2bccdd
format
Apr 23, 2024
a53bb73
add collater
Apr 23, 2024
df8d9e2
format
Apr 23, 2024
6960eea
format
Apr 23, 2024
a100543
format
Apr 23, 2024
83d6029
format
Apr 23, 2024
1fa52f3
format
Apr 23, 2024
9494383
format
Apr 23, 2024
cd5abf8
use nn.
Apr 23, 2024
785f3ba
format
Apr 23, 2024
843b121
format
Apr 23, 2024
37e2cd1
format
Apr 23, 2024
952ef20
format
Apr 23, 2024
4f54446
format
jgrss Apr 20, 2024
891412c
format
jgrss Apr 25, 2024
89162c5
update loss
jgrss Apr 25, 2024
ca38957
fix dataset test
jgrss Apr 25, 2024
e82a045
move to modules
jgrss Apr 25, 2024
6df22f1
pin changes
jgrss Apr 26, 2024
0d4cea9
stash
jgrss Apr 26, 2024
31714ad
run train session
jgrss Apr 26, 2024
5971d33
fix train data storage
jgrss Apr 29, 2024
bd56e2b
add enum
jgrss Apr 29, 2024
7e3bdf2
change default transforms to None
jgrss Apr 29, 2024
e1052ad
change time series start/end get method
jgrss Apr 29, 2024
6389d73
update script flow
jgrss Apr 29, 2024
76bac65
add tower
jgrss Apr 30, 2024
e71abb7
add worker check
jgrss Apr 30, 2024
a61d6af
Merge branch 'jgrss/transfer_ltae' of github.com:jgrss/cultionet into…
jgrss Apr 30, 2024
ef2afc2
implement
jgrss Apr 30, 2024
0f2bcd5
start on predict data
jgrss Apr 30, 2024
cb27948
fix inference steps
jgrss May 1, 2024
ae842dc
replace deprecated
jgrss May 1, 2024
1b47fbc
fix predict window
jgrss May 1, 2024
f85607e
cleanup
jgrss May 2, 2024
a2069c4
set defaults
jgrss May 2, 2024
8156697
comment
jgrss May 2, 2024
d6a4583
comment
jgrss May 2, 2024
14a6b74
changes
jgrss May 2, 2024
9e640e6
add foj loss
jgrss May 2, 2024
dd08d4c
add foj loss
jgrss May 2, 2024
024bd44
fix dropout
jgrss May 3, 2024
485eb80
cleanup params
jgrss May 3, 2024
0c327f6
cleanup transfer
jgrss May 3, 2024
16009c8
cleanup args
jgrss May 3, 2024
ea4141d
fix deep supervision
jgrss May 3, 2024
ffb4beb
fix deep supervision
jgrss May 3, 2024
337af1e
need to update the spatial gpkg
jgrss May 3, 2024
24bb1ca
update dependencies
jgrss May 9, 2024
93318c1
add 0 check
jgrss May 9, 2024
473973a
add multipolygon method
jgrss May 9, 2024
737bc02
add pool_first arg
jgrss May 9, 2024
6fbcf2b
add pool_first arg
jgrss May 9, 2024
90a39d2
add pool_first arg
jgrss May 9, 2024
95a0db7
add pool_first arg
jgrss May 9, 2024
97598f7
add pool_first arg
jgrss May 9, 2024
d35edaf
remove args
jgrss May 9, 2024
7bba8f9
script changes
jgrss May 9, 2024
cb9f430
add optional StdConv
jgrss May 9, 2024
984224d
add pyogrio
jgrss May 10, 2024
544d51b
replace gw polygon_to_array
jgrss May 10, 2024
7109ecd
add CLI arg for unknown labels
jgrss May 10, 2024
570a869
make read method
jgrss May 10, 2024
cfe4df0
update progress bars
jgrss May 11, 2024
49a8b2b
fix augmentations
jgrss May 11, 2024
55eef3e
fix augmentations
jgrss May 11, 2024
a81d3bd
fix augmentation bdist
jgrss May 11, 2024
4ad255e
add args
jgrss May 15, 2024
573d8b0
add args
jgrss May 15, 2024
8692d4f
transfer updates
jgrss May 15, 2024
e637cd0
move methods
jgrss May 15, 2024
cf617e4
move scale
jgrss May 15, 2024
4960be4
pass conv args
jgrss May 15, 2024
29ea488
move methods
jgrss May 15, 2024
b7c410e
move data methods
jgrss May 15, 2024
442707d
add module import
jgrss May 15, 2024
4ef8c3c
modfy CLI defaults
jgrss May 15, 2024
52e420b
update create methods
jgrss May 15, 2024
ba5a33f
replace tqdm with rich
jgrss May 15, 2024
236ce9f
add batch file store
jgrss May 15, 2024
aa07124
remove old code
jgrss May 15, 2024
76be1e8
support weak supervision
jgrss May 16, 2024
db626a6
fix tests
jgrss May 16, 2024
46c6a29
test mask-rcnn
jgrss May 17, 2024
64cfb3a
stash
jgrss May 21, 2024
cdbf67e
change conv args
jgrss May 22, 2024
fa3935a
change conv args
jgrss May 22, 2024
fcff273
test zoom and support score masking
jgrss May 22, 2024
975695f
modify tower unet
jgrss May 22, 2024
a94288c
change arg name
jgrss May 22, 2024
06edfcf
cleanup unused and add order option
jgrss May 22, 2024
d7a0227
cleanup unused and add order option
jgrss May 22, 2024
5239d15
change CLI
jgrss May 22, 2024
173486b
update CLI args
jgrss May 22, 2024
703c7b6
update test
jgrss May 22, 2024
11b6fce
change module names
jgrss May 22, 2024
2184241
stash
jgrss May 23, 2024
0e48cd0
stash
jgrss May 24, 2024
2910407
add temp exception
jgrss May 26, 2024
03faca4
reconfigure resconv
jgrss May 26, 2024
96eb706
remove files
jgrss May 26, 2024
0811460
modify transfer model
jgrss May 28, 2024
9ac0901
fix mask scoring
jgrss May 28, 2024
c98ff56
use transfer name
jgrss May 28, 2024
5168d40
separate spatial splits
jgrss May 28, 2024
a1eab4d
Merge branch 'jgrss/transfer_ltae_v2' of github.com:jgrss/cultionet i…
jgrss May 28, 2024
7a149db
add missing method
jgrss May 29, 2024
bd37721
transfer loss
jgrss May 29, 2024
431d5b4
all_touched=True
jgrss May 29, 2024
e2a2def
add data methods
jgrss May 29, 2024
3d3967d
add edge check
jgrss May 29, 2024
20ecca0
Merge branch 'jgrss/transfer_ltae_v2' of ssh://github.com/jgrss/culti…
jgrss May 29, 2024
68450ca
add transfer arg
jgrss May 29, 2024
49ddc27
finetune
jgrss May 29, 2024
51a8736
re-test params
jgrss May 29, 2024
4c85428
get utm zone from lat/lon
jgrss Jun 10, 2024
9f14823
change time scaler param
jgrss Jun 10, 2024
e2a50ba
pass activation param
jgrss Jun 10, 2024
799f137
experimental final layers
jgrss Jun 10, 2024
03b82fe
cleanuup refinee
jgrss Jun 14, 2024
f6ce02e
start/end
jgrss Jun 14, 2024
4848550
enums
jgrss Jun 14, 2024
36f9389
enums
jgrss Jun 14, 2024
2703fd1
topology loss
jgrss Jun 14, 2024
62b4111
cleanup
jgrss Jun 14, 2024
89ef753
test attention
jgrss Jun 14, 2024
0383ade
vit
jgrss Jun 14, 2024
a1e2994
residual in residual
jgrss Jun 14, 2024
02205dd
attention
jgrss Jun 14, 2024
f1e83f1
remove CLI option
jgrss Jun 14, 2024
e32229e
predict start/end
jgrss Jun 14, 2024
e25840d
temp test
jgrss Jun 14, 2024
e4ddef3
libraries
jgrss Jun 14, 2024
771ebe0
reconfigure augmentations
jgrss Jun 20, 2024
d172ff9
random rng support
jgrss Jun 26, 2024
6b4755e
simplify augmenter
jgrss Jun 26, 2024
dae9e53
add data check for geometry collection
jgrss Jun 26, 2024
fcb91b7
augment probability handling
jgrss Jun 26, 2024
1930a33
force new spatial data for transfer model
jgrss Jun 26, 2024
78339d3
formatting
jgrss Jun 26, 2024
c2f278b
test new loss
jgrss Jun 26, 2024
ba5fc40
move modules
jgrss Jun 26, 2024
98622f5
move modules
jgrss Jun 26, 2024
7f46354
move modules
jgrss Jun 26, 2024
9a17906
profiler
jgrss Jun 26, 2024
99fb167
module imports
jgrss Jun 26, 2024
7841fcb
docs: update README (#75)
jgrss Jun 26, 2024
d44c0a9
feat: transfer ltae v2 (#76)
jgrss Jul 30, 2024
4b34206
fix: torch20 tests (#77)
jgrss Jul 31, 2024
8225c2c
fix: remove instances of torch_geometric (#78)
jgrss Jul 31, 2024
9b7e5d3
fix: test batch save step (#80)
jgrss Jul 31, 2024
0e966be
docs: torch20 docs (#79)
jgrss Jul 31, 2024
42f90e5
fix: torch20 cleanup (#81)
jgrss Jul 31, 2024
9853319
fix: add fiona dependency (#82)
jgrss Jul 31, 2024
c685fa5
fix: use cpu device (#83)
jgrss Jul 31, 2024
85dc5d0
🐛 add supported attention
jgrss Jul 31, 2024
2550968
🐛 revert attention
jgrss Jul 31, 2024
880640f
🐛✅ attention tests
jgrss Jul 31, 2024
72041dc
🙈 hide test for now
jgrss Jul 31, 2024
a22f468
fix: torch20 train test (#84)
jgrss Jul 31, 2024
7807c13
✅ test other weights
jgrss Jul 31, 2024
1555b27
🙈 block train CLI test
jgrss Aug 1, 2024
e6dc57d
fix: torch20 network c (#85)
jgrss Aug 1, 2024
d8bcfb2
torch20 transfer (#86)
jgrss Aug 14, 2024
6ef9b9c
fix: torch20 ex (#87)
jgrss Aug 15, 2024
e25241a
➖ cleanup imports
jgrss Aug 15, 2024
8576928
fix: ➖ remove boundary loss (#88)
jgrss Aug 15, 2024
f3fdaea
➖ remove topology loss
jgrss Aug 15, 2024
37aa5db
📝 add comments for prior change
jgrss Aug 27, 2024
a2bc209
⚡️ change default loss
jgrss Aug 27, 2024
c86dbcd
🎨
jgrss Sep 16, 2024
8d1ee0c
⚡️ add CLI data-pattern; remove args
jgrss Sep 16, 2024
18fae6b
🎨
jgrss Sep 16, 2024
e0206ec
🔥 cleanup unused methods
jgrss Sep 16, 2024
80027e7
📝 new relative imports
jgrss Sep 16, 2024
a2f358b
⚡️ change defaults
jgrss Sep 16, 2024
68f5c50
🎨
jgrss Sep 16, 2024
21a6a8a
🎨
jgrss Sep 16, 2024
704c22a
⚡️ import loss
jgrss Sep 16, 2024
19cd269
🎨
jgrss Sep 16, 2024
59f3e72
🎨
jgrss Sep 16, 2024
aa068c4
🎨
jgrss Sep 16, 2024
6354b2a
🎨
jgrss Sep 16, 2024
20b6754
✅ tests
jgrss Sep 16, 2024
3f975e6
🎨
jgrss Sep 16, 2024
bbae455
🎨
jgrss Sep 16, 2024
675cce6
📝 change var name
jgrss Sep 16, 2024
3069b20
📝 change var name
jgrss Sep 16, 2024
36c4840
change to class
jgrss Sep 16, 2024
986240e
⚡️ predict callback return 3 bands
jgrss Sep 20, 2024
23ec121
🔥 remove unused import
jgrss Sep 20, 2024
2629794
🎨
jgrss Sep 20, 2024
2f34ab3
⚡️ relative loss imports
jgrss Sep 20, 2024
cb93f66
⚡️ update enums
jgrss Sep 20, 2024
319b14f
🔥 remove arg in predict method
jgrss Sep 20, 2024
7eb8bf8
🔥 remove temporal transformer
jgrss Sep 20, 2024
f68de85
🎨
jgrss Sep 20, 2024
a833945
⚡️ update main model network
jgrss Sep 20, 2024
0a84d0e
⚡️ ensure grad
jgrss Sep 20, 2024
08e4afc
🎨
jgrss Sep 20, 2024
e3d40fc
🎨
jgrss Sep 20, 2024
6545953
🎨
jgrss Sep 20, 2024
2e3b13c
🔥 remove file
jgrss Sep 20, 2024
5b67097
✨ new relative import
jgrss Sep 20, 2024
c7c18ba
🔥 remove imports
jgrss Sep 20, 2024
d042424
🎨
jgrss Sep 20, 2024
55e56e2
🔥 remove file
jgrss Sep 22, 2024
9412d4c
⚡️ tests
jgrss Sep 22, 2024
36a6db4
⚡️ scripts
jgrss Sep 22, 2024
974a8d7
🔥 remove keyword arg
jgrss Sep 22, 2024
2dd4a9d
⚡️ add keywrod arg
jgrss Sep 22, 2024
8198cac
⚡️ comment
jgrss Sep 22, 2024
33ae8bd
🎨
jgrss Sep 22, 2024
f8861d4
🎨
jgrss Sep 22, 2024
179327e
🎨
jgrss Sep 22, 2024
ced889a
🎨
jgrss Sep 22, 2024
a69fddc
🔥 remove code
jgrss Sep 22, 2024
229d495
🎨
jgrss Sep 22, 2024
c9095ad
🔥 remove imports
jgrss Sep 22, 2024
9783745
🚑️ comment
jgrss Sep 22, 2024
8b231ce
🎨
jgrss Sep 23, 2024
a8c7411
🔥 remove unused modules
jgrss Sep 23, 2024
e9fed24
🎨
jgrss Sep 23, 2024
853d029
🎨
jgrss Sep 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 16 additions & 66 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,16 @@ on:
jobs:
Tests:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9", "3.10"]
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- uses: actions/checkout@v3
- name: Setup Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: 3.8.12
- uses: syphar/restore-virtualenv@v1
id: cnetenv
with:
requirement_files: setup.cfg
- uses: syphar/restore-pip-download-cache@v1
if: steps.cnetenv.outputs.cache-hit != 'true'
python-version: ${{ matrix.python-version }}
cache: pip # caching pip dependencies based on changes to pyproject.toml
- name: Install GDAL binaries
run: |
# Temporary? dpkg fix: https://askubuntu.com/questions/1276111/error-upgrading-grub-efi-amd64-signed-special-device-old-ssd-does-not-exist
Expand All @@ -39,19 +38,19 @@ jobs:
- name: Install Python packages
run: |
# Install Python GDAL
pip install -U pip setuptools wheel
pip install -U Cython "numpy<=1.21.0"
pip install -U pip
pip install -U setuptools wheel
pip install -U numpy==1.24.4
pip install setuptools==57.5.0
GDAL_VERSION=$(gdal-config --version | awk -F'[.]' '{print $1"."$2"."$3}')
pip install GDAL==$GDAL_VERSION --no-binary=gdal
- name: Install PyTorch
run: |
TORCH_CPU="https://download.pytorch.org/whl/cpu"
TORCH_VERSION="1.13.0"
pip install --upgrade --no-cache-dir setuptools>=0.59.5
pip install torch==${TORCH_VERSION} torchvision torchaudio --extra-index-url $TORCH_CPU
PYG_TORCH_CPU="https://data.pyg.org/whl/torch-${TORCH_VERSION}+cpu.html"
pip install torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric -f $PYG_TORCH_CPU
if: steps.cnetenv.outputs.cache-hit != 'true'
TORCH_VERSION="2.2.2"
pip install -U --no-cache-dir setuptools>=65.5.1
pip install torch==${TORCH_VERSION} torchvision==0.17.2 torchaudio==${TORCH_VERSION} --extra-index-url $TORCH_CPU
pip install natten==0.17.1+torch220cpu -f https://shi-labs.com/natten/wheels
- name: Install cultionet
run: |
pip install .
Expand All @@ -60,52 +59,3 @@ jobs:
pip install pytest
cd tests/
python -m pytest

# Version:
# needs: Tests
# if: github.event_name == 'push' && github.ref == 'refs/heads/main' && !contains(github.event.head_commit.message, 'chore(release):')
# runs-on: ubuntu-latest
# concurrency: release
# steps:
# - uses: actions/checkout@v2
# with:
# fetch-depth: 0
# token: ${{ secrets.CULTIONET_TOKEN }}
# - uses: actions/setup-python@v2
# with:
# python-version: 3.8.12
# - name: Python Semantic Release
# run: |
# python -m pip install python-semantic-release
# # Add credentials
# git config user.name "github-actions"
# git config user.email "github-actions@github.com"
# # Bump cultionet version
# semantic-release publish
# env:
# GH_TOKEN: ${{ secrets.CULTIONET_TOKEN }}

# # https://github.com/fnkr/github-action-ghr
# Release:
# needs: Version
# if: github.event_name == 'push' && github.ref == 'refs/heads/main' && !contains(github.event.head_commit.message, 'chore(release):')
# name: Upload release
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v2
# - uses: actions/setup-python@v2
# with:
# python-version: 3.8.12
# - uses: syphar/restore-virtualenv@v1
# id: cnetenv
# with:
# requirement_files: setup.cfg
# - uses: syphar/restore-pip-download-cache@v1
# if: steps.cnetenv.outputs.cache-hit != 'true'
# - name: Checkout
# uses: fnkr/github-action-ghr@v1
# if: startsWith(github.ref, 'refs/tags/')
# env:
# GHR_PATH: .
# GHR_COMPRESS: gz
# GITHUB_TOKEN: ${{ secrets.CULTIONET_TOKEN }}
10 changes: 10 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@ repos:
rev: 22.3.0
hooks:
- id: black
exclude: src/cultionet/utils/stats.py
language_version: python3
args: [--skip-string-normalization]
- repo: https://github.com/myint/docformatter
rev: v1.4
hooks:
- id: docformatter
exclude: src/cultionet/utils/stats.py
args: [
--in-place,
--wrap-summaries,
Expand All @@ -20,3 +22,11 @@ repos:
rev: 6.0.0
hooks:
- id: flake8
exclude: src/cultionet/utils/stats.py
- repo: https://github.com/pycqa/isort
rev: 5.11.5
hooks:
- id: isort
exclude: src/cultionet/utils/stats.py
name: isort (python)
args: [--settings-path=pyproject.toml]
Loading
Loading