Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Balanced batch sampler+base dataset #753

Merged
merged 67 commits into from
Aug 2, 2024
Merged

Balanced batch sampler+base dataset #753

merged 67 commits into from
Aug 2, 2024

Commits on Aug 24, 2023

  1. Update BalancedBatchSampler to use datasets' data_sizes method 

    Replace BalancedBatchSampler's `force_balancing` and `throw_on_error` parameters with `on_error`
    @nimashoghi
    nimashoghi committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    ae4add3 View commit details
    Browse the repository at this point in the history

  2. Remove python 3.10 syntax

    @nimashoghi
    nimashoghi committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    01fe2b4 View commit details
    Browse the repository at this point in the history

  3. Documentation

    @nimashoghi
    nimashoghi committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    2bf8213 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2023

  1. Added set_epoch method

    Nima Shoghi committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    7ba5b8a View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2023

  1. Format

    Nima Shoghi committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    a367d1e View commit details
    Browse the repository at this point in the history

  2. Changed "resolved dataset" message to be a debug log to reduce log spam

    Nima Shoghi committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    46e3c57 View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2024

  1. update with main

    @lbluque
    lbluque committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    845bce3 View commit details
    Browse the repository at this point in the history

  2. clean up batchsampler and tests

    @lbluque
    lbluque committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    86da069 View commit details
    Browse the repository at this point in the history

Commits on May 1, 2024

  1. base dataset class

    @lbluque
    lbluque committed May 1, 2024
    Configuration menu
    Copy the full SHA
    ff628dd View commit details
    Browse the repository at this point in the history

  2. move lin_ref to base dataset

    @lbluque
    lbluque committed May 1, 2024
    Configuration menu
    Copy the full SHA
    122197f View commit details
    Browse the repository at this point in the history

  3. inherit basedataset for ase dataset

    @lbluque
    lbluque committed May 1, 2024
    Configuration menu
    Copy the full SHA
    fb4ce16 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. filter indices prop

    @lbluque
    lbluque committed May 3, 2024
    Configuration menu
    Copy the full SHA
    c9e1759 View commit details
    Browse the repository at this point in the history

Commits on May 6, 2024

  1. added create_dataset fn

    @wood-b
    wood-b committed May 6, 2024
    Configuration menu
    Copy the full SHA
    95d3e6f View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. yaml load fix

    @lbluque
    lbluque committed May 7, 2024
    Configuration menu
    Copy the full SHA
    b6c640e View commit details
    Browse the repository at this point in the history

  2. create dataset function instead of filtering in base

    @lbluque
    lbluque committed May 7, 2024
    Configuration menu
    Copy the full SHA
    7fa1904 View commit details
    Browse the repository at this point in the history

  3. remove filtered_indices

    @lbluque
    lbluque committed May 7, 2024
    Configuration menu
    Copy the full SHA
    04c96bf View commit details
    Browse the repository at this point in the history

Commits on May 8, 2024

  1. make create_dataset and LMDBDatabase importable from datasets

    @lbluque
    lbluque committed May 8, 2024
    Configuration menu
    Copy the full SHA
    ea35b57 View commit details
    Browse the repository at this point in the history

  2. create_dataset cleanup

    @lbluque
    lbluque committed May 8, 2024
    Configuration menu
    Copy the full SHA
    dc98285 View commit details
    Browse the repository at this point in the history

  3. test create_dataset

    @lbluque
    lbluque committed May 8, 2024
    Configuration menu
    Copy the full SHA
    2339916 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. use metadata.natoms directly and add it to subset

    @lbluque
    lbluque committed May 10, 2024
    Configuration menu
    Copy the full SHA
    9b58cc7 View commit details
    Browse the repository at this point in the history

  2. use self.indices to handle shard

    @lbluque
    lbluque committed May 10, 2024
    Configuration menu
    Copy the full SHA
    63c03fc View commit details
    Browse the repository at this point in the history

Commits on May 14, 2024

  1. rename _data_sizes

    @lbluque
    lbluque committed May 14, 2024
    Configuration menu
    Copy the full SHA
    76322aa View commit details
    Browse the repository at this point in the history

Commits on May 15, 2024

  1. merge with main-legacy + no more data_sizes

    @lbluque
    lbluque committed May 15, 2024
    Configuration menu
    Copy the full SHA
    bb41b13 View commit details
    Browse the repository at this point in the history

  2. fix Subset of metadata

    @lbluque
    lbluque committed May 15, 2024
    Configuration menu
    Copy the full SHA
    b4e22bc View commit details
    Browse the repository at this point in the history

Commits on May 17, 2024

  1. minor change to metadata, added full path option

    @wood-b
    wood-b committed May 17, 2024
    Configuration menu
    Copy the full SHA
    29b6e68 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2024

  1. Merge branch 'main' into balanced-batch-sampler+base-dataset

    @wood-b
    wood-b committed May 18, 2024
    Configuration menu
    Copy the full SHA
    f9b15cd View commit details
    Browse the repository at this point in the history

  2. import updates

    @wood-b
    wood-b committed May 18, 2024
    Configuration menu
    Copy the full SHA
    dc59f96 View commit details
    Browse the repository at this point in the history

Commits on May 20, 2024

  1. implement get_metadata for datasets; add tests for max_atoms and bala… 

    …nced partitioning
    @misko
    misko committed May 20, 2024
    Configuration menu
    Copy the full SHA
    64b8df2 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. a[:len(a)+1] does not throw error, change to check for this

    @misko
    misko committed May 21, 2024
    Configuration menu
    Copy the full SHA
    80fea27 View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. off by one fix

    @misko
    misko committed May 22, 2024
    Configuration menu
    Copy the full SHA
    7447bdb View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2024

  1. Merge branch 'main' into balanced-batch-sampler+base-dataset

    @misko
    misko committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    b4f272d View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into balanced-batch-sampler+base-dataset

    @misko
    misko committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    9a4af0c View commit details
    Browse the repository at this point in the history

  3. fixing tests

    @misko
    misko committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    48e1d4f View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2024

  1. plug create_dataset into trainer

    @misko
    misko committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    70508c4 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. remove datasetwithsizes; fix base dataset integration; replace close_… 

    …db with __del__
    @misko
    misko committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    6b3c012 View commit details
    Browse the repository at this point in the history

  2. merge main

    @misko
    misko committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    5f89b7d View commit details
    Browse the repository at this point in the history

  3. lint

    @misko
    misko committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    c50f8a6 View commit details
    Browse the repository at this point in the history

  4. add/fix test;

    @misko
    misko committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    a641e9d View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. adding new notebook for using fairchem models with NEBs without CatTS… 

    …unami enumeration (#764)

* adding new notebook for using fairchem models with NEBs

* adding md tutorials

* blocking code cells that arent needed or take too long
    @brookwander
    brookwander authored Jul 16, 2024
    Configuration menu
    Copy the full SHA
    5743a59 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. Add extra test case for local batch size = 1

    @misko
    misko committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    ad6a20b View commit details
    Browse the repository at this point in the history

  2. fix example

    @misko
    misko committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    81f448b View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into balanced-batch-sampler+base-dataset

    @misko
    misko committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    6ce72fc View commit details
    Browse the repository at this point in the history

  4. fix test case

    @misko
    misko committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    bf0e9d8 View commit details
    Browse the repository at this point in the history

  5. reorg changes

    @misko
    misko committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    30f468d View commit details
    Browse the repository at this point in the history

  6. remove metadata_has_sizes in favor of basedataset function metadata_h… 

    …asattr
    @misko
    misko committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    9e6de95 View commit details
    Browse the repository at this point in the history

  7. fix data_parallel typo

    @misko
    misko committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    aa710bd View commit details
    Browse the repository at this point in the history

  8. fix up some tests

    @misko
    misko committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    78c03cc View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. Merge branch 'main' into balanced-batch-sampler+base-dataset

    @mshuaibii
    mshuaibii authored Jul 19, 2024
    Configuration menu
    Copy the full SHA
    fc74fa8 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into balanced-batch-sampler+base-dataset

    @misko
    misko authored Jul 19, 2024
    Configuration menu
    Copy the full SHA
    c032641 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. more general evaluator

    @mshuaibii @misko
    mshuaibii authored and misko committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    c73a885 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. rename get_metadata to sample_property_metadata

    @misko
    misko committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    c534968 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into balanced-batch-sampler+base-dataset

    @misko
    misko committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    65b84ec View commit details
    Browse the repository at this point in the history

  3. add slow get_metadata for ase; add tests for get_metadata (ase+lmdb);… 

    … add test for make lmdb metadata sizes
    @misko
    misko committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    0219e36 View commit details
    Browse the repository at this point in the history

  4. add support for different backends and ddp in pytest

    @misko
    misko committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    2014801 View commit details
    Browse the repository at this point in the history

  5. fix tests and balanced batch sampler

    @misko
    misko committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    fbfe627 View commit details
    Browse the repository at this point in the history

  6. make default dataset lmdb

    @misko
    misko committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    69b5492 View commit details
    Browse the repository at this point in the history

  7. lint

    @misko
    misko committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    2bb2a19 View commit details
    Browse the repository at this point in the history

  8. fix tests

    @misko
    misko committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    1a8fb90 View commit details
    Browse the repository at this point in the history

  9. test with world_size=0 by default

    @misko
    misko committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    43da6fc View commit details
    Browse the repository at this point in the history

  10. fix tests

    @misko
    misko committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    8508e2f View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. fix tests..

    @misko
    misko committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    4426870 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. remove subsample from oc22 dataset

    @misko
    misko committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    1e2e4aa View commit details
    Browse the repository at this point in the history

  2. remove old datasets; add test for noddp

    @misko
    misko committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    795689f View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. remove load balancing from docs

    @misko
    misko committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    57d86ab View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. fix docs; add train_split_settings and test for this

    @misko
    misko committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    e57d701 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into balanced-batch-sampler+base-dataset

    @misko
    misko authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    79f4132 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. Merge branch 'main' into balanced-batch-sampler+base-dataset

    @misko
    misko authored Aug 2, 2024
    Configuration menu
    Copy the full SHA
    57d668d View commit details
    Browse the repository at this point in the history