Skip to content
This repository has been archived by the owner on Sep 20, 2024. It is now read-only.

Refactor Integrate Asset #3530

Merged
merged 100 commits into from
Jul 22, 2022
Merged

Refactor Integrate Asset #3530

merged 100 commits into from
Jul 22, 2022

Commits on Mar 16, 2022

  1. First draft pass of refactoring the Integrator

    @BigRoy
    BigRoy committed Mar 16, 2022
    Configuration menu
    Copy the full SHA
    d88ed91 View commit details
    Browse the repository at this point in the history

Commits on Mar 17, 2022

  1. More refactoring + draft (untested) implementation for separating Fil… 

    …e Transaction logic
    @BigRoy
    BigRoy committed Mar 17, 2022
    Configuration menu
    Copy the full SHA
    ae1a9ff View commit details
    Browse the repository at this point in the history

  2. Fix hound

    @BigRoy
    BigRoy committed Mar 17, 2022
    Configuration menu
    Copy the full SHA
    9f6cc5d View commit details
    Browse the repository at this point in the history

  3. Continue refactor, restore functionality - now can correctly publish … 

    …as before (rudimentary tested only)
    @BigRoy
    BigRoy committed Mar 17, 2022
    Configuration menu
    Copy the full SHA
    56bcd8c View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2022

  1. Merge remote-tracking branch 'upstream/develop' into refactor_integrator 

    # Conflicts:
#	openpype/plugins/publish/integrate_new.py
    @BigRoy
    BigRoy committed Mar 23, 2022
    Configuration menu
    Copy the full SHA
    4c03092 View commit details
    Browse the repository at this point in the history

  2. Reduce duplicated logic by implementing resolve_profile method

    @BigRoy
    BigRoy committed Mar 23, 2022
    Configuration menu
    Copy the full SHA
    8996280 View commit details
    Browse the repository at this point in the history

  3. Remove prepare anatomy data logic that is already collected/generated… 

    … in CollectAnatomyContextData and CollectAnatomyInstanceData.

This currently was duplicated logic and should not be handled in the Integrator
    @BigRoy
    BigRoy committed Mar 23, 2022
    Configuration menu
    Copy the full SHA
    177e244 View commit details
    Browse the repository at this point in the history

  4. Move logic to clarify what should be removed/moved and bring logic cl… 

    …oser to where it's used
    @BigRoy
    BigRoy committed Mar 23, 2022
    Configuration menu
    Copy the full SHA
    3fd2d02 View commit details
    Browse the repository at this point in the history

  5. Simplify profile filtering

    @BigRoy
    BigRoy committed Mar 23, 2022
    Configuration menu
    Copy the full SHA
    8edfb3f View commit details
    Browse the repository at this point in the history

  6. Re-use get families logic

    @BigRoy
    BigRoy committed Mar 23, 2022
    Configuration menu
    Copy the full SHA
    79286ea View commit details
    Browse the repository at this point in the history

  7. Remove todo since assetEntity already comes from Collectors + re-use … 

    …families variable
    @BigRoy
    BigRoy committed Mar 23, 2022
    Configuration menu
    Copy the full SHA
    d6c6827 View commit details
    Browse the repository at this point in the history

  8. Add todo to move get subset group logic

    @BigRoy
    BigRoy committed Mar 23, 2022
    Configuration menu
    Copy the full SHA
    47259f8 View commit details
    Browse the repository at this point in the history

Commits on Mar 24, 2022

  1. Override stored repre context udim for backwards compatibility

    @BigRoy
    BigRoy committed Mar 24, 2022
    Configuration menu
    Copy the full SHA
    b128e0a View commit details
    Browse the repository at this point in the history

  2. Encapsulate version data completely into its own function

    @BigRoy
    BigRoy committed Mar 24, 2022
    Configuration menu
    Copy the full SHA
    9997acb View commit details
    Browse the repository at this point in the history

  3. Move logic closer to where it's used

    @BigRoy
    BigRoy committed Mar 24, 2022
    Configuration menu
    Copy the full SHA
    5b1f6eb View commit details
    Browse the repository at this point in the history

  4. Preparation to delay Version document write to database closer to rep… 

    …resentation write
    @BigRoy
    BigRoy committed Mar 24, 2022
    Configuration menu
    Copy the full SHA
    3369c15 View commit details
    Browse the repository at this point in the history

  5. Fix get_profile_filter_criteria anatomy data key for app name

    @BigRoy
    BigRoy committed Mar 24, 2022
    Configuration menu
    Copy the full SHA
    42175ff View commit details
    Browse the repository at this point in the history

  6. Fix sequence functionality

    @BigRoy
    BigRoy committed Mar 24, 2022
    Configuration menu
    Copy the full SHA
    7713af5 View commit details
    Browse the repository at this point in the history

  7. Reformat code

    @BigRoy
    BigRoy committed Mar 24, 2022
    Configuration menu
    Copy the full SHA
    229626b View commit details
    Browse the repository at this point in the history

  8. Reduce database calls for register subset + prepare for bulk writes l… 

    …ogic
    @BigRoy
    BigRoy committed Mar 24, 2022
    Configuration menu
    Copy the full SHA
    e1eb088 View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2022

  1. Separate site sync logic further from Integrator plug-in (Draft)

    @BigRoy
    BigRoy committed Mar 25, 2022
    Configuration menu
    Copy the full SHA
    b906365 View commit details
    Browse the repository at this point in the history

Commits on Mar 26, 2022

  1. Move FileTransaction into lib

    @BigRoy
    BigRoy committed Mar 26, 2022
    Configuration menu
    Copy the full SHA
    e0aaa5f View commit details
    Browse the repository at this point in the history

  2. Collect subset group in a Collector instead of during Integrator

    @BigRoy
    BigRoy committed Mar 26, 2022
    Configuration menu
    Copy the full SHA
    d3cb32e View commit details
    Browse the repository at this point in the history

  3. Remove duplicate "source" in families

    @BigRoy
    BigRoy committed Mar 26, 2022
    Configuration menu
    Copy the full SHA
    d7c5ad1 View commit details
    Browse the repository at this point in the history

  4. Move remainder of prepare anatomy data to the Collector

    @BigRoy
    BigRoy committed Mar 26, 2022
    Configuration menu
    Copy the full SHA
    8fffc60 View commit details
    Browse the repository at this point in the history

  5. Restore "published_path" backwards compatibility for IntegrateFtrackI… 

    …nstance on Farm
    @BigRoy
    BigRoy committed Mar 26, 2022
    Configuration menu
    Copy the full SHA
    177e83e View commit details
    Browse the repository at this point in the history

  6. Use os.path.abspath instead of os.path.normpath when adding a tra… 

    …nsfer
    @BigRoy
    BigRoy committed Mar 26, 2022
    Configuration menu
    Copy the full SHA
    7189954 View commit details
    Browse the repository at this point in the history

  7. Move Subset Grouping Profiles settings to Collect Subset Group 

    - This is moved from the Integrate Asset New settings
    @BigRoy
    BigRoy committed Mar 26, 2022
    Configuration menu
    Copy the full SHA
    8f8b578 View commit details
    Browse the repository at this point in the history

  8. Separate get_template_name into its own method + use `self.default_te… 

    …mplate_name`
    @BigRoy
    BigRoy committed Mar 26, 2022
    Configuration menu
    Copy the full SHA
    6ff7167 View commit details
    Browse the repository at this point in the history

  9. Match comment from Integrator for consistency

    @BigRoy
    BigRoy committed Mar 26, 2022
    Configuration menu
    Copy the full SHA
    821293d View commit details
    Browse the repository at this point in the history

  10. Debug log when exclude family was found for the instance

    @BigRoy
    BigRoy committed Mar 26, 2022
    Configuration menu
    Copy the full SHA
    c3e0162 View commit details
    Browse the repository at this point in the history

  11. Perform database registering of Subset and Version in a single Bulk W… 

    …rite
    @BigRoy
    BigRoy committed Mar 26, 2022
    Configuration menu
    Copy the full SHA
    fbdb385 View commit details
    Browse the repository at this point in the history

  12. Match assertion for collection of files (allow no absolute paths) sim… 

    …ilar to single files
    @BigRoy
    BigRoy committed Mar 26, 2022
    Configuration menu
    Copy the full SHA
    1844281 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2022

  1. Also Bulk Write representation changes + more cleanup 

    - Don't create intermediate archived representations
- Move writing of Subset + Version to database to just before file transactions
- Perform ReplaceOne for version instead of update with "$set" for the full version
    @BigRoy
    BigRoy committed Mar 28, 2022
    Configuration menu
    Copy the full SHA
    8e0161b View commit details
    Browse the repository at this point in the history

  2. Fix class type

    @BigRoy
    BigRoy committed Mar 28, 2022
    Configuration menu
    Copy the full SHA
    ba2c6e6 View commit details
    Browse the repository at this point in the history

  3. Restructure code and more cleanup

    @BigRoy
    BigRoy committed Mar 28, 2022
    Configuration menu
    Copy the full SHA
    e6665e5 View commit details
    Browse the repository at this point in the history

  4. Rely on instance.data["fps"] over context.data["fps"]` if available

    @BigRoy
    BigRoy committed Mar 28, 2022
    Configuration menu
    Copy the full SHA
    2777c36 View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2022

  1. Fix message

    @BigRoy
    BigRoy committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    add4958 View commit details
    Browse the repository at this point in the history

  2. Fix message

    @BigRoy
    BigRoy committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    77b5c24 View commit details
    Browse the repository at this point in the history

  3. Streamlining some code, optimize some database queries with projection

    @BigRoy
    BigRoy committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    127f198 View commit details
    Browse the repository at this point in the history

  4. Unify usage of clique.assemble

    @BigRoy
    BigRoy committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    0c2c60d View commit details
    Browse the repository at this point in the history

  5. Organize single file code more like sequence file code

    @BigRoy
    BigRoy committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    44d6199 View commit details
    Browse the repository at this point in the history

  6. Cleanup get_files_info docstring

    @BigRoy
    BigRoy committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    a2a77b8 View commit details
    Browse the repository at this point in the history

  7. Capture edge case where all "representations" are tagged for delete

    @BigRoy
    BigRoy committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    6fe6841 View commit details
    Browse the repository at this point in the history

  8. Improve docstring

    @BigRoy
    BigRoy committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    a7a908d View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2022

  1. Merge branch 'develop' of https://github.com/pypeclub/OpenPype into r… 

    …efactor_integrator
    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    e37b9f0 View commit details
    Browse the repository at this point in the history

  2. Only add frame to context if used by the destination template

    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    3ec9684 View commit details
    Browse the repository at this point in the history

  3. Remove double entry of "task"

    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    6733df7 View commit details
    Browse the repository at this point in the history

  4. Add comment

    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    c95c9f9 View commit details
    Browse the repository at this point in the history

  5. Explain why we write subset+version first

    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    65691bf View commit details
    Browse the repository at this point in the history

  6. Add to do for potential erroneous case

    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    0d83f3c View commit details
    Browse the repository at this point in the history

  7. Also include file infos of resource files like textures into each rep… 

    …resentation

- This should fix Site Sync for lookdev textures, etc.
    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    89376a9 View commit details
    Browse the repository at this point in the history

  8. Match behavior more with what integrator did before refactor

    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    e620955 View commit details
    Browse the repository at this point in the history

  9. Add todo/question regarding intent

    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    52fd21d View commit details
    Browse the repository at this point in the history

  10. Add todo

    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    4c78976 View commit details
    Browse the repository at this point in the history

  11. Use template name for frame padding anatomy template

    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    3e095bc View commit details
    Browse the repository at this point in the history

  12. Never shift udim sequences

    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    b12b1c8 View commit details
    Browse the repository at this point in the history

  13. add todo/question

    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    f7d35c4 View commit details
    Browse the repository at this point in the history

  14. Remove old "dependencies" data

    @BigRoy
    BigRoy committed Apr 2, 2022
    Configuration menu
    Copy the full SHA
    70bfdd0 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2022

  1. Improve clarity of comment

    @BigRoy
    BigRoy committed Apr 3, 2022
    Configuration menu
    Copy the full SHA
    45745cc View commit details
    Browse the repository at this point in the history

  2. Add comment

    @BigRoy
    BigRoy committed Apr 3, 2022
    Configuration menu
    Copy the full SHA
    fe72197 View commit details
    Browse the repository at this point in the history

  3. tweak comment

    @BigRoy
    BigRoy committed Apr 3, 2022
    Configuration menu
    Copy the full SHA
    c3c8281 View commit details
    Browse the repository at this point in the history

Commits on Apr 7, 2022

  1. Merge remote-tracking branch 'upstream/develop' into refactor_integrator 

    # Conflicts:
#	openpype/plugins/publish/integrate_new.py
    @BigRoy
    BigRoy committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    7fa224a View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2022

  1. Merge remote-tracking branch 'upstream/develop' into refactor_integrator 

    # Conflicts:
#	openpype/plugins/publish/integrate_new.py
    @BigRoy
    BigRoy committed Apr 14, 2022
    Configuration menu
    Copy the full SHA
    971dfff View commit details
    Browse the repository at this point in the history

  2. Match changes that were made to original IntegrateAsset 

    Changes of:
- 312d030
- 507f361
- 29dca65
    @BigRoy
    BigRoy committed Apr 14, 2022
    Configuration menu
    Copy the full SHA
    2e2deb3 View commit details
    Browse the repository at this point in the history

  3. Fix indentation

    @BigRoy
    BigRoy committed Apr 14, 2022
    Configuration menu
    Copy the full SHA
    0fdd4f1 View commit details
    Browse the repository at this point in the history

  4. Store alt sites in a set

    @BigRoy
    BigRoy committed Apr 14, 2022
    Configuration menu
    Copy the full SHA
    1a03bbe View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2022

  1. Merge remote-tracking branch 'upstream/develop' into refactor_integrator

    @BigRoy
    BigRoy committed Apr 21, 2022
    Configuration menu
    Copy the full SHA
    cf302dc View commit details
    Browse the repository at this point in the history

  2. Use logic directly from Sync Server module

    @BigRoy
    BigRoy committed Apr 21, 2022
    Configuration menu
    Copy the full SHA
    8a970b1 View commit details
    Browse the repository at this point in the history

  3. Fix: refactor to use correct function

    @BigRoy
    BigRoy committed Apr 21, 2022
    Configuration menu
    Copy the full SHA
    ae1acb9 View commit details
    Browse the repository at this point in the history

Commits on May 2, 2022

  1. Merge remote-tracking branch 'upstream/develop' into refactor_integrator 

    # Conflicts:
#	openpype/plugins/publish/integrate_new.py
    @BigRoy
    BigRoy committed May 2, 2022
    Configuration menu
    Copy the full SHA
    398ccf9 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2022

  1. Merge remote-tracking branch 'upstream/develop' into refactor_integrator 

    # Conflicts:
#	openpype/plugins/publish/integrate_new.py
    @BigRoy
    BigRoy committed Jul 5, 2022
    Configuration menu
    Copy the full SHA
    b5e0b3b View commit details
    Browse the repository at this point in the history

  2. Move IntegrateAsset

    @BigRoy
    BigRoy committed Jul 5, 2022
    Configuration menu
    Copy the full SHA
    3e058c6 View commit details
    Browse the repository at this point in the history

  3. Revert integrator to latest develop

    @BigRoy
    BigRoy committed Jul 5, 2022
    Configuration menu
    Copy the full SHA
    fd2d07e View commit details
    Browse the repository at this point in the history

  4. Remove duplicate source family

    @BigRoy
    BigRoy committed Jul 5, 2022
    Configuration menu
    Copy the full SHA
    271a829 View commit details
    Browse the repository at this point in the history

  5. Update USD families with latest develop

    @BigRoy
    BigRoy committed Jul 5, 2022
    Configuration menu
    Copy the full SHA
    148ac26 View commit details
    Browse the repository at this point in the history

  6. Set up old vs. new integrator per host

    @BigRoy
    BigRoy committed Jul 5, 2022
    Configuration menu
    Copy the full SHA
    035c4d2 View commit details
    Browse the repository at this point in the history

  7. Remove 'intent' context data override 

    @iLLiCiTiT says: Intent should be a dictionary with "value" and "label", to be able tell if you want use value or label of the intent in templates.
    @BigRoy
    BigRoy committed Jul 5, 2022
    Configuration menu
    Copy the full SHA
    a375763 View commit details
    Browse the repository at this point in the history

  8. Refactor integrator labels

    @BigRoy
    BigRoy committed Jul 5, 2022
    Configuration menu
    Copy the full SHA
    b4697b6 View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2022

  1. Merge pull request #2898 from BigRoy/refactor_integrator

    @mkolar
    mkolar authored Jul 13, 2022
    Configuration menu
    Copy the full SHA
    f2ce000 View commit details
    Browse the repository at this point in the history

  2. add host and families settings to integrators

    @mkolar
    mkolar committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    45473c5 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2022

  1. move all hosts and families to the new integrator

    @mkolar
    mkolar committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    9ef9c79 View commit details
    Browse the repository at this point in the history

  2. move subset group collecting to early integrator

    @mkolar
    mkolar committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    7646c54 View commit details
    Browse the repository at this point in the history

  3. rename old integrator to integrate legacy

    @mkolar
    mkolar committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    0c59f15 View commit details
    Browse the repository at this point in the history

  4. call 'bulk_write' directly on legacy_io

    @iLLiCiTiT
    iLLiCiTiT committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    84781c1 View commit details
    Browse the repository at this point in the history

  5. new integrator can tell legacy one that should not process the instance

    @iLLiCiTiT
    iLLiCiTiT committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    842cf06 View commit details
    Browse the repository at this point in the history

  6. representations are checked before instance registration begins

    @iLLiCiTiT
    iLLiCiTiT committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    a1784fc View commit details
    Browse the repository at this point in the history

  7. use already prepared modules from context

    @iLLiCiTiT
    iLLiCiTiT committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    7016ca4 View commit details
    Browse the repository at this point in the history

  8. use host name from context data

    @iLLiCiTiT
    iLLiCiTiT committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    d04abc3 View commit details
    Browse the repository at this point in the history

  9. skip instances marked to be integrated on farm

    @iLLiCiTiT
    iLLiCiTiT committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    79c01bd View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2022

  1. simplified settings for skipping of families

    @iLLiCiTiT
    iLLiCiTiT committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    f398fae View commit details
    Browse the repository at this point in the history

  2. use settings to decide if new integrator should skip instances

    @iLLiCiTiT
    iLLiCiTiT committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    1a024d3 View commit details
    Browse the repository at this point in the history

  3. added remaining hosts to integrator hosts

    @iLLiCiTiT
    iLLiCiTiT committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    0ebd4b7 View commit details
    Browse the repository at this point in the history

  4. use settings for publish templates from legacy integrator

    @iLLiCiTiT
    iLLiCiTiT committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    6c457b2 View commit details
    Browse the repository at this point in the history

  5. fixed data access

    @iLLiCiTiT
    iLLiCiTiT committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    30db574 View commit details
    Browse the repository at this point in the history

  6. make sure legacy integrator happens after new integrator

    @iLLiCiTiT
    iLLiCiTiT committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    3c35cbc View commit details
    Browse the repository at this point in the history

  7. rename and move CollectSubsetGroup to IntegrateSubsetGroup in settings

    @iLLiCiTiT
    iLLiCiTiT committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    dc569c6 View commit details
    Browse the repository at this point in the history

  8. check for existence of subset group on instance before profiling

    @iLLiCiTiT
    iLLiCiTiT committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    76c594a View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2022

  1. keep subset group template settings but mark them as deprecated with … 

    …hint where to move the value
    @iLLiCiTiT
    iLLiCiTiT committed Jul 20, 2022
    Configuration menu
    Copy the full SHA
    037ed71 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2022

  1. fixing the host condition

    @mkolar
    mkolar committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    69246a7 View commit details
    Browse the repository at this point in the history