Skip to content

[TMP, do not merge] adding example for task with FuctionImage #432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

djarecka
Copy link
Collaborator

@djarecka djarecka commented Feb 26, 2021

Acknowledgment

  • I acknowledge that this contribution will be available under the Apache 2 license.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Summary

showing example with FunctionalImage as an input - print statements show when the hash value changes

test_5 is the one that failing

Checklist

  • All tests passing
  • I have added tests to cover my changes
  • I have updated documentation (if necessary)
  • My code follows the code style of this project
    (we are using black: you can pip install pre-commit,
    run pre-commit install in the pydra directory
    and black will be run automatically with each commit)

@codecov
Copy link

codecov bot commented Feb 26, 2021

Codecov Report

Attention: Patch coverage is 0% with 13 lines in your changes missing coverage. Please review.

Project coverage is 42.62%. Comparing base (b934e4b) to head (be5470d).
Report is 1469 commits behind head on main.

Files with missing lines Patch % Lines
pydra/engine/helpers.py 0.00% 5 Missing and 2 partials ⚠️
pydra/engine/workers.py 0.00% 5 Missing and 1 partial ⚠️

❌ Your patch status has failed because the patch coverage (0.00%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

❗ There is a different number of reports uploaded between BASE (b934e4b) and HEAD (be5470d). Click for more details.

HEAD has 27 uploads less than BASE
Flag BASE (b934e4b) HEAD (be5470d)
unittests 30 3
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #432       +/-   ##
===========================================
- Coverage   82.47%   42.62%   -39.86%     
===========================================
  Files          20       20               
  Lines        3858     3871       +13     
  Branches     1049     1052        +3     
===========================================
- Hits         3182     1650     -1532     
- Misses        484     1897     +1413     
- Partials      192      324      +132     
Flag Coverage Δ
unittests 42.62% <0.00%> (-39.78%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@satra
Copy link
Contributor

satra commented Feb 27, 2021

this may help track down the issue. while the functionality of an object is preserved through a roundtrip through cloudpickle, the bytestring is not preserved. so if the test that is failing is saving the task to file and loading it, it would likely get a different hash.

In [15]: def func1(a): 
    ...:     return a + 1 
    ...: class foo: 
    ...:     def test(self, b): 
    ...:         return func1(b) 
    ...:                                                                                                

In [16]: ff = foo()                                                                                     

In [17]: sha256(cp.dumps(ff)).hexdigest()                                                               
Out[17]: '9dcd775519701de26beab29199ef7259657c406caf5af36186192a0fa47967d4'

In [18]: sha256(cp.dumps(deepcopy(ff))).hexdigest()                                                     
Out[18]: '9dcd775519701de26beab29199ef7259657c406caf5af36186192a0fa47967d4'

In [19]: sha256(cp.dumps(cp.loads(cp.dumps(ff)))).hexdigest()                                           
Out[19]: 'b85dbb27044c14651596da8eaba9818cfa6351162cc296b2bc1df571891b35c2'

In [20]: ff.test(1)                                                                                     
Out[20]: 2

In [21]: cp.loads(cp.dumps(ff)).test(1)                                                                 
Out[21]: 2

@satra
Copy link
Contributor

satra commented Feb 28, 2021

indeed, we should not rely on pickle.

you could consider using the __dict__ or __getstate__ of the object if available. or something else about the object. as far as i know there is no generic solution in python for this.

in the short term we could issue exceptions and/or force those tasks to always run.

@satra
Copy link
Contributor

satra commented Feb 28, 2021

here is a related post (but old - some things in python have changed with 3.x): https://stackoverflow.com/questions/16157835/create-hash-for-arbitrary-objects/

@djarecka
Copy link
Collaborator Author

djarecka commented Mar 1, 2021

@satra - I actually did test what happens when I create a deepcopy of this specific object and I was able to get the same hash value.

@djarecka djarecka closed this Apr 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants