Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP Issue 2043 deprecate ts write #2110

Merged
merged 22 commits into from
Jun 8, 2020

Conversation

richardjgowers
Copy link
Member

Starts #2043

Changes made in this Pull Request:

  • Writer.write(Timestep) is deprecated

PR Checklist

  • Tests?
  • Docs?
  • CHANGELOG updated?
  • Issue raised/referenced?

@richardjgowers
Copy link
Member Author

Turns out a fair few Writers didn't even accept a Timestep, so this helps tighten up the API too #206

I want to go through and get rid of the old usage of w.write(ts) in docs etc so there's not a ton of dep warnings flying around.

Copy link
Member

@orbeckst orbeckst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

Minor comments:

  • add date for removal for deprecations; you might be able to automatically add my comments with the new "code suggestion" beta feature on GitHub
  • check docs!

package/MDAnalysis/coordinates/GRO.py Show resolved Hide resolved
package/MDAnalysis/coordinates/XYZ.py Show resolved Hide resolved
package/MDAnalysis/coordinates/base.py Show resolved Hide resolved
@orbeckst
Copy link
Member

Oh joy, parallel testing is broken.

@codecov
Copy link

codecov bot commented Oct 18, 2018

Codecov Report

Merging #2110 into develop will increase coverage by 0.05%.
The diff coverage is 91.30%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2110      +/-   ##
===========================================
+ Coverage    91.23%   91.28%   +0.05%     
===========================================
  Files          176      176              
  Lines        24084    24555     +471     
  Branches      3156     3160       +4     
===========================================
+ Hits         21972    22416     +444     
- Misses        1489     1516      +27     
  Partials       623      623              
Impacted Files Coverage Δ
package/MDAnalysis/coordinates/TRZ.py 83.57% <63.63%> (-0.26%) ⬇️
package/MDAnalysis/coordinates/TRJ.py 94.57% <83.33%> (-0.47%) ⬇️
package/MDAnalysis/coordinates/DCD.py 97.27% <100.00%> (+0.19%) ⬆️
package/MDAnalysis/coordinates/FHIAIMS.py 95.68% <100.00%> (ø)
package/MDAnalysis/coordinates/GRO.py 95.45% <100.00%> (+0.02%) ⬆️
package/MDAnalysis/coordinates/NAMDBIN.py 100.00% <100.00%> (ø)
package/MDAnalysis/coordinates/TRR.py 95.12% <100.00%> (+0.83%) ⬆️
package/MDAnalysis/coordinates/XTC.py 98.11% <100.00%> (+0.55%) ⬆️
package/MDAnalysis/coordinates/XYZ.py 89.44% <100.00%> (+1.39%) ⬆️
package/MDAnalysis/coordinates/base.py 93.80% <100.00%> (-0.06%) ⬇️
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e835b5b...15e5374. Read the comment docs.

@@ -410,17 +410,36 @@ def __init__(self,
is_periodic=1,
istart=istart)

def write_next_timestep(self, ts):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel that it might be useful to leave one entry point open for handing over custom-timesteps. Perhaps not deprecate here?

Or are there better alternative ways to do non-standard things (such as writing a "trajectory" of principal components... someone did this a while ago), e.g., using Universe.empty() and friends?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are the low-level functions in lib.formats good for that?

Copy link
Member

@kain88-de kain88-de Oct 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this changed in the first place? This is an internal API method to me. The write method in our base class used to handle this just fine. It also allowed us to have the code for unpacking the atomgroup into a timestep just once. I would rather vote for making this a private method and leaving childclasses as they are otherwise.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem was Writer.write(obj) then calls Writer.write_next_timestep(ts) (which is what is implemented in child classes). But really the argument for W.write_next_timestep() should still be an AtomGroup (else you end up with car crash code like this)

So basically more of #206 where it's not really clear what a Writer is meant to implement

I guess we could make it so that:

  • Child classes implement write_next_timestep() which must accept AG/Universe
  • Child classes can optionally accept a Timestep in the above method?

Currently the (very loose) default is that Timestep is accepted, and AG is optional. (But there are a few dissenters to this)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I accept consistency as reasoning. I would still like that this code of unpacking an atomgroup exists only once as a helper function. We can properly test it.

@richardjgowers richardjgowers added this to the 0.19.x milestone Oct 30, 2018
@orbeckst orbeckst force-pushed the issue-2043-deprecate_ts_write branch from a3beb57 to 01016df Compare April 3, 2019 03:17
@orbeckst
Copy link
Member

orbeckst commented Apr 3, 2019

I squshed and rebased against develope and force pushed.

@orbeckst
Copy link
Member

orbeckst commented Apr 3, 2019

I am not sure where Travis CI went – any ideas? I cannot see it as one of the running checks.

@orbeckst
Copy link
Member

orbeckst commented Apr 3, 2019

@kain88-de said in #2110 (comment)

I accept consistency as reasoning. I would still like that this code of unpacking an atomgroup exists only once as a helper function. We can properly test it.

That would be good. Is there any way that the copy&paste code

        if isinstance(ag, base.Timestep):
            warnings.warn(
                'Passing a Timestep to write is deprecated, '
                'use either an AtomGroup or Universe',
                DeprecationWarning)
            ts = ag
        else:
            try:
                ts = ag.ts
            except AttributeError:
                try:
                    # Universe?
                    ts = ag.trajectory.ts
                except AttributeError:
                    raise TypeError("No Timestep found in ag argument")

could be a private method in the baseclass or something like that?

@orbeckst orbeckst mentioned this pull request Apr 3, 2019
4 tasks
@orbeckst
Copy link
Member

orbeckst commented Apr 4, 2019

Merged develop to get the travis fix from PR #2234.

@orbeckst
Copy link
Member

orbeckst commented Mar 6, 2020

@richardjgowers is this still WIP? Don't we need the removal of write(ts) for 1.0?

@richardjgowers
Copy link
Member Author

@orbeckst Yeah I guess the deprecation will get pushed back to 2.0 or something. This isn't super critical, it's just a tightening of APIs

@richardjgowers richardjgowers modified the milestones: 0.20.x, 2.0 Mar 6, 2020
@richardjgowers richardjgowers force-pushed the issue-2043-deprecate_ts_write branch from 8841106 to 46c15af Compare June 5, 2020 08:53
@richardjgowers
Copy link
Member Author

Ok I've raised this from the dead, hopefully the last blocker to 1.0 @MDAnalysis/coredevs

Copy link
Member

@orbeckst orbeckst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests are failing with some chemfiles related issues.

Can you briefly remind me why you renamed some write() methods to write_next_timestep()? Just asking because the name "write_next_timestep" seems antithetical to the intent. Is this just a semantic mishap on our part?

write coordinate information associate with `obj`
"""

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need a deprecation warning here when using a ts?

write_next_timestep() is this the name we're using even though we don't really want to use timestep?

@richardjgowers
Copy link
Member Author

@orbeckst base.Writer.write() calls the subclass write_next_timestep(). The deprecation logic was in base.Writer() so rather than copy it over, I just made these classes use the base .write() implementation by renaming them to write_next_timestep() (which also served a purpose of making sure this method exists).

I guess really we shouldn't have two entrypoints, so maybe write_next_timestep() should also be deprecated, and the implementations put under something like Subclass._write_next()

richardjgowers and others added 8 commits June 6, 2020 14:08
- identified which Writers can currently use a ts (lots don't anyway)
- deprecated Timestep as argument to Writer.write
- Update package/MDAnalysis/coordinates/base.py
- Update package/MDAnalysis/coordinates/XYZ.py
- Update package/MDAnalysis/coordinates/GRO.py
- add test_writer_api.py
- removed obsolete test
removed chemfiles.Writer.write (to use base API method) and put logic
into write_next_timestep
now started in 1.0 and targets v2.0
CI seems to get 0.7.4 for some builds
@richardjgowers richardjgowers force-pushed the issue-2043-deprecate_ts_write branch from b9a3633 to cf4e3a8 Compare June 6, 2020 13:20
Copy link
Member

@IAlibay IAlibay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall lgtm, might want to switch to .. deprecated instead of .. versionchanged in some places.

Just to check am I correct in assuming that the logic here is that we only have deprecation warnings for all the writers that existed pre-1.0 (i.e. why FHIAIMS and Chemfiles don't raise a warning)?

It looks like FHIAIMS and Chemfiles still accept ts, so we'll probably want at the very least some kind of comment somewhere that reminds us to clean up the logic some v2.0?

package/MDAnalysis/coordinates/FHIAIMS.py Show resolved Hide resolved
package/MDAnalysis/coordinates/DCD.py Outdated Show resolved Hide resolved
package/MDAnalysis/coordinates/GRO.py Outdated Show resolved Hide resolved
package/MDAnalysis/coordinates/NAMDBIN.py Show resolved Hide resolved
package/MDAnalysis/coordinates/TRJ.py Show resolved Hide resolved
package/MDAnalysis/coordinates/TRR.py Show resolved Hide resolved
package/MDAnalysis/coordinates/XTC.py Show resolved Hide resolved
package/MDAnalysis/coordinates/base.py Outdated Show resolved Hide resolved
package/MDAnalysis/coordinates/chemfiles.py Show resolved Hide resolved
added TODO 2.0 for removing timestep logic
Copy link
Member

@IAlibay IAlibay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, just two small changes, in the interest of getting 1.0 out, I can push them through if you want @richardjgowers

package/MDAnalysis/coordinates/XYZ.py Show resolved Hide resolved
package/MDAnalysis/coordinates/FHIAIMS.py Outdated Show resolved Hide resolved
Copy link
Member

@IAlibay IAlibay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we missed out TRZ here

added tests for each Writer to make them accept AtomGroup or Universe
Copy link
Member

@IAlibay IAlibay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So testing the warnings, we currently get the following:
GRO, TRZ, NAMDBIN, and XYZ give 1 deprecation warning, everything else give two.
GRO is a write method so that's expected, but the rest are write_next_timestep. We probably want to have the same behaviour across all ts-supporting writers. My opinion here is that the easiest thing would be two put warnings in all the write_next_timestep methods that support ts. It means an extra warning for users, but it's probably not too intrusive.

use write() instead

renamed many functions to _write_next_frame for the internal workings
of how to write a frame
Copy link
Member

@IAlibay IAlibay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're pretty much there, aside from some minor docs stuff and maybe the one test, our main issue is the MOL2 writer which doesn't actually take in a universe.

P.S. I've fixed the merge conflicts to see if the tests all pass.

package/MDAnalysis/coordinates/DCD.py Show resolved Hide resolved
package/MDAnalysis/coordinates/__init__.py Show resolved Hide resolved
package/MDAnalysis/coordinates/base.py Show resolved Hide resolved
package/MDAnalysis/coordinates/MOL2.py Show resolved Hide resolved

self._write_pdb_bonds()
self.close()

# Set the trajectory to the starting position
traj[start]

def write_next_timestep(self, ts=None, **kwargs):
def _write_next_frame(self, ts=None, **kwargs):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the only _write_next_frame that takes a Timestep?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so yes, it's because PDB also defines its own write() method

@IAlibay
Copy link
Member

IAlibay commented Jun 8, 2020

As discussed privately, I've added some "dev docs" for the _write_next_frame methods in 0d35fc6, just so that it's clearer that they have been renamed.

Copy link
Member

@lilyminium lilyminium left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good to me; just a few doc suggestions. Thanks @richardjgowers and @IAlibay!

package/MDAnalysis/coordinates/TRR.py Outdated Show resolved Hide resolved
package/MDAnalysis/coordinates/XTC.py Outdated Show resolved Hide resolved
package/MDAnalysis/coordinates/XYZ.py Outdated Show resolved Hide resolved
package/MDAnalysis/coordinates/XYZ.py Outdated Show resolved Hide resolved
package/MDAnalysis/coordinates/base.py Outdated Show resolved Hide resolved
package/MDAnalysis/coordinates/TRZ.py Outdated Show resolved Hide resolved
package/MDAnalysis/coordinates/XTC.py Outdated Show resolved Hide resolved
package/MDAnalysis/coordinates/XTC.py Outdated Show resolved Hide resolved
package/MDAnalysis/coordinates/base.py Outdated Show resolved Hide resolved
package/MDAnalysis/coordinates/chemfiles.py Outdated Show resolved Hide resolved
richardjgowers and others added 3 commits June 8, 2020 15:19
Co-authored-by: Lily Wang <31115101+lilyminium@users.noreply.github.com>
Co-authored-by: Lily Wang <31115101+lilyminium@users.noreply.github.com>
Co-authored-by: Lily Wang <31115101+lilyminium@users.noreply.github.com>
Copy link
Member

@IAlibay IAlibay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we will need approval by someone else (since I've pushed a few commits), but lgtm!

@richardjgowers richardjgowers dismissed stale reviews from lilyminium and orbeckst June 8, 2020 20:19

changes added

@richardjgowers richardjgowers merged commit 3f7f5d3 into develop Jun 8, 2020
@richardjgowers richardjgowers deleted the issue-2043-deprecate_ts_write branch June 8, 2020 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants