Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IMDReader removed from imdclient #54

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

IMDReader removed from imdclient #54

wants to merge 3 commits into from

Conversation

amruthesht
Copy link
Contributor

@amruthesht amruthesht commented Feb 19, 2025

Fixes #53

This PR assumes that IMDReader has been integrated into MDAnalysis - PR

Changes made in this Pull Request:

  • IMDReader and StreamReaderBase has been removed from the main package (and potentially moved to MDAnalysis PR) i.e. files IMD.py and streambase.py
  • Tests for the IMDReader have been removed from the tests folder (and potentially moved to mdanalysis testsuite)

PR Checklist

  • Tests?
  • Docs?
  • CHANGELOG updated?
  • Issue raised/referenced?

@amruthesht
Copy link
Contributor Author

amruthesht commented Feb 19, 2025

TO-DOs

Get rid of any usage or references to MDAnalysis (as suggested by @orbeckst)

  • Remove any MDAnalysis usage for generic tests
  • Make simple pseudo "reader" that can be used for tests

Copy link
Member

@orbeckst orbeckst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove MDAnalysis from pyproject.toml etc — there shouldn't be a any place in the package where it gets installed or imported.

For right now, streamanalysis is in a weird place. We can leave it in there and test it or we can disable/remove it. Something to discuss.

@orbeckst
Copy link
Member

As discussed, we can use MDAnalysis for testing (reading trajectories) but not for streaming itself.

1. Added MDA `imdv3-dev` as dependency in `*.toml` and `*.yaml` files
2. Removed stream-analysis, backends and results files.
3. Chnages IMReader imports to MDA imports
Copy link
Member

@orbeckst orbeckst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My primary issue with the current approach is that it still requires IMDReader for testing. The way that we had concluded our discussion (ASU+MDA meeting on 2025-03-03) was the decision

imdclient: allow MDA as test dependency for imdclient tests (but do not allow IMDReader)

In my comment below I am outlining how you can use MDAnalysis to construct a Universe after obtaining the data from the stream as numpy arrays. For this to work you need to write code that only uses imdclient to pull data from a stream.

Copy link
Member

@orbeckst orbeckst Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't think that u = mda.Universe((tmp_path / topol), f"imd://localhost:{port}") should be used because it uses IMDReader. I understand that it makes it easier to write integration tests. But it really muddles the water and really tests MDAnalysis and only indirectly imdclient. I believe it should be relatively straightforward to pull the data from the streame with imdclient only and then put it into a Universe (with in-memory representation) for further processing and comparisons.

You should be able to

  1. pull the coordinates and velocities from the stream as numpy arrays

  2. save them as a trajectory: create a Universe from the topology and the numpy array with the memory reader:

    u = mda.Universe(tmp_path / topo, coordinate_array, dt=dt)

    where coordinate_array is a numpy array of shape (n_frames, n_atoms, 3) and dt is the time between frames in ps.

    There is also a way to create a Universe with velocities and forces. One might have to use MemoryReader directly and associate it ... maybe just passing the memoryreader instance as a trajectory file will work.

  3. compare the original trajectory and your new universe (with in-memory data).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @orbeckst for the detailed suggestion.
Unfortunately, the eaxct apporoach as above would be a little convoluted as all the inof the in the buffer is not readily available as a NumPy array. Instead the IMDCleint object reads one frame at a time. Thus, I'm implementing a minimalReader class that would essentially use IMDClient to read one frame at a time; and then compare that with the data in the files (read in as a MDA universe using MDA File Readers)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just iterate through the stream and copy each numpy array out of the IMDFrame and into an array like Oliver is describing?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now we're doing almost the same thing except now we're wrapping stream iteration in a Universe and then writing it to trr instead of an in-memory numpy array

Changes implemented
1. Removed MDAnalaysis installs for imdv3-dev branch
2. Implemented minimalReader class for tests (under progress)
TO-DO
1. Change `test_MDengine.py` to work with minimalReader
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

split IMDReader from imdclient
3 participants