Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix some issues #1

Merged
merged 6 commits into from
May 6, 2021
Merged

Fix some issues #1

merged 6 commits into from
May 6, 2021

Conversation

enra64
Copy link
Collaborator

@enra64 enra64 commented May 6, 2021

This PR fixes some issues. It also, yet again, reformats everything - I did this so I could use a common formatter on my diverged fork from felixmusmann/jointly, making getting my fixes much easier. I've added the formatter (black) as a GitHub action, so hopefully in the future no more divergent formatting will be considered.

The list of issues this branch fixes is as following:

  • by resampling the data for each column sequentially and creating a dataframe from that, we can avoid the large memory allocation required for resampling every column in a single dataframe. This change takes place in helpers/get_equidistant_signals.
  • There is some issue with plotting the shake segments if the logger is set to INFO - probably something similar to the problem above, since a MemoryError gets thrown. However, since I didn't need those plots, I simply catch the error if it occurs and skip the drawing - this could maybe be made into an issue.
  • I've made the start- and end-window seperately configurable, and added an error if the window lengths are longer than the data
  • If start- and end-shake are detected to be the same, an error is thrown
  • This branch should fix an off-by-two error that occured because the shift index after correlation was wrong. I think MSCL tried to fix that problem here, but that was more of a band-aid - after changing shift_in_samples in synchronizer to add 1 instead off subtracting it, the reference signal became much better synced, even without MSCL's fix
  • If the threshold is smaller than 0 or greater than 1, an error is thrown
  • If first or second shake is missing, an exception is thrown
  • This PR adds a function to export pickled dataframes for each sensor indivvidually - this makes reading them afterwards much faster, and avoids the extremely sparse CSV that could be generated if the synchronization created many different data indices (up to 7*128=896 per second per sensor for me)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants