Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update normalization notebook to work with automatic voltage updates #367

Closed
ngreenwald opened this issue May 1, 2023 · 2 comments · Fixed by #455
Closed

Update normalization notebook to work with automatic voltage updates #367

ngreenwald opened this issue May 1, 2023 · 2 comments · Fixed by #455
Assignees
Labels
enhancement New feature or request

Comments

@ngreenwald
Copy link
Member

Is your feature request related to a problem? Please describe.
The current normalization notebook calculates the MPH per image over the course of a run, fits a curve to that decay, and uses that to normalize each channel in each image. Ionpath is going to be rolling out an update to the instrument which automatically adjusts the detector voltage over the course of the run. This will significantly reduce the amount of signal decay we see over the course of the run.

Describe the solution you'd like
A side effect of this will be that our current approach for normalizing the data will need to be updated. Specifically, we'll probably want to make a separate notebook, updated_normalization or something like that, since the workflow will be different. Rather than fitting a polynomial, we'll want to check that all of the values are roughly on a straight line. We should then use that fitted flat line as the normalization value for all the FOVs for that particular channel.

The outlier detection should instead look for values that are far away from that flat line. Some of the other checks (like making sure the voltage in the JSON doesn't change) will need to be tweaked as well.

@ngreenwald ngreenwald added the enhancement New feature or request label May 1, 2023
@alex-l-kong
Copy link
Contributor

We can use sklearn.LinearRegression (or another standard linear regression library) to generate the fit. How many SDs away from the line of best fit do we want to use for determining outliers?

@ngreenwald
Copy link
Member Author

We'll need to take a look at some real data and then decide

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants