binarize() TypeError when running Cross-validate with a low threshold

[johndtanner]

Hello!

While running through the GPseer tutorial, I experienced a bug while running the following command with the raw pfcrt data:

gpseer cross-validate pfcrt-raw-data.csv --spline_order 2 --spline_smoothness 100000 --threshold 5 --output_root linear_spline2_threshold5 --n_samples 1000

Output:

[GPSeer] Reading data from pfcrt-raw-data.csv...
[GPSeer] └──> Done reading data.
[GPSeer] Fitting all data data...
Traceback (most recent call last):
  File "/home/john/anaconda3/envs/gptest2/bin/gpseer", line 8, in <module>
    sys.exit(entrypoint())
  File "/home/john/anaconda3/envs/gptest2/lib/python3.8/site-packages/gpseer/main.py", line 180, in entrypoint
    run(parser)
  File "/home/john/anaconda3/envs/gptest2/lib/python3.8/site-packages/gpseer/main.py", line 174, in run
    main(logger, **kwargs)
  File "/home/john/anaconda3/envs/gptest2/lib/python3.8/site-packages/gpseer/cross_validate.py", line 131, in main
    full_model.fit()
  File "/home/john/.local/lib/python3.8/site-packages/epistasis/models/pipeline.py", line 49, in fit
    gpm = model.fit_transform(X=X, y=y)
  File "/home/john/.local/lib/python3.8/site-packages/epistasis/models/classifiers/base.py", line 52, in fit_transform
    self.fit(X=X, y=y, **kwargs)
  File "/home/john/.local/lib/python3.8/site-packages/epistasis/models/utils.py", line 52, in inner
    return method(self, **kws)
  File "/home/john/.local/lib/python3.8/site-packages/epistasis/models/classifiers/logistic.py", line 63, in fit
    self._fit_classifier(X=X, y=y)
  File "/home/john/.local/lib/python3.8/site-packages/epistasis/models/classifiers/base.py", line 44, in _fit_classifier
    y = binarize(y.reshape(1, -1), self.threshold)[0]
TypeError: binarize() takes 1 positional argument but 2 were given

A collaborator, Jamie Robertson has been in contact with @Zsailer on this bug. This is what Zach had to say:

The issue arises when trying to fit a dataset that doesn't have data points on both sides of the "activity threshold". For example, if the threshold is set too low, it may be unlikely that you detect any data points below that value and the model won't have enough information to perform a valid fit. I see a couple issues with our software—1) we should raise a more helpful error message when this happens and 2) we should probably skip this "threshold" fitting when cross-validating since it's inevitable that a sample dataset will be drawn that doesn't demonstrate a clear threshold. In the meantime, increasing your value for the threshold should help alleviate the issue when cross validating.

I've tried rerunning the command with thresholds from -5 to 130 at increments of 5, and all thresholds return the same error, which is not what I would expect as these values span from the minimum to the maximum values of the phenotypes in the pfcrt data.
The command works fine when the threshold argument is not specified.

Thanks for any help, and please let me know if there is more information I can provide

[harmsm]

That’s very strange. I have reproduced the error on my end and am looking into it. My gut says it’s because something changed in sklearn, the underlying regression package we use. Out of curiosity, could you please let me know what version of scikit-learn you are using? (If not obvious how to check, open a python terminal and type `import sklearn` then `sklearn.__version__`). Thanks, Mike

…

On Mar 2, 2022, at 10:28 PM, johndtanner ***@***.***> wrote: Hello! While running through the GPseer tutorial, I experienced a bug while running the following command with the raw pfcrt data: gpseer cross-validate pfcrt-raw-data.csv --spline_order 2 --spline_smoothness 100000 --threshold 5 --output_root linear_spline2_threshold5 --n_samples 1000 Output: [GPSeer] Reading data from pfcrt-raw-data.csv... [GPSeer] └──> Done reading data. [GPSeer] Fitting all data data... Traceback (most recent call last): File "/home/john/anaconda3/envs/gptest2/bin/gpseer", line 8, in <module> sys.exit(entrypoint()) File "/home/john/anaconda3/envs/gptest2/lib/python3.8/site-packages/gpseer/main.py", line 180, in entrypoint run(parser) File "/home/john/anaconda3/envs/gptest2/lib/python3.8/site-packages/gpseer/main.py", line 174, in run main(logger, **kwargs) File "/home/john/anaconda3/envs/gptest2/lib/python3.8/site-packages/gpseer/cross_validate.py", line 131, in main full_model.fit() File "/home/john/.local/lib/python3.8/site-packages/epistasis/models/pipeline.py", line 49, in fit gpm = model.fit_transform(X=X, y=y) File "/home/john/.local/lib/python3.8/site-packages/epistasis/models/classifiers/base.py", line 52, in fit_transform self.fit(X=X, y=y, **kwargs) File "/home/john/.local/lib/python3.8/site-packages/epistasis/models/utils.py", line 52, in inner return method(self, **kws) File "/home/john/.local/lib/python3.8/site-packages/epistasis/models/classifiers/logistic.py", line 63, in fit self._fit_classifier(X=X, y=y) File "/home/john/.local/lib/python3.8/site-packages/epistasis/models/classifiers/base.py", line 44, in _fit_classifier y = binarize(y.reshape(1, -1), self.threshold)[0] TypeError: binarize() takes 1 positional argument but 2 were given A collaborator, Jamie Robertson has been in contact with @Zsailer <https://github.com/Zsailer> on this bug. This is what Zach had to say: The issue arises when trying to fit a dataset that doesn't have data points on both sides of the "activity threshold". For example, if the threshold is set too low, it may be unlikely that you detect any data points below that value and the model won't have enough information to perform a valid fit. I see a couple issues with our software—1) we should raise a more helpful error message when this happens and 2) we should probably skip this "threshold" fitting when cross-validating since it's inevitable that a sample dataset will be drawn that doesn't demonstrate a clear threshold. In the meantime, increasing your value for the threshold should help alleviate the issue when cross validating. I've tried rerunning the command with thresholds from -5 to 130 at increments of 5, and all thresholds return the same error, which is not what I would expect as these values span from the minimum to the maximum values of the phenotypes in the pfcrt data. The command works fine when the threshold argument is not specified. Thanks for any help, and please let me know if there is more information I can provide — Reply to this email directly, view it on GitHub <#29>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABFZA6QA2ENFLBFAK3PCEPDU6BL2PANCNFSM5PZRA2LA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.

[johndtanner]

Thanks for looking into this Mike. My Sklearn version is 1.0.2