Skip to content

Problem with pandas and python 3 #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
moonso opened this issue May 15, 2017 · 6 comments
Closed

Problem with pandas and python 3 #28

moonso opened this issue May 15, 2017 · 6 comments

Comments

@moonso
Copy link
Contributor

moonso commented May 15, 2017

This is not really a peddy problem, has anyone else experienced this?

I get:

/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/IPython/html.py:14: ShimWarning: The `IPython.html` package has been deprecated since IPython 4.0. You should import from `notebook` instead. `IPython.html.widgets` has moved to `ipywidgets`.
  "`IPython.html.widgets` has moved to `ipywidgets`.", ShimWarning)
Traceback (most recent call last):
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/mansmagnusson/Projects/peddy/peddy/__main__.py", line 15, in <module>
    sys.exit(cli())
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/mansmagnusson/Projects/peddy/peddy/cli.py", line 202, in peddy
    in ("ped_check", "het_check", "sex_check")]):
  File "/Users/mansmagnusson/Projects/peddy/peddy/cli.py", line 42, in run
    prefix=prefix, **kwargs)
  File "/Users/mansmagnusson/Projects/peddy/peddy/peddy.py", line 1037, in ped_check
    sub = df.eval('((rel > 0.1) & (pedigree_relatedness < 0.05)) | ((rel < 0.05) & (pedigree_relatedness > 0.1)) | (rel_difference > 0.1) | (rel_difference < -0.1)')
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/frame.py", line 2284, in eval
    return _eval(expr, inplace=inplace, **kwargs)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/eval.py", line 262, in eval
    truediv=truediv)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 727, in __init__
    self.terms = self.parse()
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 744, in parse
    return self._visitor.visit(self.expr)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 313, in visit
    return visitor(node, **kwargs)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 319, in visit_Module
    return self.visit(expr, **kwargs)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 313, in visit
    return visitor(node, **kwargs)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 322, in visit_Expr
    return self.visit(node.value, **kwargs)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 313, in visit
    return visitor(node, **kwargs)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 671, in visit_BoolOp
    return reduce(visitor, operands)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 664, in visitor
    rhs = self._try_visit_binop(y)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 659, in _try_visit_binop
    return self.visit(bop)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 313, in visit
    return visitor(node, **kwargs)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 644, in visit_Compare
    return self.visit(binop)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 313, in visit
    return visitor(node, **kwargs)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 417, in visit_BinOp
    left, right = self._maybe_downcast_constants(left, right)
  File "/Users/mansmagnusson/miniconda3/envs/peddy/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 369, in _maybe_downcast_constants
    name = self.env.add_tmp(np.float32(right.value))
AttributeError: 'UnaryOp' object has no attribute 'value'

Which I guess is the same as pandas-dev/pandas#11235, but that issue should have been solved in an older version of pandas...
I have pandas (0.20.1) installed.
@brentp
Copy link
Owner

brentp commented May 15, 2017 •
edited
Loading

can you print out that expression? and the values of the variable:

sub = df.eval('((rel > 0.1) & (pedigree_relatedness < 0.05)) | ((rel < 0.05) & (pedigree_relatedness > 0.1)) | (rel_difference > 0.1) | (rel_difference < -0.1)')

?
maybe there's a NaN or None in there somehow.

@moonso
Copy link
Contributor Author

moonso commented May 15, 2017

I only use the data that comes with the distribution so I'm running
python -m peddy --plot -p 4 --prefix mystudy data/ceph1463.peddy.vcf.gz data/ceph1463.ped

With python 2.7 it works fine but not 3.6...

Anyway, these are the values in each array, I can not see anything strange:

rel -0.00900662
rel 0.504888
rel 0.494056
rel 0.497319
rel 0.506728
rel 0.502237
rel 0.5
rel 0.493083
rel 0.510908
rel 0.501837
rel 0.509602
rel 0.505445
rel 0.507674
rel -0.00429473
rel -0.0053526
rel 0.492673
rel 0.493245
rel 0.518013
rel 0.508609
rel 0.505695
rel 0.481324
rel 0.500927
rel 0.491221
rel 0.496159
rel 0.497881
rel 0.503841
rel -0.0136424
rel -0.00291391
rel 0.500872
rel 0.501405
rel 0.495232
rel 0.540423
rel 0.472127
rel 0.480185
rel 0.464465
rel 0.541347
rel 0.56531
rel 0.483091
rel 0.532761
rel 0.486394
rel 0.227345
rel 0.273184
rel 0.272447
rel 0.220661
rel 0.480845
rel 0.552504
rel 0.500711
rel 0.450908
rel 0.504065
rel 0.477654
rel 0.526316
rel 0.490029
rel 0.565077
rel 0.213751
rel 0.296996
rel 0.298215
rel 0.264017
rel 0.485149
rel 0.520335
rel 0.544091
rel 0.535012
rel 0.443735
rel 0.47705
rel 0.511283
rel 0.485158
rel 0.228317
rel 0.290568
rel 0.250034
rel 0.27325
rel 0.555512
rel 0.590813
rel 0.507737
rel 0.485102
rel 0.536712
rel 0.461296
rel 0.614997
rel 0.225692
rel 0.303817
rel 0.256744
rel 0.272314
rel 0.558152
rel 0.579495
rel 0.474594
rel 0.529481
rel 0.367334
rel 0.541458
rel 0.321532
rel 0.203738
rel 0.244263
rel 0.22976
rel 0.531221
rel 0.476856
rel 0.497246
rel 0.415114
rel 0.525308
rel 0.246424
rel 0.265119
rel 0.24037
rel 0.252509
rel 0.518548
rel 0.539638
rel 0.578079
rel 0.469407
rel 0.285448
rel 0.200984
rel 0.286807
rel 0.194299
rel 0.404363
rel 0.491341
rel 0.546979
rel 0.293006
rel 0.226814
rel 0.264126
rel 0.237254
rel 0.518416
rel 0.456835
rel 0.25351
rel 0.23196
rel 0.301302
rel 0.214372
rel 0.431287
rel 0.251279
rel 0.298439
rel 0.299557
rel 0.22133
rel 0.552871
rel 0.0137777
rel -0.000402631
rel -0.0268968
rel 0.170429
rel 0.0124815
rel 0.00133815
rel 0.316568
rel -0.0148973
rel 0.168568
rel 0.327713
ped_rel 0.0
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.0
ped_rel 0.0
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.0
ped_rel 0.0
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.5
ped_rel 0.5
ped_rel 0.5
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.5
ped_rel 0.5
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.5
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.25
ped_rel 0.5
ped_rel 0.0
ped_rel 0.0
ped_rel 0.0
ped_rel 0.25
ped_rel 0.0
ped_rel 0.0
ped_rel 0.25
ped_rel 0.0
ped_rel 0.25
ped_rel 0.25
rel_dif 0.00900662
rel_dif -0.0048877
rel_dif 0.00594383
rel_dif 0.00268078
rel_dif -0.00672764
rel_dif -0.00223744
rel_dif 0.0
rel_dif 0.00691673
rel_dif -0.0109079
rel_dif -0.00183678
rel_dif -0.00960159
rel_dif -0.00544548
rel_dif -0.00767416
rel_dif 0.00429473
rel_dif 0.0053526
rel_dif 0.00732672
rel_dif 0.00675496
rel_dif -0.0180132
rel_dif -0.00860929
rel_dif -0.00569534
rel_dif 0.0186755
rel_dif -0.00092715
rel_dif 0.00877893
rel_dif 0.00384107
rel_dif 0.00211921
rel_dif -0.00384104
rel_dif 0.0136424
rel_dif 0.00291391
rel_dif -0.000872374
rel_dif -0.00140506
rel_dif 0.00476822
rel_dif -0.0404227
rel_dif 0.0278732
rel_dif 0.0198151
rel_dif 0.035535
rel_dif -0.0413474
rel_dif -0.0653099
rel_dif 0.0169089
rel_dif -0.0327609
rel_dif 0.0136063
rel_dif 0.0226552
rel_dif -0.0231836
rel_dif -0.0224467
rel_dif 0.029339
rel_dif 0.0191545
rel_dif -0.0525042
rel_dif -0.000710845
rel_dif 0.0490919
rel_dif -0.00406504
rel_dif 0.0223464
rel_dif -0.0263158
rel_dif 0.00997114
rel_dif -0.0650769
rel_dif 0.0362485
rel_dif -0.0469959
rel_dif -0.048215
rel_dif -0.0140171
rel_dif 0.0148515
rel_dif -0.0203348
rel_dif -0.0440906
rel_dif -0.0350118
rel_dif 0.056265
rel_dif 0.0229502
rel_dif -0.0112831
rel_dif 0.0148424
rel_dif 0.0216835
rel_dif -0.040568
rel_dif -3.35574e-05
rel_dif -0.0232504
rel_dif -0.0555115
rel_dif -0.0908134
rel_dif -0.00773668
rel_dif 0.0148976
rel_dif -0.0367122
rel_dif 0.0387037
rel_dif -0.114997
rel_dif 0.0243078
rel_dif -0.0538174
rel_dif -0.00674406
rel_dif -0.0223137
rel_dif -0.0581518
rel_dif -0.0794946
rel_dif 0.0254057
rel_dif -0.0294815
rel_dif 0.132666
rel_dif -0.0414582
rel_dif -0.071532
rel_dif 0.0462622
rel_dif 0.00573748
rel_dif 0.0202395
rel_dif -0.0312212
rel_dif 0.0231445
rel_dif 0.00275373
rel_dif 0.0848859
rel_dif -0.0253081
rel_dif 0.00357565
rel_dif -0.0151187
rel_dif 0.00962958
rel_dif -0.00250903
rel_dif -0.0185478
rel_dif -0.0396382
rel_dif -0.0780793
rel_dif 0.0305932
rel_dif -0.0354483
rel_dif 0.0490157
rel_dif -0.0368071
rel_dif 0.0557005
rel_dif 0.0956371
rel_dif 0.00865915
rel_dif -0.0469785
rel_dif -0.0430062
rel_dif 0.0231864
rel_dif -0.0141256
rel_dif 0.0127459
rel_dif -0.0184159
rel_dif 0.0431645
rel_dif -0.00351003
rel_dif 0.0180399
rel_dif -0.0513018
rel_dif 0.0356283
rel_dif 0.0687129
rel_dif -0.00127935
rel_dif -0.0484389
rel_dif -0.0495571
rel_dif 0.0286699
rel_dif -0.0528713
rel_dif -0.0137777
rel_dif 0.000402631
rel_dif 0.0268968
rel_dif 0.079571
rel_dif -0.0124815
rel_dif -0.00133815
rel_dif -0.0665677
rel_dif 0.0148973
rel_dif 0.081432
rel_dif -0.0777131

@brentp
Copy link
Owner

brentp commented May 15, 2017

It is an issue in pandas, I commented here: pandas-dev/pandas#11235 (comment)

@moonso
Copy link
Contributor Author

moonso commented May 15, 2017

👍 suspected that

@moonso
Copy link
Contributor Author

moonso commented May 16, 2017

Ok so it's here now pandas-dev/pandas#16363

brentp added a commit that referenced this issue Apr 16, 2018
@brentp
fix for #28
e113f8f
@brentp
Copy link
Owner

brentp commented Apr 16, 2018

this is fixed in peddy v0.3.5

@brentp brentp closed this as completed Apr 16, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@brentp @moonso