Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle unicode better #158

Closed
gijzelaerr opened this issue Dec 18, 2018 · 3 comments
Closed

handle unicode better #158

gijzelaerr opened this issue Dec 18, 2018 · 3 comments
Milestone

Comments

@gijzelaerr
Copy link
Member

python-casacore often doesn't handle unicode strings properly, giving errors like this:

Exiting with exception: ArgumentError(Python argument types in  
    Table._getcol(table, unicode, int, int, int)                                                                         
did not match C++ signature:                                                                                             
    _getcol(casa::TableProxy {lvalue}, casa::String columnname, int startrow, int nrow, int rowincr))                    
 Traceback (most recent call last):                                                                                      
  File "/home/gijs/Work/CubiCal/cubical/main.py", line 356, in main                                                      
    chunks_per_tile=chunks_per_tile, max_chunks_per_tile=GD["dist"]["max-chunks"])                                       
  File "/home/gijs/Work/CubiCal/cubical/data_handler/ms_data_handler.py", line 863, in define_chunk                      
    value = self.fetch(column)                                                                                           
  File "/home/gijs/Work/CubiCal/cubical/data_handler/ms_data_handler.py", line 702, in fetch                             
    return (subset or self.data).getcol(colname, first_row, nrows)                                                       
  File "/home/gijs/Work/CubiCal/.venv2/local/lib/python2.7/site-packages/casacore/tables/table.py", line 1034, in getcol 
    return self._getcol(columnname, startrow, nrow, rowincr)                                                             
ArgumentError: Python argument types in                                                                                  
    Table._getcol(table, unicode, int, int, int)                                                                         
did not match C++ signature:                                                                                             
    _getcol(casa::TableProxy {lvalue}, casa::String columnname, int startrow, int nrow, int rowincr)                     
@gijzelaerr
Copy link
Member Author

related to #138

I'm struggling to figure out how this actually works. It seems that casa has its own string type implementation. For python3 when a string is encoded it becomes a byte string type, while for python2 it is still a string.

I feel that is is going to be painful.

@gijzelaerr
Copy link
Member Author

gijzelaerr added a commit that referenced this issue May 16, 2019
* add simple unicode test

* resurrect convert testing

* test working for py3 and utf8 casacore branch

* mostly reformatting

* correctly check for string types

* cleanup convert test

* more compatiblity

* increase tollerance

* incorporate bruce feedback
@gijzelaerr
Copy link
Member Author

merged!

@gijzelaerr gijzelaerr added this to the 3.1.0 milestone May 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant