-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
UnicodeDecodeError: After upgrading from V7R2 to V7R3 ibm_db.result() is failing #8
Comments
Just a heads up that I'll be out for the next few weeks, but I will look in to this when I get back. Seems like maybe it's not getting back the correct length or it isn't getting null-terminated correctly. Can you find the PTF level of QSQCLI with |
@kadler Here is PTF level of QSQCLI For server having issues:
And for server working fine:
It is different on server having issue than all other servers not having issues. I am thinking to ask iSeries admins to update PTF to SAME as working servers SI70581. |
@kadler Server having issues is at a higher OS level of V7R3 with a fairly current set of groups installed like DB2 group (SF99703) is at level 22. So iSeries admins said it cannot have other servers PTF - SI70581 and question is: "which QSQCLI PTF is recommended for both V7R2 and V7R3 OS levels"? |
Issue is specifically in ibm_db.result(stmt, 'STRMNAME') when result set is fetched using ibm_db.fetch_row(stmt). Code works If I fetch result set using dict = ibm_db.fetch_assoc(stmt) and use dict['STRMNAME']. ibm_db.result(stmt, xxx) is probably not getting correct length somehow. Not sure what is causing the issue in ibm_db.result method. |
Yeah, my guess is that either the wrong length is being returned or it is not getting null-terminated properly. What is the data type for STRMNAME? |
STREAM_NAME FOR COLUMN STRMNAME |
Can you please tell me what the CCSID of the data is? Also, what is the output of Also, what is the PASE locale set to? (output of |
Data CCSID is: 37 and is same on ALL servers: working, not working.
|
I was unable to recreate this, but it seems that PTF SI74238 was related to some UTF-8 changes and had some problems elsewhere as well and a new APAR was created to address those issues: https://www.ibm.com/support/pages/apar/SE75216 Would be worth a try to see if it fixes your issue. |
I did some tracing of the data returned with SI74238 and it doesn't seem to null-terminate properly, which leads to unpredictable results. It's likely that in the initial testing I was doing the data following the buffer happened to be a null byte or valid UTF-8 data followed by a null byte. Either way, the results are unpredictable and incorrect. AFAICT, this only occurs when the results were truncated, which occurs because I will need to add code to handle expansion as well as ensure that the result is null-terminated. |
I've made some changes to handle UTF-8 expansion. You can test it out easily by unzipping ibm_db.so.zip to where your script is and re-running it (or unzip it somewhere and set |
@kadler How to unzip in PASE on IBM i? unzip, uncompress, unpack - no command is working.
|
Unzip is available from our yum repo: https://ibmi-oss-docs.readthedocs.io/en/latest/yum/README.html Otherwise, you can always uznip on a PC and upload it from there. |
@kadler yay! it's working! My company plans to upgrade live prod box to V7R3 OS level. When could your fix be available as official ibm_db install using yum or pip? |
Versions/Env:
python 3.6.10
ibm-db 2.0.5.12
OS: IBM i 7.2
local database: DB2
ibm_db.result(stmt, 'STRMNAME') is failing with "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdd in position 38: invalid continuation byte". Data in STRMNAME does NOT have 38th position, infact Column itself is 30 char long.
Here is the data:
STREAMID | STRMNAME | TRANDT | TRANTOT | TRANCAT | TRANDESC
1 | DAD SMITH | 1/1/2019 | 5833.33 | PAYCHECK | DAD SMITH PAYROLL
1 | DAD SMITH | 2/1/2019 | 5833.33 | PAYCHECK | DAD SMITH PAYROLL
1 | DAD SMITH | 3/1/2019 | 5833.33 | PAYCHECK | DAD SMITH PAYROLL
1 | DAD SMITH | 4/1/2019 | 5833.33 | PAYCHECK | DAD SMITH PAYROLL
1 | DAD SMITH | 5/1/2019 | 5833.33 | PAYCHECK | DAD SMITH PAYROLL
1 | DAD SMITH | 6/1/2019 | 9724.33 | PAYCHECK | DAD SMITH PAYROLL
Same python script works in python 3.6.9 and ibm-db 2.0.5.9 as well as ibm-db 2.0.5.12 on other IBM i boxes.
Why is it failing? How can I make it work in ibm-db 2.0.5.12?
ibm_db.client_info(conn) result on IBM i is:
DRIVER_NAME: string(13) "QSQCLI.SRVPGM"
DRIVER_VER: string(5) "07030"
DATA_SOURCE_NAME: string(8) "XXXXXXXX"
DRIVER_ODBC_VER: string(4) "3.00"
ODBC_VER: string(0) ""
ODBC_SQL_CONFORMANCE: string(4) "CORE"
APPL_CODEPAGE: int(1208)
CONN_CODEPAGE: int(1208)
The text was updated successfully, but these errors were encountered: