Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unhandled exception when using Pyadomd as generator #17

Open
ghost opened this issue Feb 14, 2022 · 0 comments
Open

Unhandled exception when using Pyadomd as generator #17

ghost opened this issue Feb 14, 2022 · 0 comments
Labels
bug Something isn't working

Comments

@ghost
Copy link

ghost commented Feb 14, 2022

I was trying to improve runtime of application and yield chunks from pyadomd, transform data and write it immediately.
Unfortunately faced unhandled exception:

File "C:\PythonEnv\CustomerProfiles2\lib\site-packages\pyadomd\pyadomd.py", line 94, in fetchmany
    l.append(next(self.fetchone()))
  File "C:\PythonEnv\CustomerProfiles2\lib\site-packages\pyadomd\pyadomd.py", line 81, in fetchone
    while(self._reader.Read()):
Microsoft.AnalysisServices.AdomdClient.AdomdUnknownResponseException: The server sent an unrecognizable response.
   at Microsoft.AnalysisServices.AdomdClient.XmlaClient.ReadEndElementS(XmlReader reader, String name, String ns)
   at Microsoft.AnalysisServices.AdomdClient.XmlaDataReader.InternalRead()

Logic to reproduce:

def get_data():
  chunk_size = 10000
  path.append('\\Program Files\\Microsoft.NET\\ADOMD.NET\\150')
  from pyadomd import Pyadomd
  
  with Pyadomd(source) as conn:
      with conn.cursor().execute(self.resource) as cur:
          while True:
              rows = cur.fetchmany(chunk_size)
              row_count = len(rows)
  
              if not rows:
                  break
  
              yield rows

def write_data(data):
  some_write_method()

def run(self):
  for chunk in get_data():
      write_customers(customers)

All rows are written successfully (unless you catch locking conflict when Tabular model is refreshed), but at the end last iteration throws an exception.

Having spent some time on it, I got that issue comes from line 93-94 lines of pydomd.py when retrieving next item which is not existing. For example if you have 6 rows to extract and you put chunk_size 1 or 2 or 3 then no exception occurs, but if you put 4 or 5, then you got exception, since first chunk passes ok and then second chunk is not complete.

@S-C-O-U-T S-C-O-U-T added the bug Something isn't working label Feb 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant