ofxtools hanging on poor FI behavior

[aclindsa]

I've seen ofxtools hang a few times (with different financial institutions), and when I Ctrl-C it, it is in the following loop:

ofxtools/ofxtools/header.py

Line 265 in 8455c43

while True:

This behavior seems to be transient (the same FI may not cause it on other days...). This is from a call to client.request_statements.

From adding some debug prints, it appears that the FIs in question are returning endless empty lines (lovely)

Maybe this could fail more gracefully (like a limit on the max number of iterations of this loop so it doesn't become infinite for poorly-behaved FIs)?

[csingley]

Yeah that is not the finest piece of production quality code I have ever written. Surely I can find a more refined algo for discriminating the header from the body.

In other news, I may soon have need to become proficient in Go. How would you feel about rank novices clawing through your codebase and submitting ill conceived PRs riddled with obvious bugs?

[aclindsa]

In other news, I may soon have need to become proficient in Go. How would you feel about rank novices clawing through your codebase and submitting ill conceived PRs riddled with obvious bugs?

GO for it (haha heh ha ... ha?), that's half of what this "open source" stuff is all about.

Just don't assume that you'll learn the most idiomatic Go from looking at my code...

[aclindsa]

Another wrinkle: ofxgo seems to be able to successfully download OFX from the offending FI... So there may be another layer to this.

[csingley]

Maybe this could fail more gracefully (like a limit on the max number of iterations of this loop so it doesn't become infinite for poorly-behaved FIs)?

I'm not sure there's a better solution than this for neverending file descriptors, and it's simple to code. What I'm wrestling with is how to write the unit test. I haven't forgotten about this.

[aclindsa]

I'm not sure there's a better solution than this for neverending file descriptors, and it's simple to code.

I actually have a version of this locally and it "works", though its not as clean as might be ideal. I played around yesterday with trying to figure out if I could catch this condition higher in the call hierarchy (closer to where the request is made) but did not yet find a generic way to detect that the response was 0 bytes.

[thehesiod]

I just hit this, pre-readline you should be able to cache to buffer position via .tell() and if it hasn't moved after a readline you can consider it EOF