TQDM progress bar

[amccaugh]

Description of your problem

The nested-loop function of the TQDM progress bar does not behave as it should.

When you run the TQDM progress bar module in Spyder, running a single loop produces the correct ouput behavior: the progress bar remains on a single line and updates itself. For instance,

from tqdm import trange
from time import sleep

for i in trange(100, desc='Single loop'):
    sleep(0.01)

Produces a single line which updates itself nicely then returns you to the console

Single loop: 50%|█████ | 50/100 [00:00<00:00, 98.52it/s]

However, if you try to use nested loops, TQDM is no longer able to update the two lines without adding newlines, resulting in spammed whitespace. For instance:

from tqdm import trange
from time import sleep

for i in trange(10, desc='1st loop'):
    for j in trange(5, desc='2nd loop', leave=False):
        for k in trange(100, desc='3nd loop'):
            sleep(0.01)

Will produce a console filled with extra whitespace and each new progress update will take up a newline like so:

1st loop:   0%|          | 0/10 [00:00<?, ?it/s]
2nd loop:   0%|          | 0/5 [00:00<?, ?it/s]

3nd loop:   0%|          | 0/100 [00:00<?, ?it/s]

3nd loop:  10%|█         | 10/100 [00:00<00:00, 97.94it/s]

3nd loop:  20%|██        | 20/100 [00:00<00:00, 96.52it/s]

3nd loop:  30%|███       | 30/100 [00:00<00:00, 96.06it/s]

3nd loop:  40%|████      | 40/100 [00:00<00:00, 96.06it/s]

3nd loop:  50%|█████     | 50/100 [00:00<00:00, 96.06it/s]

3nd loop:  60%|██████    | 60/100 [00:00<00:00, 96.21it/s]

3nd loop:  70%|███████   | 70/100 [00:00<00:00, 96.19it/s]

3nd loop:  80%|████████  | 80/100 [00:00<00:00, 96.06it/s]

However, in the regular IPython console started from the command line, this behaves as expected, with each line remaining in place and being updated in-place.

1st loop:  10%|█         | 1/10 [00:05<00:47,  5.33s/it]
2nd loop:  20%|██        | 1/5 [00:01<00:04,  1.06s/it]
3nd loop:  70%|███████   | 70/100 [00:00<00:00, 95.53it/s]

I have tried installing colorama as suggested in this TQDM issue here, but the issue remains. It is also consistent across Windows and OS X versions of Spyder.

In both OS X and Windows, the TQDM behavior works as desired if I input it directly into a command-line python console (or IPython console), but produces the spammy behavior if I run it in the Spdyer console window.

I'm not familiar with the console system, but I believe this may have something to do with "up" character not being respected as it's passed from the ipython console to the spyder console, and so the upper line cannot be updated and instead. But that's just a guess

Versions and main components

• Spyder Version: 3.2.5
• Python Version: 3.6.3
• Qt Version: 5.6.2
• PyQt Version: 5.6
• Operating system: Windows (also tested on OS X with same setup)

Dependencies

IPython >=4.0 : 6.1.0 (OK)
cython >=0.21 : 0.26.1 (OK)
jedi >=0.9.0 : 0.10.2 (OK)
nbconvert >=4.0 : 5.3.1 (OK)
numpy >=1.7 : 1.13.3 (OK)
pandas >=0.13.1 : 0.20.3 (OK)
pycodestyle >=2.3: 2.3.1 (OK)
pyflakes >=0.6.0 : 1.6.0 (OK)
pygments >=2.0 : 2.2.0 (OK)
pylint >=0.25 : 1.7.4 (OK)
qtconsole >=4.2.0: 4.3.1 (OK)
rope >=0.9.4 : 0.10.5 (OK)
sphinx >=0.6.6 : 1.6.3 (OK)
sympy >=0.7.3 : 1.1.1 (OK)

[jnsebgosselin]

@amccaugh Thanks for your well detailed bug report! It is much appreciated.

@ccordoba12 It is happening also with the bare qtconsole. May be related to #3437.

This is how this looks in an external Python console started from Spyder:

[ccordoba12]

Thanks for reporting. Two things to say here:

1. This is not really a Spyder but a qtconsole one. That's because Spyder embeds that package to use it in their consoles.

2. I'm not familiar with the console system, but I believe this may have something to do with "up" character not being respected as it's passed from the ipython console to the spyder console

   qtconsole doesn't embed IPython console, hence it doesn't use any terminal technologies (like readline). It's a graphical console, completely implemented in Qt. So you need to dig into its code to see how it handles other console characters (like \r):

   jupyter/qtconsole@8b6edfa

[ccordoba12]

Bottom line: please report this error in qtconsole.

[amccaugh]

Will do, thanks!

For future reference, I submitted this as a qtconsole issue here: jupyter/qtconsole#263

[jontis]

My 2 cents. I'm also trying to run it and it produces the undesired skipping lines in single loop as well. Bare console works, Ipython console works. Spyder console does not work.

[jontis]

just a comment on this ProgressBar works correctly in spyder.
They use some printing method that does not carry this issue.

[ccordoba12]

@jontis, so are you using nested tqdm loops?

[jontis]

I'd say no. The main loop is the only one with progessbar, but there are other loops within it. They are not supposed to output anything.
Does that qualify as nested or not?

[ccordoba12]

Please post a simple example of problematic code in the qtconsole issue referenced above.

[jontis]

Upon cleaning up the code to produce minimal example, and resetting the system, tqdm started to work just like progressbar worked all the time. I'm uncertain which combination caused the issue. However, I'm sure it will appear again and then, I'll try to pinpoint it better.

[ma-sadeghi]

It seems that Spyder still doesn't support nested progress bars. Is this still being pursued?

[ccordoba12]

I thought they were fixed. We'll take a look at this after we release 4.2.0

[seyeeet]

still have this issue via spyder 4.2.0

[dalthviz]

Hi @seyeeet yep this has not been fix but hopefully a fix will be part of Spyder 4.2.1

[ma-sadeghi]

Is this issue on your radar for future releases? Thanks!

[cJames9]

Hello, any news on this issue? Thanks in advance!

[ccordoba12]

Nop, sorry. I'll try to take a look at it in our 5.4.2 version, to be released in February.

[hifrah]

Hello, any news on this issue? Thanks in advance!