High CPU usage and slow UI performance

[ghost]

Hi,
I noticed very high cpu usage when I scroll mail list or thread. Cpu usage peaks up to 70-100% (especially in thread view). Tested on release 0.3.5 and newest revision from git.

How to reproduce: open some thread, press j and look at cpu usage...

[pazz]

This could be due to successive spawning of external html-part renderers.
Can you check if simultaniously, many new instances of w3c or whatever you use
fire up?

[ghost]

Not possible. I threat html emails like spam (with some exceptions) and they are not even indexed.

Here is a screenshot: http://i.imgur.com/LriP8Yp.png I'm just scrolling (pressing j key) some thread from arch linux mailing list (no html messages).

Here: http://pastebin.com/u8jJzbe9 is also output from command: python2 -m cProfile /usr/bin/alot It may be helpful.

[pazz]

Hmmk?
Just for the record, could you post the exact version numbers you're using of
urwid, python and alot?

[ghost]

Python 2.7.6
Urwid 1.2.0
Alot 0.3.5

[pazz]

thanks. I really have no idea here. just speculating some more:
Is it possible that you unfold (and thus open and interpret) many signed/encrypted messages?

[laarmen]

Well, my huge threads (from 75 up to 300 mails) are usually Debian related,
which means about a third of the mails are signed, which does account for a
slowness when opening the thread. However, even just scrolling down the
thread is sluggish.

My suspects are the Tree widgets, but I don't have any hard evidence.

[pazz]

Well, a mail only gets read from disk and interpreted
when you unfold it or bring an already onfolded message
into focus the first time. But in any case, it should not
use that much computing power. feel free to
hack on my tree widgets, they are a bit hacky indeed :)

[GuillaumeSeren]

Hi there,
I agree I get the same behavior here on Debian SID,
on big threads maybe it come from the tree view but I didn't see any difference folded or not.

It can come from this version, I will try to upgrade from git.

[josch]

I uploaded an xz-compressed cprofile output from me scrolling through a very long thread: https://mister-muffin.de/p/_83G

Scrolling was very sluggish and slow and the Python process consumed 100% of my CPU during scrolling.

Here is the top of the pstats output in case it helps:

         406159648 function calls (391120742 primitive calls) in 393.638 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.006    0.006  393.640  393.640 alot/__main__.py:4(<module>)
        1    0.000    0.000  393.086  393.086 alot/__main__.py:20(main)
        1    0.000    0.000  380.642  380.642 alot/ui.py:33(__init__)
        1    0.000    0.000  380.583  380.583 /usr/lib/python2.7/dist-packages/urwid/main_loop.py:268(run)
        1    0.000    0.000  380.583  380.583 /usr/lib/python2.7/dist-packages/urwid/main_loop.py:366(_run)
        1    0.000    0.000  380.582  380.582 /usr/lib/python2.7/dist-packages/urwid/main_loop.py:1174(run)
        1    0.000    0.000  380.582  380.582 /usr/lib/python2.7/dist-packages/twisted/internet/base.py:1197(run)
        1    0.290    0.290  380.581  380.581 /usr/lib/python2.7/dist-packages/twisted/internet/base.py:1202(mainLoop)
     5718    0.019    0.000  365.720    0.064 /usr/lib/python2.7/dist-packages/urwid/main_loop.py:1197(wrapper)
   231835    0.283    0.000  319.094    0.001 /usr/lib/python2.7/dist-packages/twisted/internet/base.py:790(runUntilCurrent)
     2516    0.191    0.000  318.749    0.127 /usr/lib/python2.7/dist-packages/urwid/main_loop.py:566(draw_screen)
     2512    0.017    0.000  318.688    0.127 /usr/lib/python2.7/dist-packages/urwid/main_loop.py:1157(_twisted_idle_callback)
     2512    0.014    0.000  318.670    0.127 /usr/lib/python2.7/dist-packages/urwid/main_loop.py:557(entering_idle)
2965870/2516   11.942    0.000  291.619    0.116 /usr/lib/python2.7/dist-packages/urwid/widget.py:135(cached_render)
1350334/2514    4.488    0.000  291.489    0.116 /usr/lib/python2.7/dist-packages/urwid/decoration.py:218(render)
     2514    0.045    0.000  283.203    0.113 /usr/lib/python2.7/dist-packages/urwid/container.py:1059(render)
     2511    0.021    0.000  282.206    0.112 alot/buffers.py:378(render)
8236/2511    0.031    0.000  281.918    0.112 /usr/lib/python2.7/dist-packages/urwid/widget.py:1750(render)
     2513    0.334    0.000  280.489    0.112 /usr/lib/python2.7/dist-packages/urwid/listbox.py:448(render)
128135/127996    5.366    0.000  189.412    0.001 /usr/lib/python2.7/dist-packages/urwid/container.py:2052(render)
5717/5714    0.783    0.000  115.531    0.020 /usr/lib/python2.7/dist-packages/urwid/listbox.py:313(calculate_visible)
   283146    0.368    0.000  104.461    0.000 urwidtrees/widgets.py:84(get_prev)
   316126    0.400    0.000   79.148    0.000 urwidtrees/widgets.py:63(_get)
   316120    1.375    0.000   78.748    0.000 urwidtrees/lru_cache.py:85(wrapper)
   166532    0.245    0.000   77.174    0.000 urwidtrees/widgets.py:50(__getitem__)
   166532    0.139    0.000   76.758    0.000 urwidtrees/nested.py:85(get_decorated)
498602/166532    1.965    0.000   76.619    0.000 urwidtrees/nested.py:69(_get_decorated_entry)
   166532    1.749    0.000   71.918    0.000 urwidtrees/decoration.py:433(decorate)
   231835    0.234    0.000   60.697    0.000 /usr/lib/python2.7/dist-packages/twisted/internet/epollreactor.py:206(doPoll)
  1340362    4.656    0.000   47.333    0.000 /usr/lib/python2.7/dist-packages/urwid/widget.py:779(render)
     3211    0.017    0.000   47.141    0.015 /usr/lib/python2.7/dist-packages/twisted/python/log.py:90(callWithLogger)
     3211    0.020    0.000   47.123    0.015 /usr/lib/python2.7/dist-packages/twisted/python/log.py:83(callWithContext)
     3211    0.007    0.000   47.071    0.015 /usr/lib/python2.7/dist-packages/twisted/python/context.py:117(callWithContext)
     3211    0.012    0.000   47.063    0.015 /usr/lib/python2.7/dist-packages/twisted/python/context.py:61(callWithContext)
     3209    0.016    0.000   47.049    0.015 /usr/lib/python2.7/dist-packages/twisted/internet/posixbase.py:555(_doReadOrWrite)
     3206    0.006    0.000   47.030    0.015 /usr/lib/python2.7/dist-packages/urwid/main_loop.py:1032(doRead)
     3206    0.021    0.000   46.971    0.015 /usr/lib/python2.7/dist-packages/urwid/raw_display.py:392(<lambda>)
     3206    0.027    0.000   46.664    0.015 /usr/lib/python2.7/dist-packages/urwid/raw_display.py:442(parse_input)
     3206    0.006    0.000   46.557    0.015 /usr/lib/python2.7/dist-packages/urwid/main_loop.py:382(_update)
     3206    0.014    0.000   46.550    0.015 /usr/lib/python2.7/dist-packages/urwid/main_loop.py:530(input_filter)