Importing plotly takes 1.6s and 2046924 function calls

[max-sixty]

Is this consistent with expectations? It looks like it's doing a lot of string parsing in Python (JSON?) and SSL connections, even though we're only using it for offline use?

Thanks for the wonderful library!

#%prun import plotly

         2046924 function calls (1057455 primitive calls) in 1.615 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
1142268/196040    0.476    0.000    0.566    0.000 encoder.py:341(_iterencode_dict)
       10    0.136    0.014    0.136    0.014 {method 'read' of '_ssl._SSLSocket' objects}
   196040    0.076    0.000    0.642    0.000 encoder.py:417(_iterencode)
214729/214727    0.068    0.000    0.070    0.000 {isinstance}
       12    0.041    0.003    0.689    0.057 encoder.py:186(encode)
44754/6522    0.037    0.000    0.050    0.000 utils.py:355(node_generator)
      260    0.030    0.000    0.088    0.000 doccer.py:12(docformat)
    74312    0.025    0.000    0.025    0.000 {_json.encode_basestring_ascii}
   2622/1    0.025    0.000    0.039    0.039 utils.py:301(decode_unicode)
       15    0.023    0.002    0.023    0.002 decoder.py:370(raw_decode)
       62    0.021    0.000    0.028    0.000 collections.py:293(namedtuple)
        1    0.017    0.017    0.017    0.017 index.py:3590(_add_accessors)
    16488    0.016    0.000    0.024    0.000 encoder.py:288(_iterencode_list)
     6823    0.014    0.000    0.014    0.000 {method 'expandtabs' of 'str' objects}
     3769    0.014    0.000    0.014    0.000 {method 'reduce' of 'numpy.ufunc' objects}
      260    0.013    0.000    0.022    0.000 doccer.py:128(indentcount_lines)
        1    0.012    0.012    0.031    0.031 graph_reference.py:379(_get_objects)
        1    0.012    0.012    0.030    0.030 graph_reference.py:458(_get_arrays)
       27    0.012    0.000    0.113    0.004 __init__.py:1(<module>)
    62830    0.012    0.000    0.012    0.000 {method 'append' of 'list' objects}
        1    0.011    0.011    0.028    0.028 graph_reference.py:430(_patch_objects)
        3    0.011    0.004    0.045    0.015 machar.py:116(_do_init)
7729/7712    0.010    0.000    0.014    0.000 {method 'join' of 'str' objects}

[chriddyp]

The main reason this takes so long is because we test imports on pandas, numpy, and a few others on import, and pandas itself takes about ~1 second to load!

This makes the some downstream code paths a bit easier, but still we should really only do these imports when we need to. The 1.7 second import lag is rough!!

[max-sixty]

Thanks for the speedy response @chriddyp

The main reason this takes so long is because we test imports on pandas, numpy, and a few others on import, and pandas itself takes about ~1 second to load!

That's not what I'm seeing - if I import pandas first (which also imports numpy) I still get a 1.5s load time, and that still looks to be made up of string parsing + some external connections (curious why there are external connections on import?)

import pandas

%prun import plotly

         2052914 function calls (1061125 primitive calls) in 1.452 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
1142268/196040    0.491    0.000    0.573    0.000 encoder.py:332(_iterencode_dict)
   196040    0.079    0.000    0.651    0.000 encoder.py:411(_iterencode)
      239    0.067    0.000    0.067    0.000 {built-in method marshal.loads}
        6    0.062    0.010    0.062    0.010 {method 'read' of '_ssl._SSLSocket' objects}
   212756    0.050    0.000    0.052    0.000 {built-in method builtins.isinstance}
       12    0.043    0.004    0.697    0.058 encoder.py:182(encode)
44754/6522    0.040    0.000    0.050    0.000 utils.py:355(node_generator)
    450/1    0.040    0.000    1.453    1.453 {built-in method builtins.exec}
      260    0.034    0.000    0.090    0.000 doccer.py:12(docformat)
    74312    0.028    0.000    0.028    0.000 {built-in method _json.encode_basestring_ascii}
   2622/1    0.025    0.000    0.035    0.035 utils.py:301(decode_unicode)
  666/627    0.016    0.000    0.215    0.000 {built-in method builtins.__build_class__}
    16488    0.016    0.000    0.023    0.000 encoder.py:276(_iterencode_list)
    47/38    0.015    0.000    0.030    0.001 {built-in method _imp.create_dynamic}
      260    0.013    0.000    0.024    0.000 doccer.py:128(indentcount_lines)
    58526    0.012    0.000    0.012    0.000 {method 'append' of 'list' objects}
       14    0.011    0.001    0.011    0.001 decoder.py:345(raw_decode)
     6753    0.011    0.000    0.011    0.000 {method 'splitlines' of 'str' objects}
    10312    0.009    0.000    0.010    0.000 {method 'join' of 'str' objects}
      573    0.009    0.000    0.020    0.000 inspect.py:2065(_signature_from_function)
        1    0.009    0.009    0.009    0.009 {built-in method _socket.getaddrinfo}

[max-sixty]

FYI this is still an issue in the latest version of plot.ly. The latest run spent eight seconds (cumulatively, including functions called within) in get_config_file. Let me know if you have any suggestions how to fix. Thank you

[theengineear]

I think this will be fixed when we stop:

1. updating the plot schema on import
2. auto-bundle graph objects instead of creating them dynamically on import.

[theengineear]

Resolved in 1.13.0, at least partially so. There is still some work that gets done on load. I'm going to close this since the bulk of the problem was the network connection (which no longer happens).