Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I/O performance #120

Open
krizhanovsky opened this issue Jul 31, 2019 · 0 comments
Open

I/O performance #120

krizhanovsky opened this issue Jul 31, 2019 · 0 comments
Labels
Infrastructure The framework reworking and extensions performance The framework performance
Milestone

Comments

@krizhanovsky
Copy link
Contributor

krizhanovsky commented Jul 31, 2019

tls.test_tls_handshake.TlsHandshakeTest.test_fuzzing is an example of relatively long test and profiling tool (#119) show following statistics:

   Ordered by: internal time
   List reduced from 843 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      110   20.027    0.182   20.027    0.182 {method 'recv' of '_socket.socket' objects}
1113586/1098106    1.764    0.000    3.028    0.000 packet.py:206(__setattr__)
295017/121497    0.951    0.000    2.104    0.000 copy.py:145(deepcopy)
561571/535007    0.868    0.000    1.300    0.000 packet.py:189(setfieldval)
52383/32683    0.829    0.000    7.338    0.000 packet.py:68(__init__)
483246/3180    0.798    0.000    0.798    0.000 ec.py:7(egcd)
   225017    0.474    0.000    0.514    0.000 copy.py:267(_keep_alive)
106366/32683    0.415    0.000    7.544    0.000 base_classes.py:192(__call__)
    52383    0.381    0.000    2.491    0.000 packet.py:100(do_init_fields)
   545327    0.339    0.000    0.339    0.000 packet.py:1105(setfieldval)
    25637    0.333    0.000    4.634    0.000 packet.py:653(clone_with)
86570/38236    0.328    0.000    5.173    0.000 packet.py:671(loop)
17690/3610    0.288    0.000    3.528    0.001 packet.py:145(copy)
104344/60454    0.285    0.000    3.480    0.000 packet.py:304(copy_fields_dict)
202293/105803    0.270    0.000    3.305    0.000 packet.py:307(<genexpr>)
    25637    0.230    0.000    0.267    0.000 packet.py:670(__iter__)
32816/1559    0.226    0.000    6.448    0.004 packet.py:309(self_build)
       26    0.208    0.008    0.208    0.008 {built-in method poll}
138676/72646    0.204    0.000    3.145    0.000 packet.py:302(copy_field_value)
       60    0.187    0.003    0.189    0.003 fuzzer.py:39(record)
.....
----------------------------------------------------------------------
Ran 1 test in 35.049s

OK

The test runs for 35 seconds and 20 of them are spent in recv() call which takes timeout=0.5 as a parameter. The problem was discussed in #103 (comment) and #103 (comment) , but with #118 (Debug mode for tests configuration) in mind (the rto parameter for TlsHandshake was introduced to be able to read data from Tempesta in debug mode) it's not easy to fix.

See also comments from commit e88af89 : helpers/remote.py and helpers/analyzer.py also struggle from the timeouts and can be reworked in event-driven fashion.

See and fix also TODO #120 comments around the code.

@krizhanovsky krizhanovsky added Infrastructure The framework reworking and extensions performance The framework performance labels Jul 31, 2019
@krizhanovsky krizhanovsky added this to the Backlog milestone Aug 10, 2019
krizhanovsky added a commit that referenced this issue Dec 13, 2020
'Traffic wasn't captured' in tls.test_tls_integrity.Proxy.test_tcp_segs
test.

Add notes for #120 to fix the timeouts on run_cmd() interface.

Use -n switch for tcpdump to make it run faster.
krizhanovsky added a commit that referenced this issue Dec 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Infrastructure The framework reworking and extensions performance The framework performance
Projects
None yet
Development

No branches or pull requests

1 participant