Tests for #1061 and #1535 issue #195

Dmitry-Gouriev · 2021-12-08T03:47:32Z

The tests include:

Tests for #1061 issue in accordance with the comment:

a single request with many CRLFs is blocked
a singled request with one CRLF or just LF is passed and both the CRLF or LF are not presented in proxied request
a pipelined request after another request with many CRLFs is blocked
a pipelined request after another request with one CRLF or just LF is passed and both the CRLF or LF are not presented in proxied request
all the tests above with heavily chunked CRLFs

Heavy chunked test for ss_skb_chop_head_tail() in context of refriting PURGE request as GET. The test detects a bug
decribed there

It seems that the tests partialy covers issue #115.

… in #1535

Fixed misprint in README.md

krizhanovsky

Good job, but some more work is required

framework/deproxy_client.py

framework/deproxy_server.py

malformed/test_purge_hch.py

malformed/test_malformed_crlfs.py

helpers/tistream.py

malformed/test_malformed_crlfs.py

malformed/test_purge_hch.py

…/tempesta-test into dg-tests-1061-1535

Dmitry-Gouriev · 2022-02-10T17:22:48Z

At this point I have finished all subtasks and known comments.

Except this one:

Tests for #1061 and #1535 issue #195 (review) // TRY to reproduce

However @byko3y promised to help with it.

byko3y

Found the cause of exception.

byko3y · 2022-02-10T17:28:25Z

helpers/selfproxy.py

+ self.connections.append(forward_conn)
+ forward_conn.create_socket(socket.AF_INET, socket.SOCK_STREAM)
+ forward_conn.bind((self.listen_host, 0))
+ forward_conn.connect((self.forward_host, self.forward_port))


When testing suite and Tempesta FW are on different hosts, so following thing happens:

forward_conn.bind(('127.0.0.1', 0)) forward_conn.connect(('192.168.122.160', 443))

Which fails with EINVAL on my setup. Manually hardcoding the forward_conn.bind(('192.168.122.1', 0)) fixes the problem.
Also, for some reason the problematic code is triggered by very few tests, particulary malformed.test_chunking_example.CorrectHeadersTest.test_ssl.

@byko3y
Thank you very much!

Also, for some reason the problematic code is triggered by very few tests

Because it is too new tool

Dmitry-Gouriev · 2022-02-10T18:20:54Z

@byko3y
Can you please run the test in question at your machine with the latest update?

Dmitry-Gouriev · 2022-02-10T18:44:18Z

@krizhanovsky
It seems everything has finished.

There is some problem with example test malformed/test_chunking_example.py.
With it chunking nature it runs about 4 minutes.
In the same time it does not check smth important.
And also examples dedicated to tls/ tests are created by subclassing and so repeate some tests from tls/.
May be make it test normally disabled with tests_disabled.json ?

byko3y · 2022-02-10T19:48:49Z

Revision 3cdc3b2:

test_csel_chunking (malformed.test_chunking_example.CertSelectChunkingExpampleTest) ... Node Server is available
Node Tempesta is available
Node Server is available
Node Tempesta is available
FAIL
^C
======================================================================
FAIL: test_csel_chunking (malformed.test_chunking_example.CertSelectChunkingExpampleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/shared/tempesta-test/malformed/test_chunking_example.py", line 294, in test_csel_chunking
    self.iterate_test(self.inner_test_csel_chunking, 127)
  File "/home/user/shared/tempesta-test/malformed/test_chunking_example.py", line 49, in iterate_test
    test_func(CHUNK_SIZES[i], *args, **kwargs)
  File "/home/user/shared/tempesta-test/malformed/test_chunking_example.py", line 291, in inner_test_csel_chunking
    self.test_vhost_cert_selection()
  File "/home/user/shared/tempesta-test/tls/test_tls_cert.py", line 399, in test_vhost_cert_selection
    "Cannot start Tempesta")
AssertionError: Cannot start Tempesta

----------------------------------------------------------------------

Dmitry-Gouriev · 2022-02-10T20:07:07Z

@byko3y Thank you!
It seems this is a new error in newly added code.
It seems I am to fix the only this thing for now.

~~Fix Tests for #1061 and #1535 issue #195 (comment)~~

byko3y · 2022-02-11T16:22:03Z

Seems like TLS-related failures were already existing in master branch:

./run_tests.py tls.test_tls_cert

test (tls.test_tls_cert.ECDSA_SHA256_SECP192) ... ok
test (tls.test_tls_cert.ECDSA_SHA256_SECP256) ... FAIL
test (tls.test_tls_cert.ECDSA_SHA384_SECP521) ... ok
test (tls.test_tls_cert.InvalidHash) ... ok
test (tls.test_tls_cert.RSA1024_SHA384) ... FAIL
test (tls.test_tls_cert.RSA2048_SHA512) ... FAIL
test (tls.test_tls_cert.RSA4096_SHA512) ... FAIL
test (tls.test_tls_cert.RSA512_SHA256) ... ok
test (tls.test_tls_cert.StaleCert) ... FAIL
test_vhost_cert_selection (tls.test_tls_cert.TlsCertSelect) ... FAIL

======================================================================
FAIL: test (tls.test_tls_cert.ECDSA_SHA256_SECP256)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/shared/tempesta-test/tls/test_tls_cert.py", line 243, in test
    self.check_good_cert()
  File "/home/user/shared/tempesta-test/tls/test_tls_cert.py", line 72, in check_good_cert
    "Cannot start Tempesta")
AssertionError: Cannot start Tempesta

======================================================================
FAIL: test (tls.test_tls_cert.RSA1024_SHA384)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/shared/tempesta-test/tls/test_tls_cert.py", line 180, in test
    self.check_good_cert()
  File "/home/user/shared/tempesta-test/tls/test_tls_cert.py", line 72, in check_good_cert
    "Cannot start Tempesta")
AssertionError: Cannot start Tempesta

======================================================================
FAIL: test (tls.test_tls_cert.RSA2048_SHA512)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/shared/tempesta-test/tls/test_tls_cert.py", line 160, in test
    self.check_good_cert()
  File "/home/user/shared/tempesta-test/tls/test_tls_cert.py", line 72, in check_good_cert
    "Cannot start Tempesta")
AssertionError: Cannot start Tempesta

======================================================================
FAIL: test (tls.test_tls_cert.RSA4096_SHA512)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/shared/tempesta-test/tls/test_tls_cert.py", line 140, in test
    self.check_good_cert()
  File "/home/user/shared/tempesta-test/tls/test_tls_cert.py", line 72, in check_good_cert
    "Cannot start Tempesta")
AssertionError: Cannot start Tempesta

======================================================================
FAIL: test (tls.test_tls_cert.StaleCert)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/shared/tempesta-test/tls/test_tls_cert.py", line 311, in test
    self.check_good_cert()
  File "/home/user/shared/tempesta-test/tls/test_tls_cert.py", line 72, in check_good_cert
    "Cannot start Tempesta")
AssertionError: Cannot start Tempesta

======================================================================
FAIL: test_vhost_cert_selection (tls.test_tls_cert.TlsCertSelect)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/shared/tempesta-test/tls/test_tls_cert.py", line 394, in test_vhost_cert_selection
    "Cannot start Tempesta")
AssertionError: Cannot start Tempesta

----------------------------------------------------------------------
Ran 10 tests in 22.538s

FAILED (failures=6)

Dmitry-Gouriev · 2022-02-11T16:49:25Z

So the fault was inherited rather than introduced, and shuold not prevent to merge the PR.

Dmitry-Gouriev · 2022-02-11T17:41:01Z

The fault #195 (comment) #195 (comment) moved into issue #204

krizhanovsky · 2022-02-12T18:29:41Z

cache/test_purge_resp_hch.py

+ # to None to check that request is missing
+ deproxy_srv = self.get_server('deproxy')
+ deproxy_srv.segment_size = chunksize
+ #print (deproxy_srv.response)


There is still a commented out print() :) Why don't you use the standard debug printing from the framework?

Print produces more focused informaion than huge debug printing from framework :)
This print is not needed more, I'll delete it.

malformed/test_chunking_example.py

krizhanovsky

Good and huge work! Finally it's good to merge.

Dmitry-Gouriev · 2022-02-14T15:50:08Z

This commit closes issue #115

Dmitry-Gouriev and others added 18 commits December 4, 2021 01:14

Added the 1st revision of the functional test for the #1061 issue

fd958ef

Added TiedStream class to helpers to get raw request text

5c22fb0

(Intermediary commit) Fixed errors in helpers/. 6/10 tests passed.

3d12e0a

Completed simpe (non heavy chanked) tests for #1061

86e3961

Fix a bug

f89a901

Some refactoring

9ebb9f3

Formatting a little (backend descriptions)

347d8db

Refinment to previous

19c3504

Refinment to previous

ece327c

Fix typo

c8e03e4

Remove some garbadge code

9f2015e

Drafts for heavy chunked tests (preliminary commit)

6bc612c

Typo

33e289b

Draft 2 for heavy chunked

e283628

Remove fantasy which is ahead time

69fed64

Completed heavy chunked tests for #1061 issue

f58373e

Promote "heavy chunked" parameters to client config

71df582

Added heavy chunked test for bug in ss_skb_chop_head_tail() mentioned…

c81130d

… in #1535

Dmitry-Gouriev requested a review from krizhanovsky December 8, 2021 03:48

Dmitry-Gouriev mentioned this pull request Dec 8, 2021

Process skb chain in ss_skb_chop_head_tail() tempesta-tech/tempesta#1535

Closed

Dmitry-Gouriev added 2 commits December 8, 2021 07:34

Update README.md

873b2ae

Fixed misprint in README.md

Corrected misprint in README.md

a5386a3

krizhanovsky requested changes Dec 8, 2021

View reviewed changes

krizhanovsky requested a review from a user December 8, 2021 19:58

krizhanovsky assigned krizhanovsky and ghost Dec 8, 2021

ghost reviewed Dec 8, 2021

View reviewed changes

malformed/test_purge_hch.py Outdated Show resolved Hide resolved

Dmitry-Gouriev added 3 commits December 9, 2021 21:44

Some useful renaming

28cae11

Merge branch 'dg-tests-1061-1535' of https://github.com/tempesta-tech…

66e7b23

…/tempesta-test into dg-tests-1061-1535

Fixed Coding style & Copygight

b23a961

Dmitry-Gouriev added 6 commits February 10, 2022 05:19

Chunking examples for tls/ tests (draft)

b2558c9

Syntax

781ae79

Syntax, comments++

078fb14

Improve comments

57e3884

Chunking examples for tls/ tests - debugged

69c6ced

Added modules to __init__.py files

073013b

byko3y reviewed Feb 10, 2022

View reviewed changes

Dmitry-Gouriev added 3 commits February 10, 2022 17:29

Merge branch 'master' into dg-tests-1061-1535

f78f77b

Fix for failed connection bug (wrong bind())

0292412

Another fix for failed connection bug (wrong bind())

3cdc3b2

Dmitry-Gouriev requested review from krizhanovsky and byko3y February 10, 2022 18:44

Fixed typo

ebde26b

Dmitry-Gouriev mentioned this pull request Feb 11, 2022

Testsuite faults in multi-host environment. #204

Open

krizhanovsky reviewed Feb 12, 2022

View reviewed changes

krizhanovsky approved these changes Feb 12, 2022

View reviewed changes

Remove debug print

c0a0372

Dmitry-Gouriev merged commit 656fcb1 into master Feb 14, 2022

Dmitry-Gouriev mentioned this pull request Feb 14, 2022

Deproxy: chunked network transmission #115

Closed

krizhanovsky deleted the dg-tests-1061-1535 branch June 23, 2022 21:52

krizhanovsky mentioned this pull request Apr 7, 2023

Send HTTP response in small chunks and forward it to an h2 client #431

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tests for #1061 and #1535 issue #195

Tests for #1061 and #1535 issue #195

Dmitry-Gouriev commented Dec 8, 2021 •

edited

Loading

krizhanovsky left a comment

Dmitry-Gouriev commented Feb 10, 2022 •

edited

Loading

byko3y left a comment

byko3y Feb 10, 2022

Dmitry-Gouriev Feb 10, 2022 •

edited

Loading

Dmitry-Gouriev commented Feb 10, 2022

Dmitry-Gouriev commented Feb 10, 2022

byko3y commented Feb 10, 2022

Dmitry-Gouriev commented Feb 10, 2022 •

edited

Loading

byko3y commented Feb 11, 2022

Dmitry-Gouriev commented Feb 11, 2022 •

edited

Loading

Dmitry-Gouriev commented Feb 11, 2022

krizhanovsky Feb 12, 2022

Dmitry-Gouriev Feb 14, 2022

krizhanovsky left a comment

Dmitry-Gouriev commented Feb 14, 2022

Tests for #1061 and #1535 issue #195

Tests for #1061 and #1535 issue #195

Conversation

Dmitry-Gouriev commented Dec 8, 2021 • edited Loading

krizhanovsky left a comment

Choose a reason for hiding this comment

Dmitry-Gouriev commented Feb 10, 2022 • edited Loading

byko3y left a comment

Choose a reason for hiding this comment

byko3y Feb 10, 2022

Choose a reason for hiding this comment

Dmitry-Gouriev Feb 10, 2022 • edited Loading

Choose a reason for hiding this comment

Dmitry-Gouriev commented Feb 10, 2022

Dmitry-Gouriev commented Feb 10, 2022

byko3y commented Feb 10, 2022

Dmitry-Gouriev commented Feb 10, 2022 • edited Loading

byko3y commented Feb 11, 2022

Dmitry-Gouriev commented Feb 11, 2022 • edited Loading

Dmitry-Gouriev commented Feb 11, 2022

krizhanovsky Feb 12, 2022

Choose a reason for hiding this comment

Dmitry-Gouriev Feb 14, 2022

Choose a reason for hiding this comment

krizhanovsky left a comment

Choose a reason for hiding this comment

Dmitry-Gouriev commented Feb 14, 2022

Dmitry-Gouriev commented Dec 8, 2021 •

edited

Loading

Dmitry-Gouriev commented Feb 10, 2022 •

edited

Loading

Dmitry-Gouriev Feb 10, 2022 •

edited

Loading

Dmitry-Gouriev commented Feb 10, 2022 •

edited

Loading

Dmitry-Gouriev commented Feb 11, 2022 •

edited

Loading