Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kernel waring on t_stress.test_stress.TestCtrlFrameFloodSettingsFlood.test #2267

Open
EvgeniiMekhanik opened this issue Oct 18, 2024 · 0 comments
Open

Kernel waring on t_stress.test_stress.TestCtrlFrameFloodSettingsFlood.test #2267

EvgeniiMekhanik opened this issue Oct 18, 2024 · 0 comments
Assignees
@EvgeniiMekhanik
Labels
bug
Milestone
0.8 - Beta

Comments

@EvgeniiMekhanik
Copy link
Contributor

EvgeniiMekhanik commented Oct 18, 2024 •
edited
Loading

Describe the issue
Kernel warning occurs on t_stress.test_stress.TestCtrlFrameFloodSettingsFlood.test on remote CI
Also all TestCtrlFrameFlood tests doesn't pass on remote CI

Expected Behavior
Tests passed

To Reproduce
run t_stress.test_stress.TestCtrlFrameFloodSettingsFlood.test on remote CI

Version or commit hash
Tempesta FW 717cbd0 (current master also)
tempesta-test 9a4a485344767239f82bb9fb7b2e10b2d1e96648 (current master also)

Stacktrace or debug log

WARNING: CPU: 1 PID: 0 at net/ipv4/tcp_output.c:3290 __tcp_retransmit_skb+0x7bf/0x890
Oct 18 05:50:18 192.168.50.83 [46393.731084] Modules linked in: tempesta_fw(OE) tempesta_db(OE) tempesta_tls(OE) tempesta_lib(OE) xt_mark tcp_diag inet_diag xt_nat xt_tcpudp veth sha256_ssse3 sha512_ssse3 unix_diag tls vhost_vsock vmw_vsock_virtio_transport_common vhost vhost_iotlb vsock xt_conntrack xt_MASQUERADE nf_conntrack_netlink xfrm_user xfrm_algo xt_addrtype nft_compat nft_masq nft_counter nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 bridge stp llc nf_tables nfnetlink overlay sch_fq_codel binfmt_misc kvm_amd ccp kvm joydev input_leds serio_raw mac_hid qemu_fw_cfg netconsole dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua msr ramoops reed_solomon efi_pstore ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid crct10dif_pclmul crc32_pclmul bochs_drm ghash_clmulni_intel drm_vram_helper drm_ttm_helper ttm drm_kms_helper syscopyarea
Oct 18 05:50:18 192.168.50.83 [46393.731126]  sysfillrect virtio_net sysimgblt fb_sys_fops aesni_intel virtio_scsi net_failover cec crypto_simd failover i2c_piix4 drm cryptd glue_helper psmouse pata_acpi floppy [last unloaded: tempesta_lib]
Oct 18 05:50:18 192.168.50.83 [46393.742897] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G           OE     5.10.35.tfw-fa7cd2d #1
Oct 18 05:50:18 192.168.50.83 [46393.744234] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.15.0-0-g2dd4b9b3f840-prebuilt.qemu.org 04/01/2014
Oct 18 05:50:18 192.168.50.83 [46393.747024] RIP: 0010:__tcp_retransmit_skb+0x7bf/0x890
Oct 18 05:50:18 192.168.50.83 [46393.748387] Code: 00 80 64 02 03 d7 4d 89 b7 60 01 00 00 e9 81 fe ff ff 4d 89 a7 c0 07 00 00 e9 df fd ff ff 4d 89 a7 20 08 00 00 e9 83 fd ff ff <0f> 0b c7 45 8c ea ff ff ff e9 29 fa ff ff ba 01 00 00 00 4c 89 f6
Oct 18 05:50:18 192.168.50.83 [46393.751306] RSP: 0018:ffffba70400d8d38 EFLAGS: 00010297
Oct 18 05:50:18 192.168.50.83 [46393.752979] RAX: 000000005a3749de RBX: ffff9de4cff64c00 RCX: 0000000000000001
Oct 18 05:50:18 192.168.50.83 [46393.754524] RDX: 000000005a374c1b RSI: ffff9de652944880 RDI: ffff9de4cff64c00
Oct 18 05:50:18 192.168.50.83 [46393.756045] RBP: ffffba70400d8db8 R08: 0000000000000000 R09: 000000000000000f
Oct 18 05:50:18 192.168.50.83 [46393.757555] R10: 0000000000000004 R11: 000000000000000c R12: ffff9de652944880
Oct 18 05:50:18 192.168.50.83 [46393.758904] R13: 0000000000000001 R14: ffffffffa76d8c00 R15: ffff9de4cff64c00
Oct 18 05:50:18 192.168.50.83 [46393.759817] FS:  0000000000000000(0000) GS:ffff9de6f7c80000(0000) knlGS:0000000000000000
Oct 18 05:50:18 192.168.50.83 [46393.761049] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct 18 05:50:18 192.168.50.83 [46393.762252] CR2: 00007f668cbdca70 CR3: 00000001104d8005 CR4: 0000000000770ee0
Oct 18 05:50:18 192.168.50.83 [46393.763456] PKRU: 55555554
Oct 18 05:50:18 192.168.50.83 [46393.764661] Call Trace:
Oct 18 05:50:18 192.168.50.83 [46393.765840]  <IRQ>
Oct 18 05:50:18 192.168.50.83 [46393.767023]  ? ip_protocol_deliver_rcu+0x30/0x1b0
Oct 18 05:50:18 192.168.50.83 [46393.768195]  tcp_retransmit_skb+0x19/0xc0
Oct 18 05:50:18 192.168.50.83 [46393.769371]  tcp_retransmit_timer+0x3d9/0xa80
Oct 18 05:50:18 192.168.50.83 [46393.770524]  ? kvm_clock_get_cycles+0x11/0x20
Oct 18 05:50:18 192.168.50.83 [46393.771653]  ? ktime_get+0x3e/0xa0
Oct 18 05:50:18 192.168.50.83 [46393.772782]  tcp_write_timer_handler+0x152/0x240
Oct 18 05:50:18 192.168.50.83 [46393.773891]  tcp_write_timer+0x9e/0xe0
Oct 18 05:50:18 192.168.50.83 [46393.774975]  ? tcp_write_timer_handler+0x240/0x240
Oct 18 05:50:18 192.168.50.83 [46393.776048]  call_timer_fn+0x2e/0x100
Oct 18 05:50:18 192.168.50.83 [46393.777124]  __run_timers.part.0+0x1e0/0x250
Oct 18 05:50:18 192.168.50.83 [46393.778151]  ? lapic_next_deadline+0x2c/0x40
Oct 18 05:50:18 192.168.50.83 [46393.779148]  ? clockevents_program_event+0x8f/0xe0
Oct 18 05:50:18 192.168.50.83 [46393.779776]  run_timer_softirq+0x2a/0x50
Oct 18 05:50:18 192.168.50.83 [46393.780735]  __do_softirq+0xd9/0x291
Oct 18 05:50:18 192.168.50.83 [46393.781348]  asm_call_irq_on_stack+0x12/0x20
Oct 18 05:50:18 192.168.50.83 [46393.782259]  </IRQ>
Oct 18 05:50:18 192.168.50.83 [46393.782861]  do_softirq_own_stack+0x3d/0x50
Oct 18 05:50:18 192.168.50.83 [46393.783747]  irq_exit_rcu+0xa4/0xb0
Oct 18 05:50:18 192.168.50.83 [46393.784346]  sysvec_apic_timer_interrupt+0x3d/0x90
@krizhanovsky krizhanovsky added this to the 0.8 - Beta milestone Oct 18, 2024
EvgeniiMekhanik added a commit that referenced this issue Oct 29, 2024
@EvgeniiMekhanik
Clear socket write queue and send RST if connection is dropped
ccae06b 
Connection can be dropped by calling `ss_conn_drop_guard_exit`
from several places:
- from `tcp_done, when socket is already DEAD
- from `__sk_close_locked` if all data is already sent
- from `ss_linkerror`

In all places socket write queue is already empty, except one
place, when we call `ss_linkerror` for already shutdowned connection.
In this case we should close socket using TCP RST, because we can't
send any more data after connection will be dropped, moreover sequence
numbers of skbs in socket write queue is not correct (they should be
corrected during tls encryption, but appropriate function will never
called for already dropped connection).

Part of #2267
EvgeniiMekhanik added a commit that referenced this issue Oct 29, 2024
@EvgeniiMekhanik
Clear socket write queue and send RST if connection is dropped
82d0941 
Connection can be dropped by calling `ss_conn_drop_guard_exit`
from several places:
- from `tcp_done, when socket is already DEAD
- from `__sk_close_locked` if all data is already sent
- from `ss_linkerror`

In all places socket write queue is already empty, except one
place, when we call `ss_linkerror` for already shutdowned connection.
In this case we should close socket using TCP RST, because we can't
send any more data after connection will be dropped, moreover sequence
numbers of skbs in socket write queue is not correct (they should be
corrected during tls encryption, but appropriate function will never
called for already dropped connection).

Part of #2267
EvgeniiMekhanik added a commit that referenced this issue Oct 30, 2024
@EvgeniiMekhanik
Clear socket write queue and send RST if connection is dropped
ad7cf12 
Connection can be dropped by calling `ss_conn_drop_guard_exit`
from several places:
- from `tcp_done, when socket is already DEAD
- from `__sk_close_locked` if all data is already sent
- from `ss_linkerror`

In all places socket write queue is already empty, except one
place, when we call `ss_linkerror` for already shutdowned connection.
In this case we should close socket using TCP RST, because we can't
send any more data after connection will be dropped, moreover sequence
numbers of skbs in socket write queue is not correct (they should be
corrected during tls encryption, but appropriate function will never
called for already dropped connection).

Part of #2267
EvgeniiMekhanik added a commit that referenced this issue Oct 30, 2024
@EvgeniiMekhanik
Clear socket write queue and send RST if connection is dropped
e7c3946 
Connection can be dropped by calling `ss_conn_drop_guard_exit`
from several places:
- from `tcp_done, when socket is already DEAD
- from `__sk_close_locked` if all data is already sent
- from `ss_linkerror`

In all places socket write queue is already empty, except one
place, when we call `ss_linkerror` for already shutdowned connection.
In this case we should close socket using TCP RST, because we can't
send any more data after connection will be dropped, moreover sequence
numbers of skbs in socket write queue is not correct (they should be
corrected during tls encryption, but appropriate function will never
called for already dropped connection).

Part of #2267
EvgeniiMekhanik added a commit that referenced this issue Nov 8, 2024
@EvgeniiMekhanik
Clear socket write queue and send RST if connection is dropped
c78acef 
Connection can be dropped by calling `ss_conn_drop_guard_exit`
from several places:
- from `tcp_done, when socket is already DEAD
- from `__sk_close_locked` if all data is already sent
- from `ss_linkerror`

In all places socket write queue is already empty, except one
place, when we call `ss_linkerror` for already shutdowned connection.
In this case we should close socket using TCP RST, because we can't
send any more data after connection will be dropped, moreover sequence
numbers of skbs in socket write queue is not correct (they should be
corrected during tls encryption, but appropriate function will never
called for already dropped connection).

Part of #2267
EvgeniiMekhanik added a commit that referenced this issue Nov 11, 2024
@EvgeniiMekhanik
Clear socket write queue and send RST if connection is dropped
f45e7b1 
Connection can be dropped by calling `ss_conn_drop_guard_exit`
from several places:
- from `tcp_done, when socket is already DEAD
- from `__sk_close_locked` if all data is already sent
- from `ss_linkerror`

In all places socket write queue is already empty, except one
place, when we call `ss_linkerror` for already shutdowned connection.
In this case we should close socket using TCP RST, because we can't
send any more data after connection will be dropped, moreover sequence
numbers of skbs in socket write queue is not correct (they should be
corrected during tls encryption, but appropriate function will never
called for already dropped connection).

Part of #2267
EvgeniiMekhanik added a commit that referenced this issue Nov 13, 2024
@EvgeniiMekhanik
Clear socket write queue and send RST if connection is dropped
2aaa181 
Connection can be dropped by calling `ss_conn_drop_guard_exit`
from several places:
- from `tcp_done, when socket is already DEAD
- from `__sk_close_locked` if all data is already sent
- from `ss_linkerror`

In all places socket write queue is already empty, except one
place, when we call `ss_linkerror` for already shutdowned connection.
In this case we should close socket using TCP RST, because we can't
send any more data after connection will be dropped, moreover sequence
numbers of skbs in socket write queue is not correct (they should be
corrected during tls encryption, but appropriate function will never
called for already dropped connection).

Part of #2267
EvgeniiMekhanik added a commit that referenced this issue Nov 13, 2024
@EvgeniiMekhanik
Clear socket write queue and send RST if connection is dropped
ef12112 
Connection can be dropped by calling `ss_conn_drop_guard_exit`
from several places:
- from `tcp_done, when socket is already DEAD
- from `__sk_close_locked` if all data is already sent
- from `ss_linkerror`

In all places socket write queue is already empty, except one
place, when we call `ss_linkerror` for already shutdowned connection.
In this case we should close socket using TCP RST, because we can't
send any more data after connection will be dropped, moreover sequence
numbers of skbs in socket write queue is not correct (they should be
corrected during tls encryption, but appropriate function will never
called for already dropped connection).

Part of #2267
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@EvgeniiMekhanik EvgeniiMekhanik

Labels
bug
Projects
None yet
Milestone
0.8 - Beta
Development

No branches or pull requests
2 participants
@krizhanovsky @EvgeniiMekhanik