Pcap++ test issue related to the RSS hash function 0x41 #158

sabhishepalc · 2019-01-30T05:34:51Z

Here is the snippet of when we are trying to run the Bin/Pcap++ test.EAL: Detected 24 lcore(s)
EAL: Probing VFIO support...
EAL: PCI device 0000:01:00.0 on NUMA socket -1
EAL: probe driver: 8086:10d3 net_e1000_em
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: PCI device 0000:04:00.1 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
PMD: eth_i40e_dev_init(): FW 6.0 API 1.7 NVM 06.00.01 eetrack 800035da
[src/DpdkDeviceList.cpp : initDpdkDevices : line:164 ] Found 1 DPDK ports. Constructing DpdkDevice for each one
[src/DpdkDevice.cpp : initMemPool : line:623 ] Successfully initialized packets pool of size [16383] for device [DPDK_0]
[src/DpdkDevice.cpp : setDeviceInfo : line:722 ] Device [DPDK_0] has 320 RX queues
[src/DpdkDevice.cpp : setDeviceInfo : line:723 ] Device [DPDK_0] has 320 TX queues
[src/DpdkDeviceList.cpp : initDpdkDevices : line:175 ] DpdkDevice #0: Name='DPDK_0', PCI-slot='0000:04:00.1', PMD='net_i40e', MAC Addr='3c:fd:fe:c3:38:d9'
PMD 'net_i40e' doesn't support the request RSS hash functions 0x41
TestDpdkDevice : FAILED. assertion failed: Cannot open DPDK device
PMD 'net_i40e' doesn't support the request RSS hash functions 0x41
TestDpdkMultiThread : FAILED. assertion failed: Cannot open DPDK device 'DPDK_0' with 16 RX queues
[src/DpdkDevice.cpp : close : line:455 ] Trying to close device [DPDK_0] but device is already closed
PMD 'net_i40e' doesn't support the request RSS hash functions 0x41
TestDpdkDeviceSendPackets : FAILED. assertion failed: Cannot open DPDK device 'DPDK_0' with 320 TX queues
[src/DpdkDevice.cpp : close : line:455 ] Trying to close device [DPDK_0] but device is already closed
PMD 'net_i40e' doesn't support the request RSS hash functions 0x41
TestDpdkMbufRawPacket : FAILED. assertion failed: Cannot open DPDK device
TestDpdkDeviceWorkerThreads : FAILED. assertion failed: Couldn't open DPDK device

Also, about the setup script, I did the setup complete. Here is the current status:
root@ubuntu:~/PcapPlusPlus# ./setup-dpdk.sh -s

PcapPlusPlus setup DPDK script

Network devices using DPDK-compatible driver

0000:04:00.1 'Ethernet Controller XL710 for 40GbE QSFP+' drv=igb_uio unused=uio_pci_generic

Network devices using kernel driver

0000:01:00.0 '82574L Gigabit Network Connection' if=eth0 drv=e1000e unused=igb_uio,uio_pci_generic Active

Other network devices

0000:04:00.0 'Ethernet Controller XL710 for 40GbE QSFP+' unused=igb_uio,uio_pci_generic

Crypto devices using DPDK-compatible driver

Crypto devices using kernel driver

Other crypto devices

And we are planning to use one port for now, and we are getting this error:

PMD 'net_i40e' doesn't support the request RSS hash functions 0x41
TestDpdkDevice : FAILED. assertion failed: Cannot open DPDK device
PMD 'net_i40e' doesn't support the request RSS hash functions 0x41
TestDpdkMultiThread : FAILED. assertion failed: Cannot open DPDK device 'DPDK_0' with 16 RX queues
[src/DpdkDevice.cpp : close : line:455 ] Trying to close device [DPDK_0] but device is already closed
PMD 'net_i40e' doesn't support the request RSS hash functions 0x41
TestDpdkDeviceSendPackets : FAILED. assertion failed: Cannot open DPDK device 'DPDK_0' with 320 TX queues
[src/DpdkDevice.cpp : close : line:455 ] Trying to close device [DPDK_0] but device is already closed
PMD 'net_i40e' doesn't support the request RSS hash functions 0x41
TestDpdkMbufRawPacket : FAILED. assertion failed: Cannot open DPDK device
TestDpdkDeviceWorkerThreads : FAILED. assertion failed: Couldn't open DPDK device

I have few queries:

I can see the traffic moving from one side to another, I am trying to test with the following command:
root@ubuntu:~/PcapPlusPlus/Tests/Pcap++Test# Bin/Pcap++Test -i x.x.x.x -r x.x.x.x -k p1p2 -d
May I know why this is happening, why the TestDpdkWorkerThreads test is failing?
About the RSS hash 0x41, are there any such hardware constraint which can or cannot support the function?
We have classic DPDK huge pages of 1 GB and 16 of them, and while setup I put -p as 16 only.

Need ur help in this scenario.

echo-Mike · 2019-01-30T06:20:00Z

Hi, @sabhishepalc

It is not a question
A couple of reasons:
- Your NIC don't support hash function 0x41 (refer to file rte_ethdev.h in your DPDK distribution)
- Your NIC don't support openening of all 16 RX and 320 TX queues simultaneously (refer to your NIC user manual)
The pure DPDK way to obtain RSS hash functions supported by NIC is to do next steps:
- Call rte_eth_dev_info_get with ID of NIC of interest and info struct rte_eth_dev_info
- Check rte_eth_dev_info.flow_type_rss_offloads of provided struct with values defined in rte_ethdev.h (ETH_RSS_*)
- When NIC is initialized by calling rte_eth_dev_configure provide rte_eth_conf.rx_adv_conf.rss_conf.rss_hf field with ONLY supported hash function!!!
I don't know answer. Haven't used PcapPP in long time

About 3: I don't know the PcapPP way to provide this arguments with same reason as 4. But i gave to you the DPDK functions to search in PcapPP code base, be curious abot it.

sabhishepalc · 2019-01-30T06:51:03Z

Hi, @sabhishepalc

It is not a question

A couple of reasons:

Your NIC don't support hash function 0x41 (refer to file rte_ethdev.h in your DPDK distribution)

Your NIC don't support openening of all 16 RX and 320 TX queues simultaneously (refer to your NIC user manual)

The pure DPDK way to obtain RSS hash functions supported by NIC is to do next steps:

Call rte_eth_dev_info_get with ID of NIC of interest and info struct rte_eth_dev_info

Check rte_eth_dev_info.flow_type_rss_offloads of provided struct with values defined in rte_ethdev.h (ETH_RSS_*)

When NIC is initialized by calling rte_eth_dev_configure provide rte_eth_conf.rx_adv_conf.rss_conf.rss_hf field with ONLY supported hash function!!!

I don't know answer. Haven't used PcapPP in long time

About 3: I don't know the PcapPP way to provide this arguments with same reason as 4. But i gave to you the DPDK functions to search in PcapPP code base, be curious abot it.

Thank you for the prompt response, the first one where I specified the command, my question was: "If my command is correct, the way I have specified the port in the cmd args".

Digressing from the topic a bit but it's about the port only, if you could help:

With this command I can get the list of DPDK enabled ports:

root@ubuntu:~/PcapPlusPlus/Examples/DpdkExample-FilterTraffic# Bin/DpdkTrafficFilter -l
EAL: Detected 24 lcore(s)
EAL: Probing VFIO support...
EAL: PCI device 0000:01:00.0 on NUMA socket -1
EAL: probe driver: 8086:10d3 net_e1000_em
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: PCI device 0000:04:00.1 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
PMD: eth_i40e_dev_init(): FW 6.0 API 1.7 NVM 06.00.01 eetrack 800035da
DPDK port list:
Port #0: MAC address='3c:fd:fe:c3:38:d9'; PCI address='0000:04:00.1'; PMD='net_i40e'

But when I am trying to launch:
root@ubuntu:~/PcapPlusPlus/Examples/DpdkExample-FilterTraffic# Bin/DpdkTrafficFilter -d Port_#0

Error: DPDK ports list is invalid

Although the usage menu says the following:
Usage:

DpdkTrafficFilter [-hvl] [-s PORT] [-f FILENAME] [-i IPV4_ADDR] [-I IPV4_ADDR] [-p PORT] [-P PORT] [-r PROTOCOL]
[-c CORE_MASK] [-m POOL_SIZE] -d PORT_1,PORT_3,...,PORT_N

Options:

-h|--help                                  : Displays this help message and exits
-v|--version                               : Displays the current version and exits
-l|--list                                  : Print the list of DPDK ports and exists
-d|--dpdk-ports PORT_1,PORT_3,...,PORT_N   : A comma-separated list of DPDK port numbers to receive packets from.
                                             To see all available DPDK ports use the -l switch

Not working.

In this function, please find the last print statement:
*/
void listDpdkPorts()
{
CoreMask coreMaskToUse = getCoreMaskForAllMachineCores();

    // initialize DPDK
    if (!DpdkDeviceList::initDpdk(coreMaskToUse, DEFAULT_MBUF_POOL_SIZE))
    {
            EXIT_WITH_ERROR("couldn't initialize DPDK");
    }

    printf("DPDK port list:\n");

    // go over all available DPDK devices and print info for each one
    vector<DpdkDevice*> deviceList = DpdkDeviceList::getInstance().getDpdkDeviceList();
    for (vector<DpdkDevice*>::iterator iter = deviceList.begin(); iter != deviceList.end(); iter++)
    {
            DpdkDevice* dev = *iter;
            printf("    Port #%d: MAC address='%s'; PCI address='%s'; PMD='%s'\n",
                            dev->getDeviceId(),
                            dev->getMacAddress().toString().c_str(),
                            dev->getPciAddress().toString().c_str(),
                            dev->getPMDName().c_str());
    }

}

My query:

Can you suggest me the format in which the port need to specified? Assuming you have used it before.

echo-Mike · 2019-01-30T06:57:15Z

@sabhishepalc
If i remember correctly you just need indexes of DPDK devices starting with 0, like this for example (use DPDK device 0, 3 and 7):

root@ubuntu:~/PcapPlusPlus/Examples/DpdkExample-FilterTraffic# Bin/DpdkTrafficFilter -d 0,3,7

And please can You use Preview button. It is realy hard to understand what You are writing with all this GitHub formatting. Thanks.

sabhishepalc · 2019-01-30T07:05:36Z

@sabhishepalc
If i remember correctly you just need indexes of DPDK devices starting with 0, like this for example (use DPDK device 0, 3 and 7):

root@ubuntu:~/PcapPlusPlus/Examples/DpdkExample-FilterTraffic# Bin/DpdkTrafficFilter -d 0,3,7

And please can You use Preview button. It is realy hard to understand what You are writing with all this GitHub formatting. Thanks.

Thank you it worked fine for me regarding this. Sorry for the bad formatting, I will preview from here on:
Btw, the issue reappears:

root@ubuntu:~/PcapPlusPlus/Examples/DpdkExample-FilterTraffic# Bin/DpdkTrafficFilter -d 0
EAL: Detected 24 lcore(s)
EAL: Probing VFIO support...
EAL: PCI device 0000:01:00.0 on NUMA socket -1
EAL: probe driver: 8086:10d3 net_e1000_em
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: PCI device 0000:04:00.1 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
PMD: eth_i40e_dev_init(): FW 6.0 API 1.7 NVM 06.00.01 eetrack 800035da
PMD 'net_i40e' doesn't support the request RSS hash functions 0x41
Application terminated in error: Couldn't open DPDK device #0, PMD 'net_i40e'

So, I wil go ahead and look into DPDK functions which you suggested and can we let this thread be open till the weekend, for future queries regarding the same.

echo-Mike · 2019-01-30T07:15:13Z

@sabhishepalc
I think this is the reason the NIC can't be opened:

PMD 'net_i40e' doesn't support the request RSS hash functions 0x41

Regarding issue management in this repo ask @seladb. I am contributor not maintainer.

seladb · 2019-01-31T06:22:23Z

I think @echo-Mike answered most of your questions (thanks @echo-Mike !), please let me know if you have further questions

sabhishepalc · 2019-02-01T10:28:30Z

Well, thank you guys for helping me out, I am trying to check the live traffic (Mainly ARP), I am trying to run the tutorial example:

Tutorial-LiveTraffic run snippet:

root@ubuntu:/PcapPlusPlus/Examples/Tutorials/Tutorial-LiveTraffic# cp Makefile.non_windows Makefile
root@ubuntu:/PcapPlusPlus/Examples/Tutorials/Tutorial-LiveTraffic# make all
g++ -msse -msse2 -msse3 -Wall -I/usr/local/include/pcapplusplus -I/usr/include/netinet -I/home/administrator/dpdk-stable-16.11.1//build/include -c
main.cpp: In function ‘int main(int, char**)’:
main.cpp:226:65: warning: format ‘%d’ expects argument of type ‘int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=]
printf("\nSending %d packets one by one...\n", packetVec.size());
^
main.cpp:238:46: warning: format ‘%d’ expects argument of type ‘int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=]
printf("%d packets sent\n", packetVec.size());
^
main.cpp:244:54: warning: format ‘%d’ expects argument of type ‘int’, but argument 2 has type ‘size_t {aka long unsigned int}’ [-Wformat=]
printf("\nSending %d packets...\n", packetVec.size());
^
g++ -L/home/administrator/dpdk-stable-16.11.1//build/lib -L/lib64 -static-libstdc++ -o Tutorial-LiveTraffic main.o -lPcap++ -lPacket++ -lCommon++ -lwhole-archive -lrte_pmd_bond -lrte_pmd_vmxnet3_uio -lrte_pmd_virtio -lrte_pmd_enic -lrte_pmd_i40e -lrte_pmd_fm10k -lrte_pmd_ixgbe -lrte_net -lrte_pmg -lrte_pmd_af_packet -lrte_ethdev -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring -lrte_kvargs -lrte_hash -lrte_cmdline -lrt -lm -ldl -lpthread -Wl,-
/usr/bin/ld: cannot find -lrte_pmd_bond
/usr/bin/ld: cannot find -lrte_pmd_vmxnet3_uio
/usr/bin/ld: cannot find -lrte_pmd_virtio
/usr/bin/ld: cannot find -lrte_pmd_enic
/usr/bin/ld: cannot find -lrte_pmd_i40e
/usr/bin/ld: cannot find -lrte_pmd_fm10k
/usr/bin/ld: cannot find -lrte_pmd_ixgbe
/usr/bin/ld: cannot find -lrte_net
/usr/bin/ld: cannot find -lrte_pmd_e1000
/usr/bin/ld: cannot find -lrte_pmd_ring
/usr/bin/ld: cannot find -lrte_pmd_af_packet
/usr/bin/ld: cannot find -lrte_ethdev
/usr/bin/ld: cannot find -lrte_eal
/usr/bin/ld: cannot find -lrte_mbuf
/usr/bin/ld: cannot find -lrte_mempool
/usr/bin/ld: cannot find -lrte_ring
/usr/bin/ld: cannot find -lrte_kvargs
/usr/bin/ld: cannot find -lrte_hash
/usr/bin/ld: cannot find -lrte_cmdline
collect2: error: ld returned 1 exit status
make: *** [all] Error 1

@seladb and @echo-Mike : Can you tell me what am I missing here? Thanks.

echo-Mike · 2019-02-01T13:08:34Z

@sabhishepalc
Thy to check this path for builded DPDK libraries (rte_*.a):

/home/administrator/dpdk-stable-16.11.1//build/lib

If it has none, then you forgot to build your DPDK distribution
And i am pretty sure that example applications are built within main build system of PcapPP so make sure that you build PcapPP correctly

sabhishepalc · 2019-02-01T15:10:45Z

Ohh, it worked nicely, I have one confusion regarding the igb_uio driver.
My current config when running setup-dpdk.sh -s

PcapPlusPlus setup DPDK script

Network devices using DPDK-compatible driver

0000:04:00.1 'Ethernet Controller XL710 for 40GbE QSFP+' drv=uio_pci_generic unused=vfio-pci

Network devices using kernel driver

0000:01:00.0 '82574L Gigabit Network Connection' if=eth0 drv=e1000e unused=vfio-pci,uio_pci_generic Active
0000:04:00.0 'Ethernet Controller XL710 for 40GbE QSFP+' if=p1p1 drv=i40e unused=vfio-pci,uio_pci_generic

Other network devices

Crypto devices using DPDK-compatible driver

Crypto devices using kernel driver

Other crypto devices

Running the following command:

root@ubuntu:~/PcapPlusPlus/Examples/DpdkExample-FilterTraffic# Bin/DpdkTrafficFilter -l
igb_uio driver isn't loaded, DPDK cannot be initialized. Please run <PcapPlusPlus_Root>/setup_dpdk.sh

My query: "Is igb_uio required to run the packetplusplus or it can run with the current binding driver: "uio_pci_generic" ?

Thanks for all the help.

echo-Mike · 2019-02-01T15:22:08Z

@sabhishepalc
Yes, if i remember correctly PcapPP have hard coded check in it's DPDK module for igb_uio driver.
BTW do you corectly setup DPDK? In latest distributions only 3 base drivers are avaliable: igb_uio, kni, vfio and igb_uio is the most used one.
And remember that you must bind NIC to DPDK before usage. I strongly recommend to read scripts in $(RTE_SDK)/usertools to understand DPDK setup process

daoxuans · 2019-02-02T13:12:16Z

@sabhishepalc
The answer to your third question about "PMD 'net_i40e' doesn't support the request RSS hash functions 0x41" is net_i40e not support hass function IPv6.

seladb · 2019-02-03T08:55:01Z

as @echo-Mike said, currently PcapPlusPlus only works with igb_uio driver. Please let me know if "uio_pci_generic" is a critical requirement for you. If it is then can check if this option is feasible

sabhishepalc · 2019-02-04T06:59:24Z

as @echo-Mike said, currently PcapPlusPlus only works with igb_uio driver. Please let me know if "uio_pci_generic" is a critical requirement for you. If it is then can check if this option is feasible

Thanks for the quick response.
We have now loaded the igb_uio driver and the dpdk port is taking this driver.

PcapPlusPlus setup DPDK script

Huge-pages already allocated
p1p2 is already binded to DPDK or doesn't exist. Exiting

Network devices using DPDK-compatible driver

0000:04:00.1 'Ethernet Controller XL710 for 40GbE QSFP+' drv=igb_uio unused=vfio-pci,uio_pci_generic

Network devices using kernel driver

0000:01:00.0 '82574L Gigabit Network Connection' if=eth0 drv=e1000e unused=igb_uio,vfio-pci,uio_pci_generic Active
0000:04:00.0 'Ethernet Controller XL710 for 40GbE QSFP+' if=p1p1 drv=i40e unused=igb_uio,vfio-pci,uio_pci_generic

Other network devices

Crypto devices using DPDK-compatible driver

Crypto devices using kernel driver

Other crypto devices

**Setup completed.

As shidaoxuan mentioned, we have commented the RSS hash function of 0x41 and used the RSS hash of 0x4
With this, if we run the DPDKTraffic filter on this system with 4 RX and 4 TX, we get a segmentation fault -
EAL: Detected 24 lcore(s)
--1151-- REDIR: 0x5c40130 (libc.so.6:index) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--1151-- REDIR: 0x5c44ce0 (libc.so.6:strstr) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--1151-- REDIR: 0x5c5cc90 (libc.so.6:__strstr_sse2_unaligned) redirected to 0x4c31fa0 (strstr)
--1151-- REDIR: 0x5c41f90 (libc.so.6:strnlen) redirected to 0x4c2e080 (strnlen)
--1151-- REDIR: 0x5c53680 (libc.so.6:__GI_strncpy) redirected to 0x4c2e4d0 (__GI_strncpy)
EAL: Probing VFIO support...
--1151-- REDIR: 0x5cc5650 (libc.so.6:__memcpy_chk) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--1151-- REDIR: 0x5d07360 (libc.so.6:__memcpy_chk_ssse3_back) redirected to 0x4c31e90 (__memcpy_chk)
--1151-- REDIR: 0x5c403c0 (libc.so.6:__GI_strcmp) redirected to 0x4c2f200 (__GI_strcmp)
--1151-- REDIR: 0x5c3c220 (libc.so.6:realloc) redirected to 0x4c2ce10 (realloc)
==1151== Warning: noted but unhandled ioctl 0x3b64 with no size/direction hints.
==1151== This could cause spurious value errors to appear.
==1151== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==1151== Warning: noted but unhandled ioctl 0x3b65 with no size/direction hints.
==1151== This could cause spurious value errors to appear.
==1151== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
EAL: no supported IOMMU extensions found!
EAL: VFIO support could not be initialized
EAL: WARNING: Master core has no memory on local socket!
EAL: PCI device 0000:01:00.0 on NUMA socket -1
EAL: probe driver: 8086:10d3 net_e1000_em
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: PCI device 0000:04:00.1 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: pci_uio_alloc_resource(): cannot store uio mmap details
EAL: Error - exiting with code: 1
Cause: Requested device 0000:04:00.1 cannot be used
==1151==
==1151== HEAP SUMMARY:
==1151== in use at exit: 162,945 bytes in 110 blocks
==1151== total heap usage: 2,640 allocs, 2,530 frees, 929,127 bytes allocated
==1151==

root@ubuntu:~/PcapPlusPlus# lsmod | grep vfio
vfio_pci 36474 0
vfio 24873 1 vfio_pci

./DpdkTrafficFilter -d 0 -c 7 -f /root/PcapPlusPlus/ -m 1023
EAL: Detected 24 lcore(s)
EAL: Probing VFIO support...
EAL: no supported IOMMU extensions found!
EAL: VFIO support could not be initialized
EAL: PCI device 0000:01:00.0 on NUMA socket -1
EAL: probe driver: 8086:10d3 net_e1000_em
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: PCI device 0000:04:00.1 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
PMD: eth_i40e_dev_init(): FW 6.0 API 1.7 NVM 06.00.01 eetrack 800035da
Successfully initialized packets pool of size [1023] for device [DPDK_0]
PMD supports the following RSS hash functions : 0x1fbd

Successfully called rte_eth_dev_configure for device [DPDK_0] with 4 RX queues and 4 TX queues
Successfully initialized 4 RX queues for device [DPDK_0]
Successfully initialized 4 TX queues for device [DPDK_0]
Device [DPDK_0] : Link up; Speed: 40000 Mbps; full-duplex
Using core 1
Core configuration:
DPDK device#0: RX-Queue#0; RX-Queue#1; RX-Queue#2; RX-Queue#3; RX-Queue#4; RX-Queue#5; RX-Queue#6; RX-Queue#7; RX-Queue#8;
RX-Queue#9; RX-Queue#10; RX-Queue#11; RX-Queue#12; RX-Queue#13; RX-Queue#14; RX-Queue#15; RX-Queue#16; RX-Queue#17; RX-Queue#18
; RX-Queue#19; RX-Queue#20; RX-Queue#21; RX-Queue#22; RX-Queue#23; RX-Queue#24; RX-Queue#25; RX-Queue#26; RX-Queue#27; RX-Queue
#28; RX-Queue#29; RX-Queue#30; RX-Queue#31; RX-Queue#32; RX-Queue#33; RX-Queue#34; RX-Queue#35; RX-Queue#36; RX-Queue#37; RX-Qu
eue#38; RX-Queue#39; RX-Queue#40; RX-Queue#41; RX-Queue#42; RX-Queue#43; RX-Queue#44; RX-Queue#45; RX-Queue#46; RX-Queue#47; RX
-Queue#48; RX-Queue#49; RX-Queue#50; RX-Queue#51; RX-Queue#52; RX-Queue#53; RX-Queue#54; RX-Queue#55; RX-Queue#56; RX-Queue#57;
RX-Queue#58; RX-Queue#59; RX-Queue#60; RX-Queue#61; RX-Queue#62; RX-Queue#63; RX-Queue#64; RX-Queue#65; RX-Queue#66; RX-Queue#67; RX-Queue#68; RX-Queue#69; RX-Queue#70; RX-Queue#71; RX-Queue#72; RX-Queue#73; RX-Queue#74; RX-Queue#75; RX-Queue#76; RX-Queue#77; RX-Queue#78; RX-Queue#79; RX-Queue#80; RX-Queue#81; RX-Queue#82; RX-Queue#83; RX-Queue#84; RX-Queue#85; RX-Queue#86; RX-Queue#87; RX-Queue#88; RX-Queue#89; RX-Queue#90; RX-Queue#91; RX-Queue#92; RX-Queue#93; RX-Queue#94; RX-Queue#95; RX-Queue#96; RX-Queue#97; RX-Queue#98; RX-Queue#99; RX-Queue#100; RX-Queue#101; RX-Queue#102; RX-Queue#103; RX-Queue#104;
RX-Queue#105; RX-Queue#106; RX-Queue#107; RX-Queue#108; RX-Queue#109; RX-Queue#110; RX-Queue#111; RX-Queue#112;
coming here to icrement i
Using core 2
Core configuration:
DPDK device#0: RX-Queue#160; RX-Queue#161; RX-Queue#162; RX-Queue#163; RX-Queue#164; RX-Queue#165; RX-Queue#166; RX-Queue#167; RX-Queue#168; RX-Queue#169; RX-Queue#170; RX-Queue#171; RX-Queue#172; RX-Queue#173; RX-Queue#174; RX-Queue#175; RX-Queue#176; RX-Queue#177; RX-Queue#178; RX-Queue#179; RX-Queue#180; RX-Queue#181; RX-Queue#182; RX-Queue#183; RX-Queue#184; RX-Queue#185; RX-Queue#186; RX-Queue#187; RX-Queue#188; RX-Queue#189; RX-Queue#190; RX-Queue#191; RX-Queue#192; RX-Queue#193; RX-Queue#194; RX-Queue#195; RX-Queue#196; RX-Queue#197; RX-Queue#198; RX-Queue#199; RX-Queue#200; RX-Queue#201; RX-Queue#202; RX-Queue#203; RX-Queue#204; RX-Queue#205; RX-Queue#206; RX-Queue#207; RX-Queue#208; RX-Queue#209; RX-Queue#210; RX-Queue#211; RX-Queue#212; RX-Queue#213; RX-Queue#214; RX-Queue#215; RX-Queue#216; RX-Queue#217; RX-Queue#218; RX-Queue#219; RX-Queue#220; RX-Queue#221; RX-Queue#222; RX-Queue#223; RX-Queue#224; RX-Queue#225; RX-Queue#226; RX-Queue#227; RX-Queue#228; RX-Queue#229; RX-Queue#230; RX-Queue#231; RX-Queue#232; RX-Queue#233; RX-Queue#234; RX-Queue#235; RX-Queue#236; RX-Queue#237; RX-Queue#238; RX-Queue#239; RX-Queue#240; RX-Queue#241; RX-Queue#242; RX-Queue#243; RX-Queue#244; RX-Queue#245; RX-Queue#246; RX-Queue#247; RX-Queue#248; RX-Queue#249; RX-Queue#250; RX-Queue#251; RX-Queue#252; RX-Queue#253; RX-Queue#254; RX-Queue#255; RX-Queue#256; RX-Queue#257; RX-Queue#258; RX-Queue#259; RX-Queue#260; RX-Queue#261; RX-Queue#262; RX-Queue#263; RX-Queue#264; RX-Queue#265; RX-Queue#266; RX-Queue#267; RX-Queue#268; RX-Queue#269; RX-Queue#270; RX-Queue#271; RX-Queue#272; RX-Queue#273; RX-Queue#274; RX-Queue#275; RX-Queue#276; RX-Queue#277; RX-Queue#278; RX-Queue#279; RX-Queue#280; RX-Queue#281; RX-Queue#282; RX-Queue#283; RX-Queue#284; RX-Queue#285; RX-Queue#286; RX-Queue#287; RX-Queue#288; RX-Queue#289; RX-Queue#290; RX-Queue#291; RX-Queue#292; RX-Queue#293; RX-Queue#294; RX-Queue#295; RX-Queue#296; RX-Queue#297; RX-Queue#298; RX-Queue#299; RX-Queue#300; RX-Queue#301; RX-Queue#302; RX-Queue#303; RX-Queue#304; RX-Queue#305; RX-Queue#306; RX-Queue#307; RX-Queue#308; RX-Queue#309; RX-Queue#310; RX-Queue#311; RX-Queue#312; RX-Queue#313; RX-Queue#314; RX-Queue#315; RX-Queue#316; RX-Queue#317; RX-Queue#318; RX-Queue#319;
Coming here
coming here to icrement i
Coming here to get new worker thread
Coming here to get new worker thread
Segmentation fault (core dumped)

Request you to shed light on the above.
Our hugepage is the same as the standalonr DPDK setup of 1GB and 16 pages.

root@ubuntu:~/PcapPlusPlus# grep HugePages /proc/meminfo
AnonHugePages: 6932480 kB
HugePages_Total: 16
HugePages_Free: 16
HugePages_Rsvd: 0
HugePages_Surp: 0

We have added a few debug comments to see if the code is hitting specific points.
but the valgrind log show that the Device cannot be used.. What does that indicate?
We received the same error for pci_generic driver too.

Thanks in advance. @seladb @echo-Mike @shidaoxuan

seladb · 2019-02-04T08:53:02Z

Something is wrong with the RX queues configuration, it doesn't make sense that your device has more than 300 RX queues. Could you please try to get a stack trace of the segmentation fault?

sabhishepalc · 2019-02-04T09:18:30Z

Something is wrong with the RX queues configuration, it doesn't make sense that your device has more than 300 RX queues. Could you please try to get a stack trace of the segmentation fault?

I believe this can help:

root@ubuntu:~/PcapPlusPlus/Examples/DpdkExample-FilterTraffic# valgrind -v --tool=memcheck --leak-check=full --show-reachable=yes Bin/DpdkTrafficFilter -d 0 -f /root/PcapPlusPlus/ -r UDP
==32584== Memcheck, a memory error detector
==32584== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==32584== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==32584== Command: Bin/DpdkTrafficFilter -d 0 -f /root/PcapPlusPlus/ -r UDP
==32584==
--32584-- Valgrind options:
--32584-- -v
--32584-- --tool=memcheck
--32584-- --leak-check=full
--32584-- --show-reachable=yes
--32584-- Contents of /proc/version:
--32584-- Linux version 3.13.0-45-generic (buildd@phianna) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #74-Ubuntu SMP Tue Jan 13 19:36:28 UTC 2015
--32584-- Arch and hwcaps: AMD64, LittleEndian, amd64-cx16-rdtscp-sse3
--32584-- Page sizes: currently 4096, max supported 4096
--32584-- Valgrind library directory: /usr/lib/valgrind
--32584-- Reading syms from /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter
--32584-- Reading syms from /lib/x86_64-linux-gnu/ld-2.19.so
--32584-- Considering /lib/x86_64-linux-gnu/ld-2.19.so ..
--32584-- .. CRC mismatch (computed c0e20d20 wanted 6a260d6d)
--32584-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/ld-2.19.so ..
--32584-- .. CRC is valid
--32584-- Reading syms from /usr/lib/valgrind/memcheck-amd64-linux
--32584-- Considering /usr/lib/valgrind/memcheck-amd64-linux ..
--32584-- .. CRC mismatch (computed fed8e6c5 wanted e6be554a)
--32584-- object doesn't have a symbol table
--32584-- object doesn't have a dynamic symbol table
--32584-- Scheduler: using generic scheduler lock implementation.
--32584-- Reading suppressions file: /usr/lib/valgrind/default.supp
==32584== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-32584-by-root-on-???
==32584== embedded gdbserver: writing to /tmp/vgdb-pipe-to-vgdb-from-32584-by-root-on-???
==32584== embedded gdbserver: shared mem /tmp/vgdb-pipe-shared-mem-vgdb-32584-by-root-on-???
==32584==
==32584== TO CONTROL THIS PROCESS USING vgdb (which you probably
==32584== don't want to do, unless you know exactly what you're doing,
==32584== or are doing some strange experiment):
==32584== /usr/lib/valgrind/../../bin/vgdb --pid=32584 ...command...
==32584==
==32584== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==32584== /path/to/gdb Bin/DpdkTrafficFilter
==32584== and then give GDB the following command
==32584== target remote | /usr/lib/valgrind/../../bin/vgdb --pid=32584
==32584== --pid is optional if only one valgrind process is running
==32584==
--32584-- REDIR: 0x4019d70 (ld-linux-x86-64.so.2:strlen) redirected to 0x380764b1 (???)
--32584-- Reading syms from /usr/lib/valgrind/vgpreload_core-amd64-linux.so
--32584-- Considering /usr/lib/valgrind/vgpreload_core-amd64-linux.so ..
--32584-- .. CRC mismatch (computed 4bcdfe99 wanted 3143e841)
--32584-- object doesn't have a symbol table
--32584-- Reading syms from /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so
--32584-- Considering /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so ..
--32584-- .. CRC mismatch (computed 3df18bf1 wanted 14fefe1c)
--32584-- object doesn't have a symbol table
==32584== WARNING: new redirection conflicts with existing -- ignoring it
--32584-- old: 0x04019d70 (strlen ) R-> (0000.0) 0x380764b1 ???
--32584-- new: 0x04019d70 (strlen ) R-> (2007.0) 0x04c2e1a0 strlen
--32584-- REDIR: 0x4019b20 (ld-linux-x86-64.so.2:index) redirected to 0x4c2dd50 (index)
--32584-- REDIR: 0x4019d40 (ld-linux-x86-64.so.2:strcmp) redirected to 0x4c2f2f0 (strcmp)
--32584-- REDIR: 0x401aa90 (ld-linux-x86-64.so.2:mempcpy) redirected to 0x4c31da0 (mempcpy)
--32584-- Reading syms from /usr/lib/x86_64-linux-gnu/libpcap.so.1.5.3
--32584-- object doesn't have a symbol table
--32584-- Reading syms from /lib/x86_64-linux-gnu/libpthread-2.19.so
--32584-- Considering /lib/x86_64-linux-gnu/libpthread-2.19.so ..
--32584-- .. CRC mismatch (computed d432accf wanted 317a970d)
--32584-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/libpthread-2.19.so ..
--32584-- .. CRC is valid
--32584-- Reading syms from /lib/x86_64-linux-gnu/librt-2.19.so
--32584-- Considering /lib/x86_64-linux-gnu/librt-2.19.so ..
--32584-- .. CRC mismatch (computed 751a3be9 wanted b8b87a5f)
--32584-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/librt-2.19.so ..
--32584-- .. CRC is valid
--32584-- Reading syms from /lib/x86_64-linux-gnu/libdl-2.19.so
--32584-- Considering /lib/x86_64-linux-gnu/libdl-2.19.so ..
--32584-- .. CRC mismatch (computed ee8e2c67 wanted 20f67a06)
--32584-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/libdl-2.19.so ..
--32584-- .. CRC is valid
--32584-- Reading syms from /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19
--32584-- Considering /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19 ..
--32584-- .. CRC mismatch (computed dc37bb90 wanted ea8c3b40)
--32584-- object doesn't have a symbol table
--32584-- Reading syms from /lib/x86_64-linux-gnu/libgcc_s.so.1
--32584-- Considering /lib/x86_64-linux-gnu/libgcc_s.so.1 ..
--32584-- .. CRC mismatch (computed 6116126e wanted 54e3f1f2)
--32584-- object doesn't have a symbol table
--32584-- Reading syms from /lib/x86_64-linux-gnu/libc-2.19.so
--32584-- Considering /lib/x86_64-linux-gnu/libc-2.19.so ..
--32584-- .. CRC mismatch (computed 26a92edf wanted 7a008b73)
--32584-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.19.so ..
--32584-- .. CRC is valid
--32584-- Reading syms from /lib/x86_64-linux-gnu/libm-2.19.so
--32584-- Considering /lib/x86_64-linux-gnu/libm-2.19.so ..
--32584-- .. CRC mismatch (computed 0511ff25 wanted e2dba104)
--32584-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/libm-2.19.so ..
--32584-- .. CRC is valid
--32584-- REDIR: 0x5c46070 (libc.so.6:strcasecmp) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--32584-- REDIR: 0x5c48360 (libc.so.6:strncasecmp) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--32584-- REDIR: 0x5c45840 (libc.so.6:memcpy@GLIBC_2.2.5) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--32584-- REDIR: 0x5c43ad0 (libc.so.6:rindex) redirected to 0x4c2da30 (rindex)
--32584-- REDIR: 0x5c41dd0 (libc.so.6:strlen) redirected to 0x4c2e0e0 (strlen)
--32584-- REDIR: 0x5c452b0 (libc.so.6:__GI_memcmp) redirected to 0x4c30b80 (__GI_memcmp)
--32584-- REDIR: 0x5c40380 (libc.so.6:strcmp) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--32584-- REDIR: 0x5c53780 (libc.so.6:__strcmp_sse2_unaligned) redirected to 0x4c2f1b0 (strcmp)
--32584-- REDIR: 0x56fdd90 (libstdc++.so.6:operator new(unsigned long)) redirected to 0x4c2b070 (operator new(unsigned long))
--32584-- REDIR: 0x5c4aa40 (libc.so.6:memcpy@@GLIBC_2.14) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--32584-- REDIR: 0x5c53a30 (libc.so.6:__memcpy_sse2_unaligned) redirected to 0x4c2f6b0 (memcpy@@GLIBC_2.14)
--32584-- REDIR: 0x5c3c550 (libc.so.6:calloc) redirected to 0x4c2cbf0 (calloc)
--32584-- REDIR: 0x5c45270 (libc.so.6:bcmp) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--32584-- REDIR: 0x5d1c8e0 (libc.so.6:__memcmp_sse4_1) redirected to 0x4c30c00 (__memcmp_sse4_1)
--32584-- REDIR: 0x5c4cdd0 (libc.so.6:strchrnul) redirected to 0x4c319b0 (strchrnul)
--32584-- REDIR: 0x5c41850 (libc.so.6:__GI_strcpy) redirected to 0x4c2e2a0 (__GI_strcpy)
--32584-- REDIR: 0x5c40160 (libc.so.6:__GI_strchr) redirected to 0x4c2db90 (__GI_strchr)
--32584-- REDIR: 0x56fc0f0 (libstdc++.so.6:operator delete(void*)) redirected to 0x4c2c250 (operator delete(void*))
--32584-- REDIR: 0x5c421f0 (libc.so.6:strncmp) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--32584-- REDIR: 0x5cfdce0 (libc.so.6:__strncmp_ssse3) redirected to 0x4c2e8c0 (strncmp)
--32584-- REDIR: 0xffffffffff600400 (???:???) redirected to 0x3807649d (???)
--32584-- REDIR: 0x5c44f20 (libc.so.6:memchr) redirected to 0x4c2f390 (memchr)
--32584-- REDIR: 0x5c456b0 (libc.so.6:__GI_memmove) redirected to 0x4c31660 (__GI_memmove)
--32584-- REDIR: 0x5c43a90 (libc.so.6:strncpy) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--32584-- REDIR: 0x5c58c20 (libc.so.6:__strncpy_sse2_unaligned) redirected to 0x4c2e770 (__strncpy_sse2_unaligned)
--32584-- REDIR: 0x5d0c910 (libc.so.6:__memmove_ssse3_back) redirected to 0x4c2f450 (memcpy@GLIBC_2.2.5)
--32584-- REDIR: 0x5c3ba80 (libc.so.6:malloc) redirected to 0x4c2ab10 (malloc)
--32584-- REDIR: 0x5c4aa90 (libc.so.6:__GI_memcpy) redirected to 0x4c2fc90 (__GI_memcpy)
--32584-- REDIR: 0x5c3c120 (libc.so.6:free) redirected to 0x4c2bd80 (free)
--32584-- REDIR: 0x56fdea0 (libstdc++.so.6:operator new[](unsigned long)) redirected to 0x4c2b790 (operator new[](unsigned long))
--32584-- REDIR: 0x5c41810 (libc.so.6:strcpy) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--32584-- REDIR: 0x5c585f0 (libc.so.6:__strcpy_sse2_unaligned) redirected to 0x4c2e1c0 (strcpy)
--32584-- REDIR: 0x5c42240 (libc.so.6:__GI_strncmp) redirected to 0x4c2e930 (__GI_strncmp)
--32584-- REDIR: 0x5c44720 (libc.so.6:__GI_strstr) redirected to 0x4c32030 (__strstr_sse2)
EAL: Detected 24 lcore(s)
--32584-- REDIR: 0x5c40130 (libc.so.6:index) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--32584-- REDIR: 0x5c44ce0 (libc.so.6:strstr) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--32584-- REDIR: 0x5c5cc90 (libc.so.6:__strstr_sse2_unaligned) redirected to 0x4c31fa0 (strstr)
--32584-- REDIR: 0x5c41f90 (libc.so.6:strnlen) redirected to 0x4c2e080 (strnlen)
--32584-- REDIR: 0x5c458d0 (libc.so.6:memset) redirected to 0x4c31350 (memset)
--32584-- REDIR: 0x5c53680 (libc.so.6:__GI_strncpy) redirected to 0x4c2e4d0 (__GI_strncpy)
EAL: Probing VFIO support...
--32584-- REDIR: 0x5cc5650 (libc.so.6:__memcpy_chk) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--32584-- REDIR: 0x5d07360 (libc.so.6:__memcpy_chk_ssse3_back) redirected to 0x4c31e90 (__memcpy_chk)
--32584-- REDIR: 0x5c403c0 (libc.so.6:__GI_strcmp) redirected to 0x4c2f200 (__GI_strcmp)
--32584-- REDIR: 0x5c3c220 (libc.so.6:realloc) redirected to 0x4c2ce10 (realloc)
==32584== Warning: noted but unhandled ioctl 0x3b64 with no size/direction hints.
==32584== This could cause spurious value errors to appear.
==32584== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==32584== Warning: noted but unhandled ioctl 0x3b65 with no size/direction hints.
==32584== This could cause spurious value errors to appear.
==32584== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
EAL: no supported IOMMU extensions found!
EAL: VFIO support could not be initialized
EAL: WARNING: Master core has no memory on local socket!
EAL: PCI device 0000:01:00.0 on NUMA socket -1
EAL: probe driver: 8086:10d3 net_e1000_em
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: PCI device 0000:04:00.1 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: pci_uio_alloc_resource(): cannot store uio mmap details
EAL: Error - exiting with code: 1
Cause: Requested device 0000:04:00.1 cannot be used
==32584==
==32584== HEAP SUMMARY:
==32584== in use at exit: 169,553 bytes in 131 blocks
==32584== total heap usage: 2,665 allocs, 2,534 frees, 936,002 bytes allocated
==32584==
==32584== Searching for pointers to 131 not-freed blocks
==32584== Checked 202,582,664 bytes
==32584==
==32584== 4 bytes in 1 blocks are still reachable in loss record 1 of 16
==32584== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x47CBF0: pcpp::IPv4Address::IPv4Address(pcpp::IPv4Address const&) (IpAddress.cpp:64)
==32584== by 0x41FC20: main (main.cpp:279)
==32584==
==32584== 4 bytes in 1 blocks are still reachable in loss record 2 of 16
==32584== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x47CBF0: pcpp::IPv4Address::IPv4Address(pcpp::IPv4Address const&) (IpAddress.cpp:64)
==32584== by 0x41FC34: main (main.cpp:280)
==32584==
==32584== 4 bytes in 1 blocks are still reachable in loss record 3 of 16
==32584== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x428057: __gnu_cxx::new_allocator::allocate(unsigned long, void const*) (new_allocator.h:104)
==32584== by 0x426BD4: std::_Vector_base<int, std::allocator >::_M_allocate(unsigned long) (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x424B0E: std::vector<int, std::allocator >::_M_insert_aux(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator > >, int const&) (vector.tcc:345)
==32584== by 0x42326B: std::vector<int, std::allocator >::push_back(int const&) (stl_vector.h:913)
==32584== by 0x41FDBC: main (main.cpp:310)
==32584==
==32584== 10 bytes in 1 blocks are still reachable in loss record 4 of 16
==32584== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x5C41B49: strdup (strdup.c:42)
==32584== by 0x56B5E2: get_hugepage_dir (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x56B9E8: eal_hugepage_info_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x56AE56: rte_eal_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x436F4A: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:124)
==32584== by 0x420287: main (main.cpp:434)
==32584==
==32584== 16 bytes in 1 blocks are still reachable in loss record 5 of 16
==32584== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x5C41B49: strdup (strdup.c:42)
==32584== by 0x56A2A7: rte_eal_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x436F4A: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:124)
==32584== by 0x420287: main (main.cpp:434)
==32584==
==32584== 26 bytes in 1 blocks are possibly lost in loss record 6 of 16
==32584== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x5759248: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x5759E0A: std::string::_Rep::_M_clone(std::allocator const&, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x5759EA3: std::string::reserve(unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x5714C16: std::basic_istream<char, std::char_traits >& std::operator>><char, std::char_traits, std::allocator >(std::basic_istream<char, std::char_traits >&, std::basic_string<char, std::char_traits, std::allocator >&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x436CDC: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:109)
==32584== by 0x420287: main (main.cpp:434)
==32584==
==32584== 44 bytes in 1 blocks are possibly lost in loss record 7 of 16
==32584== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x5759248: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x575A970: char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator const&, std::forward_iterator_tag) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x575AD87: std::basic_string<char, std::char_traits, std::allocator >::basic_string(char const*, std::allocator const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x41FECE: main (main.cpp:332)
==32584==
==32584== 56 bytes in 1 blocks are still reachable in loss record 8 of 16
==32584== at 0x4C2B800: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x436CA9: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:106)
==32584== by 0x420287: main (main.cpp:434)
==32584==
==32584== 140 bytes in 7 blocks are still reachable in loss record 9 of 16
==32584== at 0x4C2B800: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x436CF0: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:110)
==32584== by 0x420287: main (main.cpp:434)
==32584==
==32584== 192 bytes in 6 blocks are possibly lost in loss record 10 of 16
==32584== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x5759248: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x5759E0A: std::string::_Rep::_M_clone(std::allocator const&, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x5759EA3: std::string::reserve(unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x575A0EE: std::string::append(char const*, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x5714B81: std::basic_istream<char, std::char_traits >& std::operator>><char, std::char_traits, std::allocator >(std::basic_istream<char, std::char_traits >&, std::basic_string<char, std::char_traits, std::allocator >&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x436CDC: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:109)
==32584== by 0x420287: main (main.cpp:434)
==32584==
==32584== 256 bytes in 1 blocks are still reachable in loss record 11 of 16
==32584== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x47E9C5: allocate (new_allocator.h:104)
==32584== by 0x47E9C5: _M_allocate (stl_vector.h:168)
==32584== by 0x47E9C5: std::vector<pcpp::SystemCore, std::allocatorpcpp::SystemCore >::_M_insert_aux(__gnu_cxx::__normal_iterator<pcpp::SystemCore*, std::vector<pcpp::SystemCore, std::allocatorpcpp::SystemCore > >, pcpp::SystemCore const&) (vector.tcc:345)
==32584== by 0x47E902: push_back (stl_vector.h:913)
==32584== by 0x47E902: pcpp::createCoreVectorFromCoreMask(unsigned int, std::vector<pcpp::SystemCore, std::allocatorpcpp::SystemCore >&) (SystemUtils.cpp:163)
==32584== by 0x42023F: main (main.cpp:425)
==32584==
==32584== 280 bytes in 1 blocks are still reachable in loss record 12 of 16
==32584== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x5C275BF: fopencookie@@GLIBC_2.2.5 (iofopncook.c:197)
==32584== by 0x56E308: rte_eal_log_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x56A9EB: rte_eal_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x436F4A: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:124)
==32584== by 0x420287: main (main.cpp:434)
==32584==
==32584== 304 bytes in 1 blocks are possibly lost in loss record 13 of 16
==32584== at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x4012EE4: allocate_dtv (dl-tls.c:296)
==32584== by 0x4012EE4: _dl_allocate_tls (dl-tls.c:460)
==32584== by 0x507DD92: allocate_stack (allocatestack.c:589)
==32584== by 0x507DD92: pthread_create@@GLIBC_2.2.5 (pthread_create.c:500)
==32584== by 0x57387B: rte_eal_intr_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x56AB04: rte_eal_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x436F4A: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:124)
==32584== by 0x420287: main (main.cpp:434)
==32584==
==32584== 537 bytes in 1 blocks are possibly lost in loss record 14 of 16
==32584== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x5759248: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x5759E0A: std::string::_Rep::_M_clone(std::allocator const&, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x5759EA3: std::string::reserve(unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x57381E5: std::basic_stringbuf<char, std::char_traits, std::allocator >::overflow(int) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x573C415: std::basic_streambuf<char, std::char_traits >::xsputn(char const*, long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x5733C04: std::basic_ostream<char, std::char_traits >& std::__ostream_insert<char, std::char_traits >(std::basic_ostream<char, std::char_traits >&, char const*, long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==32584== by 0x436B71: operator<< <std::char_traits > (ostream:535)
==32584== by 0x436B71: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:97)
==32584== by 0x420287: main (main.cpp:434)
==32584==
==32584== 6,992 bytes in 23 blocks are possibly lost in loss record 15 of 16
==32584== at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x4012EE4: allocate_dtv (dl-tls.c:296)
==32584== by 0x4012EE4: _dl_allocate_tls (dl-tls.c:460)
==32584== by 0x507DD92: allocate_stack (allocatestack.c:589)
==32584== by 0x507DD92: pthread_create@@GLIBC_2.2.5 (pthread_create.c:500)
==32584== by 0x56AB96: rte_eal_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x436F4A: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:124)
==32584== by 0x420287: main (main.cpp:434)
==32584==
==32584== 160,688 bytes in 83 blocks are still reachable in loss record 16 of 16
==32584== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==32584== by 0x56FABC: pci_scan_one (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x5706F5: rte_eal_pci_scan (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x570A18: rte_eal_pci_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x56A9F8: rte_eal_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==32584== by 0x436F4A: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:124)
==32584== by 0x420287: main (main.cpp:434)
==32584==
==32584== LEAK SUMMARY:
==32584== definitely lost: 0 bytes in 0 blocks
==32584== indirectly lost: 0 bytes in 0 blocks
==32584== possibly lost: 8,095 bytes in 33 blocks
==32584== still reachable: 161,458 bytes in 98 blocks
==32584== suppressed: 0 bytes in 0 blocks
==32584==
==32584== ERROR SUMMARY: 6 errors from 6 contexts (suppressed: 0 from 0)
==32584== ERROR SUMMARY: 6 errors from 6 contexts (suppressed: 0 from 0)

Will this one work?

seladb · 2019-02-04T09:33:07Z

I'm not sure where I can see the stack trace that caused the segmentation fault. Could you please point me to it?
Also, please put output in a code block so it's easier to read

sabhishepalc · 2019-02-04T11:19:46Z

The issue is very close to this one : http://mails.dpdk.org/archives/dev/2014-December/009234.html

Ran the leak-check on valgrind:

root@ubuntu:~/PcapPlusPlus# valgrind -v --leak-check=yes --leak-check=full --show-leak-kinds=all Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter -d 0 -f /root/PcapPlusPlus/ -r UDP
==7665== Memcheck, a memory error detector
==7665== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==7665== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==7665== Command: Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter -d 0 -f /root/PcapPlusPlus/ -r UDP
==7665==
--7665-- Valgrind options:
--7665-- -v
--7665-- --leak-check=yes
--7665-- --leak-check=full
--7665-- --show-leak-kinds=all
--7665-- Contents of /proc/version:
--7665-- Linux version 3.13.0-45-generic (buildd@phianna) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #74-Ubuntu SMP Tue Jan 13 19:36:28 UTC 2015
--7665-- Arch and hwcaps: AMD64, LittleEndian, amd64-cx16-rdtscp-sse3
--7665-- Page sizes: currently 4096, max supported 4096
--7665-- Valgrind library directory: /usr/lib/valgrind
--7665-- Reading syms from /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter
--7665-- Reading syms from /lib/x86_64-linux-gnu/ld-2.19.so
--7665-- Considering /lib/x86_64-linux-gnu/ld-2.19.so ..
--7665-- .. CRC mismatch (computed c0e20d20 wanted 6a260d6d)
--7665-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/ld-2.19.so ..
--7665-- .. CRC is valid
--7665-- Reading syms from /usr/lib/valgrind/memcheck-amd64-linux
--7665-- Considering /usr/lib/valgrind/memcheck-amd64-linux ..
--7665-- .. CRC mismatch (computed fed8e6c5 wanted e6be554a)
--7665-- object doesn't have a symbol table
--7665-- object doesn't have a dynamic symbol table
--7665-- Scheduler: using generic scheduler lock implementation.
--7665-- Reading suppressions file: /usr/lib/valgrind/default.supp
==7665== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-7665-by-root-on-???
==7665== embedded gdbserver: writing to /tmp/vgdb-pipe-to-vgdb-from-7665-by-root-on-???
==7665== embedded gdbserver: shared mem /tmp/vgdb-pipe-shared-mem-vgdb-7665-by-root-on-???
==7665==
==7665== TO CONTROL THIS PROCESS USING vgdb (which you probably
==7665== don't want to do, unless you know exactly what you're doing,
==7665== or are doing some strange experiment):
==7665== /usr/lib/valgrind/../../bin/vgdb --pid=7665 ...command...
==7665==
==7665== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==7665== /path/to/gdb Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter
==7665== and then give GDB the following command
==7665== target remote | /usr/lib/valgrind/../../bin/vgdb --pid=7665
==7665== --pid is optional if only one valgrind process is running
==7665==
--7665-- REDIR: 0x4019d70 (ld-linux-x86-64.so.2:strlen) redirected to 0x380764b1 (???)
--7665-- Reading syms from /usr/lib/valgrind/vgpreload_core-amd64-linux.so
--7665-- Considering /usr/lib/valgrind/vgpreload_core-amd64-linux.so ..
--7665-- .. CRC mismatch (computed 4bcdfe99 wanted 3143e841)
--7665-- object doesn't have a symbol table
--7665-- Reading syms from /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so
--7665-- Considering /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so ..
--7665-- .. CRC mismatch (computed 3df18bf1 wanted 14fefe1c)
--7665-- object doesn't have a symbol table
==7665== WARNING: new redirection conflicts with existing -- ignoring it
--7665-- old: 0x04019d70 (strlen ) R-> (0000.0) 0x380764b1 ???
--7665-- new: 0x04019d70 (strlen ) R-> (2007.0) 0x04c2e1a0 strlen
--7665-- REDIR: 0x4019b20 (ld-linux-x86-64.so.2:index) redirected to 0x4c2dd50 (index)
--7665-- REDIR: 0x4019d40 (ld-linux-x86-64.so.2:strcmp) redirected to 0x4c2f2f0 (strcmp)
--7665-- REDIR: 0x401aa90 (ld-linux-x86-64.so.2:mempcpy) redirected to 0x4c31da0 (mempcpy)
--7665-- Reading syms from /usr/lib/x86_64-linux-gnu/libpcap.so.1.5.3
--7665-- object doesn't have a symbol table
--7665-- Reading syms from /lib/x86_64-linux-gnu/libpthread-2.19.so
--7665-- Considering /lib/x86_64-linux-gnu/libpthread-2.19.so ..
--7665-- .. CRC mismatch (computed d432accf wanted 317a970d)
--7665-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/libpthread-2.19.so ..
--7665-- .. CRC is valid
--7665-- Reading syms from /lib/x86_64-linux-gnu/librt-2.19.so
--7665-- Considering /lib/x86_64-linux-gnu/librt-2.19.so ..
--7665-- .. CRC mismatch (computed 751a3be9 wanted b8b87a5f)
--7665-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/librt-2.19.so ..
--7665-- .. CRC is valid
--7665-- Reading syms from /lib/x86_64-linux-gnu/libdl-2.19.so
--7665-- Considering /lib/x86_64-linux-gnu/libdl-2.19.so ..
--7665-- .. CRC mismatch (computed ee8e2c67 wanted 20f67a06)
--7665-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/libdl-2.19.so ..
--7665-- .. CRC is valid
--7665-- Reading syms from /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19
--7665-- Considering /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19 ..
--7665-- .. CRC mismatch (computed dc37bb90 wanted ea8c3b40)
--7665-- object doesn't have a symbol table
--7665-- Reading syms from /lib/x86_64-linux-gnu/libgcc_s.so.1
--7665-- Considering /lib/x86_64-linux-gnu/libgcc_s.so.1 ..
--7665-- .. CRC mismatch (computed 6116126e wanted 54e3f1f2)
--7665-- object doesn't have a symbol table
--7665-- Reading syms from /lib/x86_64-linux-gnu/libc-2.19.so
--7665-- Considering /lib/x86_64-linux-gnu/libc-2.19.so ..
--7665-- .. CRC mismatch (computed 26a92edf wanted 7a008b73)
--7665-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.19.so ..
--7665-- .. CRC is valid
--7665-- Reading syms from /lib/x86_64-linux-gnu/libm-2.19.so
--7665-- Considering /lib/x86_64-linux-gnu/libm-2.19.so ..
--7665-- .. CRC mismatch (computed 0511ff25 wanted e2dba104)
--7665-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/libm-2.19.so ..
--7665-- .. CRC is valid
--7665-- REDIR: 0x5c46070 (libc.so.6:strcasecmp) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--7665-- REDIR: 0x5c48360 (libc.so.6:strncasecmp) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--7665-- REDIR: 0x5c45840 (libc.so.6:memcpy@GLIBC_2.2.5) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--7665-- REDIR: 0x5c43ad0 (libc.so.6:rindex) redirected to 0x4c2da30 (rindex)
--7665-- REDIR: 0x5c41dd0 (libc.so.6:strlen) redirected to 0x4c2e0e0 (strlen)
--7665-- REDIR: 0x5c452b0 (libc.so.6:__GI_memcmp) redirected to 0x4c30b80 (__GI_memcmp)
--7665-- REDIR: 0x5c40380 (libc.so.6:strcmp) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--7665-- REDIR: 0x5c53780 (libc.so.6:__strcmp_sse2_unaligned) redirected to 0x4c2f1b0 (strcmp)
--7665-- REDIR: 0x56fdd90 (libstdc++.so.6:operator new(unsigned long)) redirected to 0x4c2b070 (operator new(unsigned long))
--7665-- REDIR: 0x5c4aa40 (libc.so.6:memcpy@@GLIBC_2.14) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--7665-- REDIR: 0x5c53a30 (libc.so.6:__memcpy_sse2_unaligned) redirected to 0x4c2f6b0 (memcpy@@GLIBC_2.14)
--7665-- REDIR: 0x5c3c550 (libc.so.6:calloc) redirected to 0x4c2cbf0 (calloc)
--7665-- REDIR: 0x5c45270 (libc.so.6:bcmp) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--7665-- REDIR: 0x5d1c8e0 (libc.so.6:__memcmp_sse4_1) redirected to 0x4c30c00 (__memcmp_sse4_1)
--7665-- REDIR: 0x5c4cdd0 (libc.so.6:strchrnul) redirected to 0x4c319b0 (strchrnul)
--7665-- REDIR: 0x5c41850 (libc.so.6:__GI_strcpy) redirected to 0x4c2e2a0 (__GI_strcpy)
--7665-- REDIR: 0x5c40160 (libc.so.6:__GI_strchr) redirected to 0x4c2db90 (__GI_strchr)
--7665-- REDIR: 0x56fc0f0 (libstdc++.so.6:operator delete(void*)) redirected to 0x4c2c250 (operator delete(void*))
--7665-- REDIR: 0x5c421f0 (libc.so.6:strncmp) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--7665-- REDIR: 0x5cfdce0 (libc.so.6:__strncmp_ssse3) redirected to 0x4c2e8c0 (strncmp)
--7665-- REDIR: 0xffffffffff600400 (???:???) redirected to 0x3807649d (???)
--7665-- REDIR: 0x5c44f20 (libc.so.6:memchr) redirected to 0x4c2f390 (memchr)
--7665-- REDIR: 0x5c456b0 (libc.so.6:__GI_memmove) redirected to 0x4c31660 (__GI_memmove)
--7665-- REDIR: 0x5c43a90 (libc.so.6:strncpy) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--7665-- REDIR: 0x5c58c20 (libc.so.6:__strncpy_sse2_unaligned) redirected to 0x4c2e770 (__strncpy_sse2_unaligned)
--7665-- REDIR: 0x5d0c910 (libc.so.6:__memmove_ssse3_back) redirected to 0x4c2f450 (memcpy@GLIBC_2.2.5)
--7665-- REDIR: 0x5c3ba80 (libc.so.6:malloc) redirected to 0x4c2ab10 (malloc)
--7665-- REDIR: 0x5c4aa90 (libc.so.6:__GI_memcpy) redirected to 0x4c2fc90 (__GI_memcpy)
--7665-- REDIR: 0x5c3c120 (libc.so.6:free) redirected to 0x4c2bd80 (free)
--7665-- REDIR: 0x56fdea0 (libstdc++.so.6:operator new[](unsigned long)) redirected to 0x4c2b790 (operator new[](unsigned long))
--7665-- REDIR: 0x5c41810 (libc.so.6:strcpy) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--7665-- REDIR: 0x5c585f0 (libc.so.6:__strcpy_sse2_unaligned) redirected to 0x4c2e1c0 (strcpy)
--7665-- REDIR: 0x5c42240 (libc.so.6:__GI_strncmp) redirected to 0x4c2e930 (__GI_strncmp)
--7665-- REDIR: 0x5c44720 (libc.so.6:__GI_strstr) redirected to 0x4c32030 (__strstr_sse2)
EAL: Detected 24 lcore(s)
--7665-- REDIR: 0x5c40130 (libc.so.6:index) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--7665-- REDIR: 0x5c44ce0 (libc.so.6:strstr) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--7665-- REDIR: 0x5c5cc90 (libc.so.6:__strstr_sse2_unaligned) redirected to 0x4c31fa0 (strstr)
--7665-- REDIR: 0x5c41f90 (libc.so.6:strnlen) redirected to 0x4c2e080 (strnlen)
--7665-- REDIR: 0x5c458d0 (libc.so.6:memset) redirected to 0x4c31350 (memset)
--7665-- REDIR: 0x5c53680 (libc.so.6:__GI_strncpy) redirected to 0x4c2e4d0 (__GI_strncpy)
EAL: Probing VFIO support...
--7665-- REDIR: 0x5cc5650 (libc.so.6:__memcpy_chk) redirected to 0x4a25720 (_vgnU_ifunc_wrapper)
--7665-- REDIR: 0x5d07360 (libc.so.6:__memcpy_chk_ssse3_back) redirected to 0x4c31e90 (__memcpy_chk)
--7665-- REDIR: 0x5c403c0 (libc.so.6:__GI_strcmp) redirected to 0x4c2f200 (__GI_strcmp)
--7665-- REDIR: 0x5c3c220 (libc.so.6:realloc) redirected to 0x4c2ce10 (realloc)
==7665== Warning: noted but unhandled ioctl 0x3b64 with no size/direction hints.
==7665== This could cause spurious value errors to appear.
==7665== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==7665== Warning: noted but unhandled ioctl 0x3b65 with no size/direction hints.
==7665== This could cause spurious value errors to appear.
==7665== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
EAL: no supported IOMMU extensions found!
EAL: VFIO support could not be initialized
EAL: WARNING: Master core has no memory on local socket!
EAL: PCI device 0000:01:00.0 on NUMA socket -1
EAL: probe driver: 8086:10d3 net_e1000_em
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: PCI device 0000:04:00.1 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: pci_uio_alloc_resource(): cannot store uio mmap details
EAL: Error - exiting with code: 1
Cause: Requested device 0000:04:00.1 cannot be used
==7665==
==7665== HEAP SUMMARY:
==7665== in use at exit: 169,553 bytes in 131 blocks
==7665== total heap usage: 2,667 allocs, 2,536 frees, 936,142 bytes allocated
==7665==
==7665== Searching for pointers to 131 not-freed blocks
==7665== Checked 202,582,712 bytes
==7665==
==7665== 4 bytes in 1 blocks are still reachable in loss record 1 of 16
==7665== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x47CBF0: pcpp::IPv4Address::IPv4Address(pcpp::IPv4Address const&) (IpAddress.cpp:64)
==7665== by 0x41FC20: main (main.cpp:279)
==7665==
==7665== 4 bytes in 1 blocks are still reachable in loss record 2 of 16
==7665== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x47CBF0: pcpp::IPv4Address::IPv4Address(pcpp::IPv4Address const&) (IpAddress.cpp:64)
==7665== by 0x41FC34: main (main.cpp:280)
==7665==
==7665== 4 bytes in 1 blocks are still reachable in loss record 3 of 16
==7665== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x428057: __gnu_cxx::new_allocator::allocate(unsigned long, void const*) (new_allocator.h:104)
==7665== by 0x426BD4: std::_Vector_base<int, std::allocator >::_M_allocate(unsigned long) (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x424B0E: std::vector<int, std::allocator >::_M_insert_aux(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator > >, int const&) (vector.tcc:345)
==7665== by 0x42326B: std::vector<int, std::allocator >::push_back(int const&) (stl_vector.h:913)
==7665== by 0x41FDBC: main (main.cpp:310)
==7665==
==7665== 10 bytes in 1 blocks are still reachable in loss record 4 of 16
==7665== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x5C41B49: strdup (strdup.c:42)
==7665== by 0x56B5E2: get_hugepage_dir (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x56B9E8: eal_hugepage_info_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x56AE56: rte_eal_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x436F4A: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:124)
==7665== by 0x420287: main (main.cpp:434)
==7665==
==7665== 16 bytes in 1 blocks are still reachable in loss record 5 of 16
==7665== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x5C41B49: strdup (strdup.c:42)
==7665== by 0x56A2A7: rte_eal_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x436F4A: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:124)
==7665== by 0x420287: main (main.cpp:434)
==7665==
==7665== 26 bytes in 1 blocks are possibly lost in loss record 6 of 16
==7665== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x5759248: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x5759E0A: std::string::_Rep::_M_clone(std::allocator const&, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x5759EA3: std::string::reserve(unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x5714C16: std::basic_istream<char, std::char_traits >& std::operator>><char, std::char_traits, std::allocator >(std::basic_istream<char, std::char_traits >&, std::basic_string<char, std::char_traits, std::allocator >&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x436CDC: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:109)
==7665== by 0x420287: main (main.cpp:434)
==7665==
==7665== 44 bytes in 1 blocks are possibly lost in loss record 7 of 16
==7665== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x5759248: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x575A970: char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator const&, std::forward_iterator_tag) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x575AD87: std::basic_string<char, std::char_traits, std::allocator >::basic_string(char const*, std::allocator const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x41FECE: main (main.cpp:332)
==7665==
==7665== 56 bytes in 1 blocks are still reachable in loss record 8 of 16
==7665== at 0x4C2B800: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x436CA9: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:106)
==7665== by 0x420287: main (main.cpp:434)
==7665==
==7665== 140 bytes in 7 blocks are still reachable in loss record 9 of 16
==7665== at 0x4C2B800: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x436CF0: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:110)
==7665== by 0x420287: main (main.cpp:434)
==7665==
==7665== 192 bytes in 6 blocks are possibly lost in loss record 10 of 16
==7665== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x5759248: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x5759E0A: std::string::_Rep::_M_clone(std::allocator const&, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x5759EA3: std::string::reserve(unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x575A0EE: std::string::append(char const*, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x5714B81: std::basic_istream<char, std::char_traits >& std::operator>><char, std::char_traits, std::allocator >(std::basic_istream<char, std::char_traits >&, std::basic_string<char, std::char_traits, std::allocator >&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x436CDC: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:109)
==7665== by 0x420287: main (main.cpp:434)
==7665==
==7665== 256 bytes in 1 blocks are still reachable in loss record 11 of 16
==7665== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x47E9C5: allocate (new_allocator.h:104)
==7665== by 0x47E9C5: _M_allocate (stl_vector.h:168)
==7665== by 0x47E9C5: std::vector<pcpp::SystemCore, std::allocatorpcpp::SystemCore >::_M_insert_aux(__gnu_cxx::__normal_iterator<pcpp::SystemCore*, std::vector<pcpp::SystemCore, std::allocatorpcpp::SystemCore > >, pcpp::SystemCore const&) (vector.tcc:345)
==7665== by 0x47E902: push_back (stl_vector.h:913)
==7665== by 0x47E902: pcpp::createCoreVectorFromCoreMask(unsigned int, std::vector<pcpp::SystemCore, std::allocatorpcpp::SystemCore >&) (SystemUtils.cpp:163)
==7665== by 0x42023F: main (main.cpp:425)
==7665==
==7665== 280 bytes in 1 blocks are still reachable in loss record 12 of 16
==7665== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x5C275BF: fopencookie@@GLIBC_2.2.5 (iofopncook.c:197)
==7665== by 0x56E308: rte_eal_log_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x56A9EB: rte_eal_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x436F4A: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:124)
==7665== by 0x420287: main (main.cpp:434)
==7665==
==7665== 304 bytes in 1 blocks are possibly lost in loss record 13 of 16
==7665== at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x4012EE4: allocate_dtv (dl-tls.c:296)
==7665== by 0x4012EE4: _dl_allocate_tls (dl-tls.c:460)
==7665== by 0x507DD92: allocate_stack (allocatestack.c:589)
==7665== by 0x507DD92: pthread_create@@GLIBC_2.2.5 (pthread_create.c:500)
==7665== by 0x57387B: rte_eal_intr_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x56AB04: rte_eal_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x436F4A: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:124)
==7665== by 0x420287: main (main.cpp:434)
==7665==
==7665== 537 bytes in 1 blocks are possibly lost in loss record 14 of 16
==7665== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x5759248: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x5759E0A: std::string::_Rep::_M_clone(std::allocator const&, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x5759EA3: std::string::reserve(unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x57381E5: std::basic_stringbuf<char, std::char_traits, std::allocator >::overflow(int) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x573C415: std::basic_streambuf<char, std::char_traits >::xsputn(char const*, long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x5733C04: std::basic_ostream<char, std::char_traits >& std::__ostream_insert<char, std::char_traits >(std::basic_ostream<char, std::char_traits >&, char const*, long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==7665== by 0x436B71: operator<< <std::char_traits > (ostream:535)
==7665== by 0x436B71: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:97)
==7665== by 0x420287: main (main.cpp:434)
==7665==
==7665== 6,992 bytes in 23 blocks are possibly lost in loss record 15 of 16
==7665== at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x4012EE4: allocate_dtv (dl-tls.c:296)
==7665== by 0x4012EE4: _dl_allocate_tls (dl-tls.c:460)
==7665== by 0x507DD92: allocate_stack (allocatestack.c:589)
==7665== by 0x507DD92: pthread_create@@GLIBC_2.2.5 (pthread_create.c:500)
==7665== by 0x56AB96: rte_eal_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x436F4A: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:124)
==7665== by 0x420287: main (main.cpp:434)
==7665==
==7665== 160,688 bytes in 83 blocks are still reachable in loss record 16 of 16
==7665== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7665== by 0x56FABC: pci_scan_one (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x5706F5: rte_eal_pci_scan (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x570A18: rte_eal_pci_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x56A9F8: rte_eal_init (in /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter)
==7665== by 0x436F4A: pcpp::DpdkDeviceList::initDpdk(unsigned int, unsigned int, unsigned char) (DpdkDeviceList.cpp:124)
==7665== by 0x420287: main (main.cpp:434)
==7665==
==7665== LEAK SUMMARY:
==7665== definitely lost: 0 bytes in 0 blocks
==7665== indirectly lost: 0 bytes in 0 blocks
==7665== possibly lost: 8,095 bytes in 33 blocks
==7665== still reachable: 161,458 bytes in 98 blocks
==7665== suppressed: 0 bytes in 0 blocks
==7665==
==7665== ERROR SUMMARY: 6 errors from 6 contexts (suppressed: 0 from 0)
==7665== ERROR SUMMARY: 6 errors from 6 contexts (suppressed: 0 from 0)

=========================================================
And these three messages are coming up:

EAL: PCI device 0000:04:00.1 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: pci_uio_alloc_resource(): cannot store uio mmap details
EAL: Error - exiting with code: 1
Cause: Requested device 0000:04:00.1 cannot be used

Please let me know if you need any other kind of trace.

seladb · 2019-02-04T17:14:08Z

I think running the app with gdb instead of valgrind will give a more clear stack trace.

Also, did you try to run the examples shipped with dpdk on that machine? do they work?

sabhishepalc · 2019-02-05T05:49:18Z

I think running the app with gdb instead of valgrind will give a more clear stack trace.

Also, did you try to run the examples shipped with dpdk on that machine? do they work?

(gdb) r
Starting program: /root/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin/DpdkTrafficFilter -d 0 -r UDP -f /root/PcapPlusPlus/ -c 15
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
EAL: Detected 24 lcore(s)
EAL: Probing VFIO support...
EAL: no supported IOMMU extensions found!
EAL: VFIO support could not be initialized
[New Thread 0x7ffff6988700 (LWP 13852)]
[New Thread 0x7ffff6187700 (LWP 13853)]
[New Thread 0x7ffff5986700 (LWP 13854)]
[New Thread 0x7ffff5185700 (LWP 13855)]
EAL: PCI device 0000:01:00.0 on NUMA socket -1
EAL: probe driver: 8086:10d3 net_e1000_em
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: PCI device 0000:04:00.1 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
PMD: eth_i40e_dev_init(): FW 6.0 API 1.7 NVM 06.00.01 eetrack 800035da
Successfully initialized packets pool of size [4095] for device [DPDK_0]
PMD supports the following RSS hash functions : 0x1fbd

Successfully called rte_eth_dev_configure for device [DPDK_0] with 4 RX queues and 4 TX queues
Successfully initialized 4 RX queues for device [DPDK_0]
Successfully initialized 4 TX queues for device [DPDK_0]
Device [DPDK_0] : Link up; Speed: 40000 Mbps; full-duplex
......... Core configuration (I have not pasted them here)

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff6187700 (LWP 13853)]
0x00000000004d5037 in i40e_recv_pkts_vec ()
(gdb) bt
#0 0x00000000004d5037 in i40e_recv_pkts_vec ()
#1 0x0000000000434c2c in rte_eth_rx_burst (nb_pkts=64, rx_pkts=0x7ffff6186660, queue_id=, port_id=0 '\000')
at /usr/local/dpdk/include/dpdk/rte_ethdev.h:2669
#2 pcpp::DpdkDevice::receivePackets (this=, rawPacketsArr=0x7ffff61869a0, rawPacketArrLength=64, rxQueueId=)
at src/DpdkDevice.cpp:1092
#3 0x00000000004229c0 in AppWorkerThread::run (this=0x901730, coreId=1) at AppWorkerThread.h:84
#4 0x00000000004354d4 in pcpp::DpdkDeviceList::dpdkWorkerThreadStart (ptr=) at src/DpdkDeviceList.cpp:287
#5 0x000000000056e115 in eal_thread_loop ()
#6 0x00007ffff7986184 in start_thread (arg=0x7ffff6187700) at pthread_create.c:312
#7 0x00007ffff6d8d03d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

========================================================
We have tried running the ethtool example in dpdk :
root@ubuntu:/home/administrator/dpdk-stable-16.11.1/tools# python dpdk-devbind.py -s

Network devices using DPDK-compatible driver

0000:04:00.1 'Ethernet Controller XL710 for 40GbE QSFP+' drv=igb_uio unused=vfio-pci,uio_pci_generic

Network devices using kernel driver

0000:01:00.0 '82574L Gigabit Network Connection' if=eth0 drv=e1000e unused=igb_uio,vfio-pci,uio_pci_generic Active
0000:04:00.0 'Ethernet Controller XL710 for 40GbE QSFP+' if=p1p1 drv=i40e unused=igb_uio,vfio-pci,uio_pci_generic

root@ubuntu:/home/administrator/dpdk-stable-16.11.1/examples/ethtool/ethtool-app/ethtool-app/x86_64-native-linuxapp-gcc# ./ethtool -l 6-11
EAL: Detected 24 lcore(s)
EAL: Probing VFIO support...
EAL: no supported IOMMU extensions found!
EAL: VFIO support could not be initialized
EAL: PCI device 0000:01:00.0 on NUMA socket -1
EAL: probe driver: 8086:10d3 net_e1000_em
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: PCI device 0000:04:00.1 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
PMD: eth_i40e_dev_init(): FW 6.0 API 1.7 NVM 06.00.01 eetrack 800035da
Number of NICs: 1
Init port 0..
EthApp>?
drvinfo [Fixed STRING]: drvinfo
Print driver info
eeprom [Fixed STRING]: eeprom <port_id>
Dump EEPROM to file

.....
EthApp> drvinfo
Port 0 driver: net_i40e (ver: DPDK 16.11.1)
EthApp>

Here, the driver for the DPDK port is net_i40e but the devbind status shows that the driver is igb_uio. Is this the expected behavior?
Thanks

seladb · 2019-02-05T06:20:09Z

net_i40e is the PMD (Poll Mode Driver) dpdk uses to receive and send packets from/to the NIC. igb_uio is the Linux kernel driver dpdk uses to bypass the kernel packet processing. So these are two different things.

From the stack trace you printed it seems the PMD is crashing in i40e_recv_pkts_vec(). Let's try to find out why. Could you please try to do the following:

Run one of the dpdk examples that process traffic, for example: L2 Forwarding
Run DpdkTrafficFilter -l. This switch just displays the dpdk interfaces and exists. Let's see if that works

echo-Mike · 2019-02-05T07:50:44Z

@sabhishepalc @seladb @shidaoxuan
The first step to identify DPDK problems is to run testpmd with your device and play with settings. If testpmd show the statistics about received packets (assuming you have income flow of packets to your port from some source) then it is time to search for error in Pcap++
Within your last messages the two separate sets of problems are occurred:

First:

EAL: pci_uio_alloc_resource(): cannot store uio mmap details

I suppose that this is caused by running the valgrind on DPDK as provoded log identifies problem:

==32584== Warning: noted but unhandled ioctl 0x3b64 with no size/direction hints.
==32584== This could cause spurious value errors to appear.
==32584== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.

Second:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff6187700 (LWP 13853)]
0x00000000004d5037 in i40e_recv_pkts_vec ()

Is caused by currently unknown problem.

Recommendations:

Switch to newer DPDK. I propose 18.02.2 By my experience it is the only version currently buildable under -lwhole-archive linker flag as newer mempools give unresolved symbol errors;
Switch to newer compiler gcc 4.8 is really old one;
Understand how to work with testpmd app and test your card in it.

And just a general trend - the i40e driver is screwed up one. I currently have a lot of problems with it (RSS not working properly, invalid mbufs comming and other)

sabhishepalc · 2019-02-05T13:20:41Z

@sabhishepalc @seladb @shidaoxuan
The first step to identify DPDK problems is to run testpmd with your device and play with settings. If testpmd show the statistics about received packets (assuming you have income flow of packets to your port from some source) then it is time to search for error in Pcap++
Within your last messages the two separate sets of problems are occurred:

First:

EAL: pci_uio_alloc_resource(): cannot store uio mmap details

I suppose that this is caused by running the valgrind on DPDK as provoded log identifies problem:

==32584== Warning: noted but unhandled ioctl 0x3b64 with no size/direction hints.
==32584== This could cause spurious value errors to appear.
==32584== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.

Second:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff6187700 (LWP 13853)]
0x00000000004d5037 in i40e_recv_pkts_vec ()

Is caused by currently unknown problem.

Recommendations:

Switch to newer DPDK. I propose 18.02.2 By my experience it is the only version currently buildable under -lwhole-archive linker flag as newer mempools give unresolved symbol errors;

Switch to newer compiler gcc 4.8 is really old one;

Understand how to work with testpmd app and test your card in it.

And just a general trend - the i40e driver is screwed up one. I currently have a lot of problems with it (RSS not working properly, invalid mbufs comming and other)

Thanks a lot @echo-Mike and @seladb .
We will try testpmd and verify.
For now, we had to hardcode the number of Rx threads/workers to a lower number to see if the problem still persists and we are happy to let you know that the segmentation fault is no longer seen.

=======================================================
Sample output:
root@ubuntu:~/PcapPlusPlus/Examples/DpdkExample-FilterTraffic/Bin# ./DpdkTrafficFilter -d 0 -f /home/administrator/ -c 15 -r UDP
EAL: Detected 24 lcore(s)
EAL: Probing VFIO support...
EAL: no supported IOMMU extensions found!
EAL: VFIO support could not be initialized
EAL: PCI device 0000:01:00.0 on NUMA socket -1
EAL: probe driver: 8086:10d3 net_e1000_em
EAL: PCI device 0000:04:00.0 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
EAL: PCI device 0000:04:00.1 on NUMA socket -1
EAL: probe driver: 8086:1583 net_i40e
PMD: eth_i40e_dev_init(): FW 6.0 API 1.7 NVM 06.00.01 eetrack 800035da
Successfully initialized packets pool of size [4095] for device [DPDK_0]
PMD supports the following RSS hash functions : 0x1fbd

Successfully called rte_eth_dev_configure for device [DPDK_0] with 4 RX queues and 4 TX queues
Successfully initialized 4 RX queues for device [DPDK_0]
Successfully initialized 4 TX queues for device [DPDK_0]
Device [DPDK_0] : Link up; Speed: 40000 Mbps; full-duplex
Using core 1
Core configuration:
DPDK device#0: RX-Queue#0; RX-Queue#1;
Coming here
coming here to icrement i
Using core 2
Core configuration:
DPDK device#0: RX-Queue#2;
Coming here
coming here to icrement i
Using core 3
Core configuration:
DPDK device#0: RX-Queue#3;
Coming here
coming here to icrement i
Coming to matchingEngine
Coming here to get new worker thread
Coming here to get new worker thread
Coming here to get new worker thread
Calling PacketMatching engine

Application stopped

| Core ID | Packet Cnt | Eth Cnt | ARP Cnt | IPv4 Cnt | IPv6 Cnt | TCP Cnt | UDP Cnt | HTTP Cnt | Matched TCP Flows | Matched UDP Flows | Matched Packets |

| 1 | 17 | 17 | 17 | 17 | 0 | 0 | 17 | 0 | 0 | 17 | 17 |
| 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

| Total | 17 | 17 | 17 | 17 | 0 | 0 | 17 | 0 | 0 | 17 | 17 |

==============================================

Thank you.

seladb · 2019-02-14T17:14:33Z

Do you still need this issue open? If not, can you close it?

sabhishepalc · 2019-02-15T05:07:04Z

@seladb Certainly you have helped a lot and kudos to you and other contributors. I will happily close this issue.

Thanks and cheers.

sabhishepalc closed this as completed Feb 15, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pcap++ test issue related to the RSS hash function 0x41 #158

Pcap++ test issue related to the RSS hash function 0x41 #158

sabhishepalc commented Jan 30, 2019 •

edited

Loading

echo-Mike commented Jan 30, 2019

sabhishepalc commented Jan 30, 2019

echo-Mike commented Jan 30, 2019

sabhishepalc commented Jan 30, 2019

echo-Mike commented Jan 30, 2019

seladb commented Jan 31, 2019

sabhishepalc commented Feb 1, 2019

echo-Mike commented Feb 1, 2019

sabhishepalc commented Feb 1, 2019

echo-Mike commented Feb 1, 2019

daoxuans commented Feb 2, 2019

seladb commented Feb 3, 2019

sabhishepalc commented Feb 4, 2019

seladb commented Feb 4, 2019

sabhishepalc commented Feb 4, 2019

seladb commented Feb 4, 2019

sabhishepalc commented Feb 4, 2019

seladb commented Feb 4, 2019

sabhishepalc commented Feb 5, 2019

seladb commented Feb 5, 2019

echo-Mike commented Feb 5, 2019

sabhishepalc commented Feb 5, 2019

First:

Second:

Recommendations:

seladb commented Feb 14, 2019

sabhishepalc commented Feb 15, 2019

Pcap++ test issue related to the RSS hash function 0x41 #158

Pcap++ test issue related to the RSS hash function 0x41 #158

Comments

sabhishepalc commented Jan 30, 2019 • edited Loading

Network devices using DPDK-compatible driver

Network devices using kernel driver

Other network devices

Crypto devices using DPDK-compatible driver

Crypto devices using kernel driver

Other crypto devices

echo-Mike commented Jan 30, 2019

sabhishepalc commented Jan 30, 2019

With this command I can get the list of DPDK enabled ports:

Error: DPDK ports list is invalid

Although the usage menu says the following: Usage:

Not working.

echo-Mike commented Jan 30, 2019

sabhishepalc commented Jan 30, 2019

echo-Mike commented Jan 30, 2019

seladb commented Jan 31, 2019

sabhishepalc commented Feb 1, 2019

echo-Mike commented Feb 1, 2019

sabhishepalc commented Feb 1, 2019

echo-Mike commented Feb 1, 2019

daoxuans commented Feb 2, 2019

seladb commented Feb 3, 2019

sabhishepalc commented Feb 4, 2019

**Setup completed.

seladb commented Feb 4, 2019

sabhishepalc commented Feb 4, 2019

seladb commented Feb 4, 2019

sabhishepalc commented Feb 4, 2019

seladb commented Feb 4, 2019

sabhishepalc commented Feb 5, 2019

seladb commented Feb 5, 2019

echo-Mike commented Feb 5, 2019

First:

Second:

Recommendations:

sabhishepalc commented Feb 5, 2019

First:

Second:

Recommendations:

Application stopped

| Core ID | Packet Cnt | Eth Cnt | ARP Cnt | IPv4 Cnt | IPv6 Cnt | TCP Cnt | UDP Cnt | HTTP Cnt | Matched TCP Flows | Matched UDP Flows | Matched Packets |

| 1 | 17 | 17 | 17 | 17 | 0 | 0 | 17 | 0 | 0 | 17 | 17 | | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

| Total | 17 | 17 | 17 | 17 | 0 | 0 | 17 | 0 | 0 | 17 | 17 |

seladb commented Feb 14, 2019

sabhishepalc commented Feb 15, 2019

sabhishepalc commented Jan 30, 2019 •

edited

Loading

Although the usage menu says the following:
Usage:

| 1 | 17 | 17 | 17 | 17 | 0 | 0 | 17 | 0 | 0 | 17 | 17 |
| 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |