-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathdpdk-pf_ring.html
122 lines (108 loc) · 8.23 KB
/
dpdk-pf_ring.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
---
title: DPDK & PF_RING Support
layout: default
section: dpdk-pfring
---
<h1>PcapPlusPlus DPDK and PF_RING wrappers</h1>
<h2>DPDK support</h2>
<p>The Data Plane Development Kit (DPDK) is a set of data plane libraries and network interface controller drivers for fast packet processing. The DPDK provides a programming framework for Intel x86 processors and enables faster development of high speed data packet networking applications. It is provided and supported under the open source BSD license (taken from <a href="https://en.wikipedia.org/wiki/Data_Plane_Development_Kit">Wikipedia</a> )</p>
<p>DPDK provides packet processing in line rate using kernel bypass for a large range of network interface cards. Notice that not every NIC supports DPDK as the NIC needs to support the kernel bypass feature. You can read more about DPDK in <a href="http://dpdk.org/">DPDK's web-site</a> and get a list of supported NICs <a href="http://dpdk.org/doc/nics">here</a>.</p>
<p>Also, you can get more details about DPDK and PcapPlusPlus wrap for DPDK in the documentation of DpdkDevice.h and DpdkDeviceList.h header files.</p>
<h6><strong>Download and install</strong></h6>
<p>Download and install instructions for DPDK are on this page: <a href="http://dpdk.org/download">http://dpdk.org/download</a></p>
<p>Please make sure to install DPDK in the following way:</p>
<pre><code class="language-shell">make config T=[platform_type_string] && make</code></pre><p></p>
<h6><strong>So what does PcapPlusPlus offer for DPDK?</strong></h6>
<ul>
<li>An easy-to-use C++ wrapper (as DPDK is written in C) that encapsulates DPDK's main functionality but doesn't hit packet procssing performance</li>
<li>Encapsulation of DPDK's initialization process - both outside and inside the application - using simple scripts and methods</li>
<li>A C++ class wrapper for DPDK's packet struct (mbuf) which offers most common functionality</li>
<li>Seamless integration to other PcapPlusPlus capabilities, for example: receivce packets with DPDK, parse them with Packet++ protocol layers and save them to a pcap file</li>
</ul>
<h6><strong>PcapPlusPlus configuration for DPDK</strong></h6>
<ol>
<li>Download and compile DPDK on your system (see the link above)</li>
<li>Note that PcapPlusPlus supports DPDK versions from 16.11 to 19.02 (inclusive). Previous (and maybe also newer) versions won't work</li>
<li>Once DPDK compiles successfully you need to run PcapPlusPlus <strong>configure-linux.sh</strong> and type "y" in "Compile PcapPlusPlus with DPDK?"</li>
<li><strong>configure-linux.sh</strong> will ask for DPDK's path (i.e /home/user/dpdk-stable-16.11.1)</li>
<li>Then you can compile PcapPlusPlus as usual (using make, see below)</li>
</ol>
<h6><strong>DPDK initialization with PcapPlusPlus</strong></h6>
<p>DPDK has 2 steps of initialization: one that configures Linux to support DPDK applications and the other at application startup that initializes DPDK. PcapPlusPlus wraps both of them with easy-to-use interfaces:</p>
<h6><strong>Before application is run</strong></h6>
<p>DPDK requires several Linux configurations to run:</p>
<ol>
<li>DPDK uses the Linux huge-pages mechanism for faster virtual to physical page conversion resulting in better performance. So huge-pages must be set before a DPDK application is run</li>
<li>DPDK uses a designated kernel module for the kernel-bypass mechanism. This module should be loaded into the kernel</li>
<li>The user needs to state which NICs will move to DPDK control and which will stay under Linux control</li>
</ol>
<p>PcapPlusPlus offers a simple script that automatically configures all of these. The script is under PcapPlusPlus root directory and is called <strong>setup-dpdk.sh</strong>. The script takes as an input the following parameters:</p>
<ol>
<li>-p : the amount of huge pages to allocate. By default each huge-page size is 2048KB</li>
<li>-n : a comma-separated list of all NICs that will be unbinded from Linux and move to DPDK control. Only these NICs will be used by DPDK, the others will stay under Linux control. For example: eth0,eth1 will move these 2 interfaces under DPDK control - assuming this NIC is supported by DPDK. You can use the -h switch for help.</li>
</ol>
<p>If everything went well the system is ready to run a DPDK application and the script output should look like this:</p>
<pre>
<code class="language-shell">*******************************
PcapPlusPlus setup DPDK script
*******************************
1. Reserve huge-pages - DONE!
2. Install kernel module - DONE!
3. Bind network adapters - DONE!
Network devices using DPDK-compatible driver
============================================
0000:00:03.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=e1000
0000:00:08.0 '82545EM Gigabit Ethernet Controller (Copper)' drv=igb_uio unused=e1000
Network devices using kernel driver
===================================
0000:00:09.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth2 drv=e1000 unused=igb_uio *Active*
0000:00:0a.0 '82545EM Gigabit Ethernet Controller (Copper)' if=eth3 drv=e1000 unused=igb_uio
Other network devices
=====================
<none>
Setup DPDK completed</code>
</pre>
<h6><strong>At application startup</strong></h6>
<p>before using DPDK inside the application DPDK should be configured on application startup. This configuration includes:</p>
<ol>
<li>Verify huge-pages, kernel module and NICs are set</li>
<li>Initialize DPDK internal structures and memory, poll-mode-drivers etc.</li>
<li>Prepare CPU cores that will be used by the application</li>
<li>Initialize packet memory pool </li>
<li>Configure each NIC controlled by DPDK</li>
</ol>
<p>These steps are wrapped in one static method that should be called once in application startup:</p>
<pre><code class="language-cpp">DpdkDeviceList::initDpdk()</code></pre>
</p>
<h6><strong>Tests and limitations</strong></h6>
<ul>
<li>All unit-tests I perfromed are in Pcap++Test</li>
<li>In addition you try the DPDK example application (Examples/DpdkExample-FilterTraffic)</li>
<li>Supported DPDK versions are 16.11 to 18.02 (inclusive)</li>
<li>So far I managed to test the code on 2 virtual PMDs only:
<ol>
<li>VMXNET3 - a VMWare guest driver</li>
<li>E1000/EM - 1GbE Intel NIC but I tested it as virtual NIC in VirtualBox guest</li>
</ol>
</li>
<li>I hope I'll be able to test it on some more (preferebly non-virtual) NICs soon - I'll update if/when I do</li>
<li>Operating systems and configurations tested:
<ol>
<li>Ubuntu 14.04.1 LTS 64bit with kernel 3.x and gcc 4.8.x</li>
<li>Ubuntu 12.04.5 LTS 32bit with kernel 3.x and gcc 4.6.x</li>
<li>CentOS 7.1 64bit with kernel 3.x and gcc 4.8.x</li>
</ol>
</li>
</ul>
<h2>PF_RING support</h2>
<p>PcapPlusPlus provides a clean and simple C++ wrapper API for Vanilla PF_RING. Currently only Vanilla PF_RING is supported which provides significant performance improvement in comparison to libpcap or Linux kernel, but PF_RING DNA or ZC (which allows kernel bypass and zero-copy of packets from NIC to user-space) isn't supported. I hope I'll be able to add it in the future.</p>
<p>You can read more about PF_RING in ntop web-site: <a href="http://www.ntop.org/products/pf_ring/">http://www.ntop.org/products/pf_ring/</a> and in PF_RING user guide: <a href="https://svn.ntop.org/svn/ntop/trunk/PF_RING/doc/UsersGuide.pdf">https://svn.ntop.org/svn/ntop/trunk/PF_RING/doc/UsersGuide.pdf</a></p>
<p>In order to compile PcapPlusPlus with PF_RING you need to:</p>
<ol>
<li>Download PF_RING from ntop's web-site: <a href="http://www.ntop.org/get-started/download/#PF_RING">http://www.ntop.org/get-started/download/#PF_RING</a></li>
<li>Note that I used PcapPlusPlus with PF_RING versions 6.2.0 - 6.4.1. I can't guarantee it'll work with previous or later versions</li>
<li>Once PF_RING is compiled successfully, you need to run PcapPlusPlus <strong>configure-linux.sh</strong> and type "y" in "Compile PcapPlusPlus with PF_RING?"</li>
<li>Then you can compile PcapPlusPlus as usual</li>
<li>Before you activate any PcapPlusPlus application that uses PF_RING, don't forget to enable PF_RING kernel module. If you forget to do that, PcapPlusPlus will output an appropriate error on startup which will remind you:</li>
</ol>
<pre><code class="language-shell">sudo insmod <PF_RING_LOCATION>/kernel/pf_ring.ko</code></pre>