Update to ONVM 19.07 #8
Conversation
|
Oops looks like I forgot to update the openNetVM version in the repository. Would it be better to use a openNetVM as a submodule here? |
|
Yes, using a submodule would be ideal. Let us know when it is ready for us to take a look again. Thanks @archit-p ! |
|
Hi, I've updated the repository to include openNetVM 19.07 as a submodule. Have also updated the installation instructions in install.md, and installation script in install.sh. I'd really appreciate it if someone could follow these instructions and try a fresh installation! |
|
Hello team, Just a gentle reminder on the open pull request. Please provide suggestions on how to take this forward. |
|
Thanks @archit-p . @dennisafa is going to take a look but he may not get to it for another week because of class work. |
|
@dennisafa is the current code building alright? |
|
Hi Archit, i’ve run into a few issues compiling the code. I’ll get to it
this Thursday - i have a few deadlines before then.
…On Tue, Oct 22, 2019 at 12:35 PM Archit Pandey ***@***.***> wrote:
@dennisafa <https://github.com/dennisafa> is the current code building
alright?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#8?email_source=notifications&email_token=AH3EIZUMNB3OYYI32PDN2NLQP4TUXA5CNFSM4I76KWHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEB6MN4I#issuecomment-545048305>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AH3EIZWAO4AGJWFQ77BMZ7LQP4TUXANCNFSM4I76KWHA>
.
|
|
Hi dennis, No problems, you could share the issues here |
|
That's strange, I didn't need to update my $PATH. Could you share the snort
`config.log` file? It might contain what the error is exactly.
…On Fri, 25 Oct 2019, 8:37 am Dennis Afanasev, ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In Install.md
<#8 (comment)>:
> ```sh
- ./configure --enable-sourcefire
+ aclocal
+ autoconf
+ autoheader
+ automake -a
+ ```
+3. Run the configuration script.
I think the issue is that the $PATH variable needs to be initialized to
point to the daq-modules-config file. This file is found in
/daq-2.0.6/os-daq-modules
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#8?email_source=notifications&email_token=AHMM4LEO5PPRTH5LVSR32NLQQJPINA5CNFSM4I76KWHKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCJF6B4I#discussion_r338869726>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHMM4LE5ACXAWGBTY3XAC3LQQJPINANCNFSM4I76KWHA>
.
|
|
|
Looks like $DPDK_TARGET is not being interpreted correctly.
Could you try `echo $DPDK_TARGET` and if nothing pops up, `export
$DPDK_TARGET=x86_64-native-linuxapp-gcc` or your corresponding architecture.
On a side note, I believe I should remove the use of $DPDK_TARGET and stick
to $RTE_TARGET in daq-2.0.6/configure.ac. WDYT?
On Fri, 25 Oct 2019, 8:52 am Dennis Afanasev, <notifications@github.com>
wrote:
… That's strange, I didn't need to update my $PATH. Could you share the
snort config.log file? It might contain what the error is exactly.
… <#m_306845524257962216_>
On Fri, 25 Oct 2019, 8:37 am Dennis Afanasev, *@*.*> wrote: @.**
commented on this pull request. ------------------------------ In
Install.md <#8 (comment)
<#8 (comment)>>: > sh
- ./configure --enable-sourcefire + aclocal + autoconf + autoheader +
automake -a + +3. Run the configuration script. I think the issue is that
the $PATH variable needs to be initialized to point to the
daq-modules-config file. This file is found in /daq-2.0.6/os-daq-modules —
You are receiving this because you were mentioned. Reply to this email
directly, view it on GitHub <#8
<#8>?email_source=notifications&email_token=AHMM4LEO5PPRTH5LVSR32NLQQJPINA5CNFSM4I76KWHKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCJF6B4I#discussion_r338869726>,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHMM4LE5ACXAWGBTY3XAC3LQQJPINANCNFSM4I76KWHA
.
[image: Screen Shot 2019-10-24 at 11 21 13 PM]
<https://user-images.githubusercontent.com/32916582/67540859-01ba2280-f6b5-11e9-992d-2af22fd63aba.png>
Here is the relevant config.log section. Looks like there are issue with
-lonvm and -lonvmhelper
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#8?email_source=notifications&email_token=AHMM4LDHD6AWV3HYZOG34JLQQJQ53A5CNFSM4I76KWHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECHB2HQ#issuecomment-546184478>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHMM4LDBWT62T5AI2VI5BKDQQJQ53ANCNFSM4I76KWHA>
.
|
Yes, let's stick with using $RTE_TARGET as that is part of the openNetVM install guide. Thank you. |
|
Sure, will do that! I remember the old guide using two variables |
daq-2.0.6/os-daq-modules/daq_netvm.c
Outdated
| onvm_nflib_start_signal_handler(nf_local_ctx, sig_handler); | ||
| /*function table not needed */ | ||
|
|
||
| /* |
There was a problem hiding this comment.
Nit: Remove unnecessary commented code
daq-2.0.6/os-daq-modules/daq_netvm.c
Outdated
| printf("netvm init done\n"); | ||
| /* Complete onvm handshake */ | ||
| onvm_nflib_nf_ready(nf_info); | ||
| /* not necessary honestly |
daq-2.0.6/os-daq-modules/daq_netvm.c
Outdated
| netvmc->rx_ring = onvm_nflib_get_rx_ring(nf_info); | ||
| netvmc->tx_stats = onvm_nflib_get_tx_stats(nf_info); | ||
|
|
||
| /* these functions do not exist anymore |
daq-2.0.6/os-daq-modules/daq_netvm.c
Outdated
| @@ -273,7 +331,7 @@ static int netvm_daq_start(void *handle) | |||
| //printf("->netvm_daq_start()\n"); | |||
| netvm_daq_reset_stats(handle); | |||
|
|
|||
| //nf_info->status = NF_RUNNING; | |||
| //nf->status = NF_RUNNING; | |||
daq-2.0.6/os-daq-modules/daq_netvm.c
Outdated
| for (j = 0; j < tx_batch_size; j++) { | ||
| rte_pktmbuf_free(pktsTX[j]); | ||
| } | ||
| } else { | ||
| netvmc->tx_stats->tx[info->instance_id] += tx_batch_size; | ||
| /* update to the new pointer, update name */ | ||
| nf->stats.tx += tx_batch_size; | ||
| } | ||
| } | ||
| //printf("<-netvm_daq_acquire() - count\n"); |
There was a problem hiding this comment.
Remove commented out print statements
|
There are extra print statements in several places. I believe, they were used when debugging daq_netvm.c. |
|
There are too many commits in my repository right now, I wish to squash or fixup most of them. Could you let me know once you approve of the changes? I'll make a force push with modified history. |
Install.md
Outdated
| ```sh | ||
| cd ../ | ||
| ./patching-Makefile.sh | ||
| ``` | ||
|
|
||
| 4. Navigate to the src folder of snort and Make snort. | ||
| 5. Navigate to the src folder of snort and Make snort. |
There was a problem hiding this comment.
Please add a step asking the user to run autoreconf -ivf in case of a version mismatch error, then to do step 3 again.
Also, I am getting this issue:
Could you elaborate on -ldpdk a bit more?
There was a problem hiding this comment.
Yes, looks like I forgot to add autoreconf -fvi. Thanks for pointing that out!
There was a problem hiding this comment.
Hi Archit, please add this step to the install guide. Thank you.
|
-ldpdk is found in $RTE_SDK/$RTE_TARGET/lib
From the screenshot it looks to me like your $RTE_SDK variable is still set
to the dpdk-16.11 directory. You might need to set it again.
…On Fri, 25 Oct 2019, 9:28 am Dennis Afanasev, ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In Install.md
<#8 (comment)>:
> ```sh
cd ../
./patching-Makefile.sh
```
-4. Navigate to the src folder of snort and Make snort.
+5. Navigate to the src folder of snort and Make snort.
Please add a step asking the user to run autoreconf -ivf in case of a
version mismatch error, then to do step 3 again.
Also, I am getting this issue:
[image: Screen Shot 2019-10-24 at 11 57 43 PM]
<https://user-images.githubusercontent.com/32916582/67542278-2369d880-f6ba-11e9-80b7-dcceb6cb1cf6.png>
Could you elaborate on -ldpdk a bit more?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#8?email_source=notifications&email_token=AHMM4LFSHMT2VGLOO7RECDLQQJVHTA5CNFSM4I76KWHKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCJGANEA#pullrequestreview-306972304>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHMM4LGLPKGSNW5DDL46A6DQQJVHTANCNFSM4I76KWHA>
.
|
It looks like my $RTE_SDK output is correct (set to onvm-snort/openNetVM/dpdk) I will look into it. |
|
Might be helpful to try running once DAQ is done installing. |
|
I managed to get a working build running with a few modifications to the install guide (see comments above) |
The snort NF will now print out processing time for every batch of packets and the related packets per second rate
patching-Makefile.sh has been modified to accept default values. This file is now called from inside setenv.sh. setenv.sh sets up all initial enviornment variables assuming defaults. The variables in this file can be changed if defaults are not used. The setenv.sh script is run from inside install.sh. This script essentially, sets up all environment variables, installs all packages and builds all the tools.
Flags needed to compile daq-2.0.6 with DPDK 18.11
daq-2.0.6/build.sh - automates daq build snort-2.9.8.3/build.sh - automates snort build
.gitignore contains common patterns to ignore for C projects.
Removed extra print statements, added comments to daq_netvm.c
Removed patching-makefile.sh step, changed variable names
Maintains consistency with openNetVM 19.07
Using --disable-shared while building snort, does not build dynamic preprocessors.
Steps to build onvm-snort with openNetVM 19.07
|
Hi @dennisafa Thanks a lot for reviewing the changes. I've updated the pull request according to your suggestions above. |
|
@dennisafa I've merged commits into a more manageable number. You could merge these if the changes are approved. Thanks. |
|
Hi, does this merge request require anymore changes? |
Hi Archit, I ran into a few issues when re-building on a fresh instance. I've had a lot of work pile up so I haven't gotten around to figuring out the exact issue quite yet. I will retry and submit a change to the install guide by this weekend. Thanks for your patience! |
|
Hey @archit-p let me know if you have any questions about the comments! |
Updated installation instructions
|
Hi, apologies for being away for so long. I have made the requested changes and updated the pull request! Thanks for reviewing! |
|
Hi @twood02! Could you look at the proposed changes? |
|
Thanks @archit-p! I'll be discussing this with Dennis tomorrow and expect to merge it after that. We already have other students who are interested in using your code so your contribution is greatly appreciated! |
This pull request updates the onvm-snort repository to run with onvm 19.07.
Changes were made in libdaq code and configure scripts. A gitignore file was added to ensure clean commits.