Adds Capability for NFs to Share CPU Cores (sdnfv#311)

This code introduces **EXPERIMENTAL** support to allow NFs to efficiently run on **shared** CPU cores.  NFs wait on semaphores when idle and are signaled by the manager when new packets arrive. Once the NF is in wake state, no additional notifications will be sent until it goes back to sleep.  Shared cpu variables for mgr are in the `nf_wakeup_info` structs, the NF shared cpu vars were moved to the `onvm_nf` struct.

 - The scaling/pthread model required modifications to the signal handling process. To accommodate proper signal handling and fix other issues, children will block all signals and only the parent will handle signals. When terminating the parent NF will tell all children to stop running, which in turn will turn their children to stop running, which in turn will tell... this ensures that the parent thread blocks until absolutely all descendants have exited (this is required because even though a pthread child is detached if the main process exits it will be abruptly killed)
 - Creating an `onvm_nf_context` to be malloced before nflib starts and has info about that specific NF (shutdown information/nf and nf_info structs). This struct replaces `onvm_nf_info` in most API calls. _In the next pr `onvm_nf_info` will be reworked into a nf initialization structure, more on that later_
 - Running the `onvm_nflib_start_signal_handler(nf_context, NULL);` call before `onvm_nflib_init` to ensure proper signal handling even on NF startup (pretty sure that resolves sdnfv#93)
 - `onvm_nflib_stop` is now executed from the main(), instead of inside `onvm_nflib_run`. Also, the `onvm_nflib_stop` will be automatically called for the children spawned by the scaling API.
  
Other related changes:

 - Stats `-v` mode gets an update with the shared cpu information if such is available
 - onvm gets a config structure which contains onvm configuration flags for now but might be expanded later. This will be stored in a memzone and shared with the NFs.

**API additions:**

 - `onvm_nflib_start_signal_handler(struct onvm_nf_context *nf_context, handle_signal_func signal_hanlder);`

**API changes:**

 - `onvm_nflib_init(int argc, char *argv[], const char *nf_tag, struct onvm_nf_info **nf_info_p);` -> `onvm_nflib_init(int argc, char *argv[], const char *nf_tag, struct onvm_nf_context *nf_context);`
 - `onvm_nflib_run_callback(struct onvm_nf_info *info, pkt_handler_func pkt_handler, callback_handler_func callback_handler);` -> `onvm_nflib_run_callback(struct onvm_nf_context *nf_context, pkt_handler_func pkt_handler, callback_handler_func callback_handler);`
 - `onvm_nflib_run(struct onvm_nf_info *info, pkt_handler_func pkt_handler);` -> `onvm_nflib_run(struct onvm_nf_context *nf_context, pkt_handler_func pkt_handler);`
 - `onvm_nflib_stop(struct onvm_nf_info *nf_info);` -> `onvm_nflib_stop(struct onvm_nf_context *nf_context);`

All code changes are featurized using INTERRUPT_SEM macro.

See the ONVM README.md file for more information and warnings.

Commit log:

* Shared CPU NFs (sdnfv#150)

* merges rebased to master with sdnfv#147

* reverting num_clients update.

* replace tabs with spaces

* Readme describing experimental shared CPU support

* Add usage and known limitations for shared CPU code

* Bug Fix: Fixes instance ID assignment race condition (sdnfv#183) (sdnfv#185)

Fixes a race condition in NF instance ID assignment.  Before, if two NFs were started at the same time, they would be assigned the same instance ID.  This change modifies logic in `onvm_nf_next_instance_id` to avoid assigning the same ID to two starting NFs.

* Bug Fix: Fixes instance ID assignment race condition
  - Fixes sdnfv#178 
  - Changed incrementation of next_instance_id in onvm_mgr/onvm_nf.c
such that the ID can't be reused before an NF becomes active.

* partial refactoring of api w accordance of nf_info

* onvm mgr compiles

* flow table compiles -- all examples compile

* changed onvm_get_rx and onvm_get_tx to get rx_q and tx_q associated with onvm_nf struct

* Did requested changes. Also updated newer NFs (such as arp_response and flow_tracker). Also did some small style changes in  some other documentation in onvm_nflib.h

* Corrected some incorrect references to nflib functions in NF_Dev.md

* removed typing error

* Minor cleanup

* Stats cleanup

 - Some stats were unused and broken(?) nuked them
 - added stats that are possible usefull to stdout (slightly ugly)

* Add functionality to scale same NF instances

* Fixed a few global vars for multithreading to work

* Code clean up, comments and style fixes

* Added scale return value

* Fixed nf_mode for running advanced rings

* Refactored the onvm_nf/onvm_nf_info passing

- Keeping onvm_nf_info small, moved vars to onvm_nf
- Added a callback function for the scaling child thread
- Cleaned up code

* Adv rings NF scaling, setup func for spawned NFS

- Adds ability to scale NFs with advanced rings
- Adds a setup function for NFs
- More docs and cleanup

* Fixed build errors, changed NF typedef func names

* Launching NFs with different state/pkt processing

 -  Moving nf_info as passed arg to packet_handler -> allows for spawned NF to access their own nf_info
 -  Launching new NFs now accepts arguments for packet_handler function, setup function, callback function, advanced rings function
 -  Add a data pointer to onvm_nflib_info to maintain state
 -  Working on a scaling example (currently testing code, not in usable state) will showcase different features of spawning NFs

* Major changes, init breakdown, argv for spawned nf

- This is a major change, a lot of nflib initlization is refactored to allow spawned NFs to launch with their own service id
- There is a bug when the spawn a lot of new NFs we can no longer retrieve the nf_info from nf_info_mp, this needs to be fixed as it breaks stuff

* Fixing bugs, cleaning up code, updating example NF

- Fixed the nf_info bug where the mempool would get messed up as children weren't cleared properly.
- Updated the example scaling NF
- Removed most of the debugging code

* Clean up of shared cpu branch

Cleaned up messy bits, replacing macro branches with if statements, encapsulating code in functions, and getting rid of unnecessary code.

* The code cleaning commit has arrived

 - Style fixes
 - Removing debug code
 - Making code prettier
 - Adding better docs
 - Implementing requested changes

* Minor cleanup/whitespace fixes

* Fix Large ServieNum port crash bug

* Implementing requested changes

* Update scaling  docs

* Doc typos fixed

* Doc fixes

* Performace fixes

 - Changed MAX_NFS back to 128, improves performance
 - Changed the order inside of the onvm_nf struct, due to struct packing this reduced the overall size of onvm_nf down to 168 from 176

* Fixed memzone init size issue

* Update Releases.md

* Fix space

* initial pthread testing

* Small fixes

* small fixes

* More minor fixes

* More tiny fixes

* Scaling nf fixes

* Small fixes

* Logging, README fixes

* Added Manual/MGR core assignment mode, cleanup

* Added dedicated core option, error codes

* Combining shared cpu & pthreads

 - Dedicated pthread to watch for signals, this ensures that when user sends a SIGINT the threads are properly woken up and cleaned up. Not possible with the previous signal handler as it didn't have NF infromation

* Implemented shared cpu mutex logic for adv rings

* Add hacky shared cpu fix

* Fix some unused/old code

* Minor fixes, cleanup

* Remove testing print statement

* Remove unused func

* Fix shutdown thread join and cleanup

* Shared CPU mode as an argument

 - Add shared cpu '-c' flag passed to onvm_mgr
 - Add custom flags memzone
 - Surround all shared cpu code with if(shared_cpu)
 - Overall cleanup

* Update README.md

* Update NF_Dev.md

* Fix onvm_manager README

* Scaling nf share cpu flag

* Style fixes

* Cleanup

* Code cleanup

* Cleanup

* Onvm config struct for flags

* Refactoring and things

* Remove unused vars, rename var

* Fix stats & remove unrelated code

* Style fixes

* Fix imports in speed tester

* Fix race by properly killing grandchildren

* Rename vars, cleanup, comments

* Final cleanup

* Minore style fixes

* Minore style fixes

* Only cleanup shm when in shared cpu mode

* Fix unused macro

* Fix flag_p name to sleep_state

* Fix comment about signal thread

* Fix some style issues

* Fix scaling adv ring children termination

* Fix sleep issue with shared cpu scaling

* Fix cleanup, logical redesign

* The bug is strong with this one

* Fix type

* Pre-init signaling thread

* Better pre-init termination

* Working on combining signal handlers

* Remove unused dpdk_init_finished var

* Rewrote some termination logic

* Add defines in header file

* Fix signal handling, add nf context (api changes)

 - For proper termination of both advanced rings and normal NFs create
   nf context which is malloced as opposed to rte_malloc and is created
   prior to nflib init.
 - To properly handle signals and simplify the process instead of using
   a dedicated pthread, use signal() + global termination context
 - Involves a bunch of API changes to handle the new context variable,
   some API changes are still missing

* Cleanup comments, remove unused vars

* Further cleanup

* Cleanup

* More cleanup/updates

* Resolve needed signaling issues

* Update NFs to new API

* Fix fake getopts hack

* Add custom nf specific sig handling

* Update docs with paper description

* Update NF_Dev.md

* Update NF_Dev.md

* update references to research papers

* Remove debug pring

* Make children_cnt atomic

* Fix docs and comments

* Fix style issues

* Redo termination cleanup

* Removed ugly flag, proposed by the one and only @nks5295

* Fixed API calls for all example NFs

* Remove extra onvm_nflib_stop call in SpeedTester

* Add time based stop

* Fix Basic Monitor NF

* Fixed iter cnt for shutdown

* Fix comment

* Make keep_running atomic

docs/NF_Dev.md

@@ -42,6 +42,20 @@ Example use of Advanced Rings can be seen in the speed_tester NF or the scaling
 NFs can scale by running multiple threads. For launching more threads the main NF had to be launched with more than 1 core. For running a new thread the NF should call `onvm_nflib_scale(struct onvm_nf_scale_info *scale_info)`. The `struct scale_info` has all the required information for starting a new child NF, service and instance ids, NF state data, and the packet handling functions. The struct can be obtained either by calling the `onvm_nflib_get_empty_scaling_config(struct onvm_nf_info *parent_info)` and manually filling it in or by inheriting the parent behavior by using `onvm_nflib_inherit_parent_config(struct onvm_nf_info *parent_info)`. As the spawned NFs are threads they will share all the global variables with its parent, the `onvm_nf_info->data` is a void pointer that should be used for NF state data.
 Example use of Multithreading NF scaling functionality can be seen in the scaling_example NF.
 
+### Shared CPU mode
+This is an **EXPERIMENTAL** mode for OpenNetVM. It allows multiple NFs to run on a shared core.  In "normal" OpenNetVM, each NF will poll its RX queue for packets, monopolizing the CPU even if it has a low load.  This branch adds a semaphore-based communication system so that NFs will block when there are no packets available.  The NF Manger will then signal the semaphore once one or more packets arrive.
+
+This code allows you to evaluate resource management techniques for NFs that share cores, however it has not been fully tested with complex NFs, therefore if you encounter any bugs please create an issue or a pull request with a proposed fix.
+
+The code is based on the hybrid-polling model proposed in [_Flurries: Countless Fine-Grained NFs for Flexible Per-Flow Customization_ by Wei Zhang, Jinho Hwang, Shriram Rajagopalan, K. K. Ramakrishnan, and Timothy Wood, published at _Co-NEXT 16_][flurries_paper] and extended in [_NFVnice: Dynamic Backpressure and Scheduling for NFV Service Chains_ by Sameer G. Kulkarni, Wei Zhang, Jinho Hwang, Shriram Rajagopalan, K. K. Ramakrishnan, Timothy Wood, Mayutan Arumaithurai and Xiaoming Fu, published at _SIGCOMM '17_][nfvnice_paper]. Note that this code does not contain the full Flurries or NFVnice systems, only the basic support for shared-CPU NFs.
+
+Usage / Known Limitations:
+  - To enable pass a `-c` flag to the onvm_mgr, and use a `-s` flag when starting a NF to specify that they want to share cores
+  - All code for sharing CPUs is within `if (ONVM_ENABLE_SHARED_CPU)` blocks
+  - When enabled, you can run multiple NFs on the same CPU core with much less interference than if they are polling for packets
+  - This code does not provide any particular intelligence for how NFs are scheduled or when they wakeup/sleep
+  - Note that the manager threads all still use polling
+
 Packet Helper Library
 --
 
@@ -103,4 +117,5 @@ values.
 [flow_director]: ../onvm/onvm_nflib/onvm_flow_dir.h
 [srvc_chains]: ../onvm/onvm_nflib/onvm_sc_common.h
 [msg_passing]: ../onvm/onvm_nflib/onvm_msg_common.h
-
+[flurries_paper]: https://dl.acm.org/citation.cfm?id=2999602
+[nfvnice_paper]: https://dl.acm.org/citation.cfm?id=3098828

examples/aes_decrypt/aesdecrypt.c

@@ -215,23 +215,36 @@ packet_handler(struct rte_mbuf *pkt, struct onvm_pkt_meta *meta, __attribute__((
 int
 main(int argc, char *argv[]) {
         int arg_offset;
-
+        struct onvm_nf_context *nf_context;
         const char *progname = argv[0];
 
-        if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, &nf_info)) < 0)
-                return -1;
+        nf_context = onvm_nflib_init_nf_context();
+        onvm_nflib_start_signal_handler(nf_context, NULL);
+
+        if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, nf_context)) < 0) {
+                onvm_nflib_stop(nf_context);
+                if (arg_offset == ONVM_SIGNAL_TERMINATION) {
+                        printf("Exiting due to user termination\n");
+                        return 0;
+                } else {
+                        rte_exit(EXIT_FAILURE, "Failed ONVM init\n");
+                }
+        }
+
         argc -= arg_offset;
         argv += arg_offset;
 
         if (parse_app_args(argc, argv, progname) < 0) {
-                onvm_nflib_stop(nf_info);
+                onvm_nflib_stop(nf_context);
                 rte_exit(EXIT_FAILURE, "Invalid command-line arguments\n");
         }
 
         /* Initialise encryption engine. Key should be configurable. */
         aes_key_setup(key[0], key_schedule, 256);
 
-        onvm_nflib_run(nf_info, &packet_handler);
+        onvm_nflib_run(nf_context, &packet_handler);
+
+        onvm_nflib_stop(nf_context);
         printf("If we reach here, program is ending\n");
         return 0;
 }

examples/aes_encrypt/aesencrypt.c

@@ -214,24 +214,38 @@ packet_handler(struct rte_mbuf *pkt, struct onvm_pkt_meta *meta, __attribute__((
 
 int
 main(int argc, char *argv[]) {
+        struct onvm_nf_context *nf_context;
         int arg_offset;
 
         const char *progname = argv[0];
 
-        if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, &nf_info)) < 0)
-                return -1;
+        nf_context = onvm_nflib_init_nf_context();
+        onvm_nflib_start_signal_handler(nf_context, NULL);
+
+        if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, nf_context)) < 0) {
+                onvm_nflib_stop(nf_context);
+                if (arg_offset == ONVM_SIGNAL_TERMINATION) {
+                        printf("Exiting due to user termination\n");
+                        return 0;
+                } else {
+                        rte_exit(EXIT_FAILURE, "Failed ONVM init\n");
+                }
+        }
+
         argc -= arg_offset;
         argv += arg_offset;
 
         if (parse_app_args(argc, argv, progname) < 0) {
-                onvm_nflib_stop(nf_info);
+                onvm_nflib_stop(nf_context);
                 rte_exit(EXIT_FAILURE, "Invalid command-line arguments\n");
         }
 
         /* Initialise encryption engine. Key should be configurable. */
         aes_key_setup(key[0], key_schedule, 256);
 
-        onvm_nflib_run(nf_info, &packet_handler);
+        onvm_nflib_run(nf_context, &packet_handler);
+
+        onvm_nflib_stop(nf_context);
         printf("If we reach here, program is ending\n");
         return 0;
 }

examples/arp_response/arp_response.c

@@ -311,33 +311,45 @@ packet_handler(struct rte_mbuf *pkt, struct onvm_pkt_meta *meta, __attribute__((
 int
 main(int argc, char *argv[]) {
         int arg_offset;
+        struct onvm_nf_context *nf_context;
         const char *progname = argv[0];
 
-        if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, &nf_info)) < 0) {
-                return -1;
+        nf_context = onvm_nflib_init_nf_context();
+        onvm_nflib_start_signal_handler(nf_context, NULL);
+
+        if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, nf_context)) < 0) {
+                onvm_nflib_stop(nf_context);
+                if (arg_offset == ONVM_SIGNAL_TERMINATION) {
+                        printf("Exiting due to user termination\n");
+                        return 0;
+                } else {
+                        rte_exit(EXIT_FAILURE, "Failed ONVM init\n");
+                }
         }
 
         argc -= arg_offset;
         argv += arg_offset;
 
         state_info = rte_calloc("state", 1, sizeof(struct state_info), 0);
         if (state_info == NULL) {
-                onvm_nflib_stop(nf_info);
+                onvm_nflib_stop(nf_context);
                 rte_exit(EXIT_FAILURE, "Unable to initialize NF state");
         }
 
         state_info->pktmbuf_pool = rte_mempool_lookup(PKTMBUF_POOL_NAME);
         if (state_info->pktmbuf_pool == NULL) {
-                onvm_nflib_stop(nf_info);
+                onvm_nflib_stop(nf_context);
                 rte_exit(EXIT_FAILURE, "Cannot find mbuf pool!\n");
         }
 
         if (parse_app_args(argc, argv, progname) < 0) {
-                onvm_nflib_stop(nf_info);
+                onvm_nflib_stop(nf_context);
                 rte_exit(EXIT_FAILURE, "Invalid command-line arguments");
         }
 
-        onvm_nflib_run(nf_info, &packet_handler);
+        onvm_nflib_run(nf_context, &packet_handler);
+
+        onvm_nflib_stop(nf_context);
         printf("If we reach here, program is ending\n");
         return 0;
 }

examples/basic_monitor/monitor.c

@@ -180,24 +180,37 @@ packet_handler(struct rte_mbuf *pkt, struct onvm_pkt_meta *meta, __attribute__((
 
 int
 main(int argc, char *argv[]) {
+        struct onvm_nf_context *nf_context;
         int arg_offset;
         const char *progname = argv[0];
 
-        if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, &nf_info)) < 0)
-                return -1;
+        nf_context = onvm_nflib_init_nf_context();
+        onvm_nflib_start_signal_handler(nf_context, NULL);
+
+        if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, nf_context)) < 0) {
+                onvm_nflib_stop(nf_context);
+                if (arg_offset == ONVM_SIGNAL_TERMINATION) {
+                        printf("Exiting due to user termination\n");
+                        return 0;
+                } else {
+                        rte_exit(EXIT_FAILURE, "Failed ONVM init\n");
+                }
+        }
 
         argc -= arg_offset;
         argv += arg_offset;
 
         if (parse_app_args(argc, argv, progname) < 0) {
-                onvm_nflib_stop(nf_info);
+                onvm_nflib_stop(nf_context);
                 rte_exit(EXIT_FAILURE, "Invalid command-line arguments\n");
         }
 
         cur_cycles = rte_get_tsc_cycles();
         last_cycle = rte_get_tsc_cycles();
 
-        onvm_nflib_run_callback(nf_info, &packet_handler, &callback_handler);
+        onvm_nflib_run_callback(nf_context, &packet_handler, &callback_handler);
+
+        onvm_nflib_stop(nf_context);
         printf("If we reach here, program is ending\n");
         return 0;
 }

examples/bridge/bridge.c

@@ -161,20 +161,33 @@ packet_handler(struct rte_mbuf *pkt, struct onvm_pkt_meta *meta, __attribute__((
 int
 main(int argc, char *argv[]) {
         int arg_offset;
-
+        struct onvm_nf_context *nf_context;
         const char *progname = argv[0];
 
-        if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, &nf_info)) < 0)
-                return -1;
+        nf_context = onvm_nflib_init_nf_context();
+        onvm_nflib_start_signal_handler(nf_context, NULL);
+
+        if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, nf_context)) < 0) {
+                onvm_nflib_stop(nf_context);
+                if (arg_offset == ONVM_SIGNAL_TERMINATION) {
+                        printf("Exiting due to user termination\n");
+                        return 0;
+                } else {
+                        rte_exit(EXIT_FAILURE, "Failed ONVM init\n");
+                }
+        }
+
         argc -= arg_offset;
         argv += arg_offset;
 
         if (parse_app_args(argc, argv, progname) < 0) {
-                onvm_nflib_stop(nf_info);
+                onvm_nflib_stop(nf_context);
                 rte_exit(EXIT_FAILURE, "Invalid command-line arguments\n");
         }
 
-        onvm_nflib_run(nf_info, &packet_handler);
+        onvm_nflib_run(nf_context, &packet_handler);
+
+        onvm_nflib_stop(nf_context);
         printf("If we reach here, program is ending\n");
         return 0;
 }

examples/firewall/firewall.c

@@ -353,29 +353,41 @@ struct onvm_fw_rule
 }
 
 int main(int argc, char *argv[]) {
-        int arg_offset, num_rules;
+        struct onvm_nf_context *nf_context;
         struct onvm_fw_rule **rules;
+        int arg_offset, num_rules;
 
+        const char *progname = argv[0];
         stats.pkt_drop = 0;
         stats.pkt_accept = 0;
 
-        const char *progname = argv[0];
+        nf_context = onvm_nflib_init_nf_context();
+        onvm_nflib_start_signal_handler(nf_context, NULL);
+
+        if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, nf_context)) < 0) {
+                onvm_nflib_stop(nf_context);
+                if (arg_offset == ONVM_SIGNAL_TERMINATION) {
+                        printf("Exiting due to user termination\n");
+                        return 0;
+                } else {
+                        rte_exit(EXIT_FAILURE, "Failed ONVM init\n");
+                }
+        }
 
-        if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, &nf_info)) < 0)
-                return -1;
         argc -= arg_offset;
         argv += arg_offset;
 
         if (parse_app_args(argc, argv, progname) < 0) {
-                onvm_nflib_stop(nf_info);
+                onvm_nflib_stop(nf_context);
                 rte_exit(EXIT_FAILURE, "Invalid command-line arguments\n");
         }
 
         rules = setup_rules(&num_rules, rule_file);
         lpm_setup(rules, num_rules);
-        onvm_nflib_run(nf_info, &packet_handler);
+        onvm_nflib_run(nf_context, &packet_handler);
 
         lpm_teardown(rules, num_rules);
+        onvm_nflib_stop(nf_context);
         printf("If we reach here, program is ending\n");
         return 0;
 }

examples/flow_table/flow_table.c

@@ -71,6 +71,7 @@
 #define NF_TAG "flow_table"
 
 /* Struct that contains information about this NF */
+struct onvm_nf_context *global_termination_context;
 struct onvm_nf_info *nf_info;
 
 struct rte_ring *ring_to_sdn;
@@ -99,7 +100,7 @@ setup_rings(void) {
         ring_to_sdn = rte_ring_create("ring_to_sdn", SDN_RING_SIZE, rte_socket_id(), RING_F_SP_ENQ | RING_F_SC_DEQ);
         ring_from_sdn = rte_ring_create("ring_from_sdn", SDN_RING_SIZE, rte_socket_id(), RING_F_SP_ENQ | RING_F_SC_DEQ);
         if (ring_to_sdn == NULL || ring_from_sdn == NULL) {
-                onvm_nflib_stop(nf_info);
+                onvm_nflib_stop(global_termination_context);
                 rte_exit(EXIT_FAILURE, "Unable to create SDN rings\n");
         }
 }
@@ -235,7 +236,7 @@ packet_handler(struct rte_mbuf *pkt, struct onvm_pkt_meta *meta, __attribute__((
                 printf("Error in flow lookup: %d (ENOENT=%d, EINVAL=%d)\n", tbl_index, ENOENT, EINVAL);
                 onvm_pkt_print(pkt);
 #endif
-                onvm_nflib_stop(nf_info);
+                onvm_nflib_stop(global_termination_context);
                 rte_exit(EXIT_FAILURE, "Error in flow lookup\n");
         }
 
@@ -251,19 +252,33 @@ packet_handler(struct rte_mbuf *pkt, struct onvm_pkt_meta *meta, __attribute__((
 
 int
 main(int argc, char *argv[]) {
-        int retval;
+        int arg_offset;
+        struct onvm_nf_context *nf_context;
         unsigned sdn_core = 0;
 
-        if ((retval = onvm_nflib_init(argc, argv, NF_TAG, &nf_info)) < 0)
-                return -1;
-        argc -= retval;
-        argv += retval;
+        nf_context = onvm_nflib_init_nf_context();
+        global_termination_context = nf_context;
+        onvm_nflib_start_signal_handler(nf_context, NULL);
+
+        if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, nf_context)) < 0) {
+                onvm_nflib_stop(nf_context);
+                if (arg_offset == ONVM_SIGNAL_TERMINATION) {
+                        printf("Exiting due to user termination\n");
+                        return 0;
+                } else {
+                        rte_exit(EXIT_FAILURE, "Failed ONVM init\n");
+                }
+        }
+
+        argc -= arg_offset;
+        argv += arg_offset;
         if (parse_app_args(argc, argv) < 0) {
-                onvm_nflib_stop(nf_info);
+                onvm_nflib_stop(nf_context);
                 rte_exit(EXIT_FAILURE, "Invalid command-line arguments\n");
         }
         printf("Flow table running on %d\n", rte_lcore_id());
 
+        nf_info = nf_context->nf_info;
         def_destination = nf_info->service_id + 1;
         printf("Setting up hash table with default destination: %d\n", def_destination);
         total_flows = 0;
@@ -278,7 +293,9 @@ main(int argc, char *argv[]) {
         /* Map sdn_ft table */
         onvm_flow_dir_nf_init();
         printf("Starting packet handler.\n");
-        onvm_nflib_run(nf_info, &packet_handler);
+        onvm_nflib_run(nf_context, &packet_handler);
+
+        onvm_nflib_stop(nf_context);
         printf("NF exiting...\n");
         cleanup();
         return 0;