add Xena functionality #153

grdumas · 2020-08-10T17:21:47Z

Adds argument parser option, allows user to specify a particular module of a Xena chassis.

(@ctrautma for reference)

ctrautma

Looks OK to me.

k-rister

I would prefer if we were adding code in larger, more usable chunks. For example, this patch just adds a parameter and doesn't actually do anything with it.

grdumas · 2020-08-10T19:26:36Z

Revised commit; adds Xena throughput test functionality.

ctrautma

Will test. Then give approval and other comments.

k-rister

I'm confused by the patches to binary-search.py. This looks like it is just calling Valkyrie and then exiting. Shouldn't this be happening inside the binary search loop?

if _DEBUG = True, XenaVerify.py will print out additional info about the parameters and values it receives and the command output to Mono

Parameter sets the maximum transmission rate of a Valkyrie2544 test

k-rister · 2020-09-01T20:01:15Z

@ctrautma I think we can restart work on this. Obviously the first step we need to do is fix the merge conflicts.

ctrautma · 2020-09-03T01:48:03Z

OK I will take a look.

…

----- Original Message ----- From: "Karl Rister" <notifications@github.com> To: "atheurer/trafficgen" <trafficgen@noreply.github.com> Cc: "Christian Trautman" <ctrautma@redhat.com>, "Mention" <mention@noreply.github.com> Sent: Tuesday, September 1, 2020 4:01:30 PM Subject: Re: [atheurer/trafficgen] add Xena functionality (#153) @ctrautma I think we can restart work on this. Obviously the first step we need to do is fix the merge conflicts. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or unsubscribe .

k-rister · 2020-09-08T14:53:45Z

README-xena-trafficgen-integration.md

+3. Obtain .x2544 config file & executable
+   * Create a test config inside xena2544 utility on the xenaweb system (ask @ctrautma for IP if necessary)
+   * Save config into the ftp folder off the c drive
+   * Use FTP commands to download the config and the latest copy of Valkyrie2544.exe
+   * Place both config and exe inside the trafficgen folder
+```
+ftp 11.11.111.11 [user]/[pwd]
+bin
+get valkyrie2544.exe
+get <your config file>


Is there anything more generic we could say her that would be useful to other people who might try to use this?

k-rister · 2020-09-08T14:56:18Z

README-xena-trafficgen-integration.md

+5. Install TRex
+   * Within trafficgen, open `install-trex.sh` for editing
+   * Make the following change:
+        * original: `if curl --output ${tarfile} ${trex_url} && tar zxf ${tarfile}; then`
+        * modified: ``if curl --insecure --output ${tarfile} ${trex_url} && tar zxf ${tarfile}; then``
+   * As root or using `sudo`, run: `./install-trex.sh`
+   * After install completes, replace the following files with copies from test ZIP `fix_trex_install` (or make changes listed in Known Issues):
+        * `trex_client.py` --> `/opt/trex/current/automation/trex_control_plane/interactive/trex/common/trex_client.py`
+        * `trex_conn.py` --> `/opt/trex/current/automation/trex_control_plane/interactive/trex/common/trex_conn.py`
+        * `trex_global_stats.py` --> `/opt/trex/current/automation/trex_control_plane/interactive/trex/common/stats/trex_global_stats.py`


Why is TRex stuff included here?

Also, lines 41-43 have been addressed directly in install-trex.sh.

Also a bit confused as to the relevance/reason why lines 45-48 exist?

The TRex inclusions are because of the following line from the beginning of binary-search.py:
from trex_tg_lib import *

Importing everything from trex_tg_lib in turn kicks off a chain of other imports - end result is that without TRex installed on a system, binary search crashes. For the sake of clarity I'll address the other questions in their own respective comments.

Ok, that makes sense. I'll have to think if there is a way to work around this. Having to install TRex when you aren't using it just seems so broken...

@ctrautma

I put together a patch that I think will solve these TRex dependency issues:

k-rister@fa7235c

Can you have somebody test if this patch allows the Valkrie2544 stuff (this PR) to work without having the TRex stuff present as is described in the proposed README files?

k-rister · 2020-09-08T14:58:16Z

README-xena-trafficgen-integration.md

+## Known Issues
+* SyntaxError from `trex_client.py`
+  * Open `/opt/trex/current/automation/trex_control_plane/interactive/trex/common/trex_client.py`
+  * Change line 1188 from: `self.conn.async.set_timeout_sec(timeout_sec)` to: `self.conn.async_.set_timeout_sec(timeout_sec)`
+
+* SyntaxError from `trex_conn.py`
+  * Open `/opt/trex/current/automation/trex_control_plane/interactive/trex/common/trex_conn.py`
+  * Change line 36 from: `self.async = TRexSubscriber(self.ctx, self.rpc)` to: `self.async_ = TRexSubscriber(self.ctx, self.rpc)`
+  * Change line 60 from: `self.async.disconnect()` to: `self.async_.disconnect()`
+  * Change line 89 from: `self.async.barrier()` to: `self.async_.barrier()`
+  * Change line 98 from: `return self.async.barrier(baseline = True)` to: `return self.async_.barrier(baseline = True)`
+  * Change line 110 from: `self.async.set_as_zombie()` to: `self.async_.set_as_zombie()`
+  * Change line 143 from: `return ( self.async.last_data_recv_ts is not None and ((time.time() - self.async.last_data_recv_ts) <= 3) )` to `return ( self.async_.last_data_recv_ts is not None and ((time.time() - self.async_.last_data_recv_ts) <= 3) )`
+  * Change line 189 from: `rc = self.async.connect()` to `rc = self.async_.connect()`
+
+* SyntaxError from `trex_global_stats.py`
+  * Open `/opt/trex/current/automation/trex_control_plane/interactive/trex/common/stats/trex_global_stats.py`
+  * Change line 81 from: `("async_util.", "{0}% / {1}".format( format_threshold(round_float(self.client.conn.async.monitor.get_cpu_util()), [85, 100], [0, 85]),` to: `("async_util.", "{0}% / {1}".format( format_threshold(round_float(self.client.conn.async_.monitor.get_cpu_util()), [85, 100], [0, 85]),`
+  * Change line 82 from: `format_num(self.client.conn.async.monitor.get_bps() / 8.0, suffix = "B/sec"))),` to: `format_num(self.client.conn.async_.monitor.get_bps() / 8.0, suffix = "B/sec"))),`


Again, what relevance does TRex have to Valkyrie2544?

We don't have problems like this when running TRex so I'm confused about why this stuff is here and why these problems are being seen.

Egads, I thought I had commented on this already - my apologies! Back at the end of August I wrote this README as a "setup from scratch" sort of guide, which presumed the user had nothing already installed or configured. I walked through the process using a fresh Fedora 32 install, and wrote down the bare minimum steps necessary just to get the Xena functionality working.

(At the time of writing the patch) binary-search.py required that a user also install TRex in order to function (as mentioned in a previous comment). But even after installing TRex I still couldn't run binary-search.py due to several crash-inducing errors I'd never seen before. I still don't know why these problems suddenly popped up, but I eventually traced the errors back to the TRex library. Perhaps some update from Cisco unintentionally introduced a bug? I also can't rule out that the problem might have been self-inflicted from some flaw in my install/setup process.

Whatever the cause, I found that binary-search.py would stop crashing (at least enough so that the Xena bits worked) if I made a few tweaks to the TRex install. Hacky? Absolutely. But I lacked the necessary knowledge or skills to sort this problem out (or the time to develop such competencies). So I wrote those workarounds into the README in the hopes they'd be "good-enough" for folks needing to use Xena in the short term, at least while the TRex dependency issues persisted.

TLDR; the TRex changes were just duct tape and will hopefully become unneeded/irrelevant/obsolete and removed as soon as possible.

k-rister · 2020-09-08T14:59:46Z

binary-search.py

+    if t_global.args.traffic_generator == 'valkyrie2544' and (t_global.args.num_flows < 1 or t_global.args.num_flows > 1000000):
+        bs_logger('Please enter a number of flows between 1 and 1000000')
+        bs_logger_cleanup(bs_logger_exit, bs_logger_thread)
+        return(1)


I wonder if we shouldn't make this a generic error that isn't specific to Valkyrie2544...

ctrautma approved these changes Aug 10, 2020

View reviewed changes

k-rister suggested changes Aug 10, 2020

View reviewed changes

add Xena functionality

6f90db7

grdumas force-pushed the master branch from 799801c to 6f90db7 Compare August 10, 2020 19:24

grdumas requested a review from k-rister August 10, 2020 19:26

grdumas changed the title ~~add parameter to specify a Xena module number~~ add Xena functionality Aug 10, 2020

ctrautma reviewed Aug 10, 2020

View reviewed changes

k-rister reviewed Aug 10, 2020

View reviewed changes

grdumas and others added 16 commits August 21, 2020 13:14

read report filename from config

3c659ca

add option to print debug info

42621af

if _DEBUG = True, XenaVerify.py will print out additional info about the parameters and values it receives and the command output to Mono

Add --valkyrie2544-max_tput

85b8ffa

Parameter sets the maximum transmission rate of a Valkyrie2544 test

fix Valkyrie2544 parser arguments

4d97a78

handle invalid Valkyrie2544 test options

050879f

fix Valkyrie2544 parameter processing

89118c7

fix Valkyrie2544 packet_size parameter

ed504c6

remove unused imports

bc470fd

fix arg parser whitespace

5a9c848

update initial_tput and max_rate parser args

5e5044f

print extra info if _DEBUG = True

b5ee509

revise --valkyrie2544-debug help test

855d51c

fix --packet_sizes and --initial_tput

bbca7ff

fix --initial_tput and --max_tput

ef6c516

add README overview of trafficgen/xena integration project

13eb33d

update Xena/trafficgen project READMEs

aade65d

Merge branch 'master' into master

dfe32d8

k-rister suggested changes Sep 8, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add Xena functionality #153

add Xena functionality #153

grdumas commented Aug 10, 2020

ctrautma left a comment

k-rister left a comment

grdumas commented Aug 10, 2020

ctrautma left a comment

k-rister left a comment

k-rister commented Sep 1, 2020

ctrautma commented Sep 3, 2020 via email

k-rister Sep 8, 2020

k-rister Sep 8, 2020

grdumas Sep 8, 2020

k-rister Sep 8, 2020

k-rister Sep 8, 2020

k-rister Sep 8, 2020

grdumas Sep 25, 2020

k-rister Sep 8, 2020

add Xena functionality #153

Are you sure you want to change the base?

add Xena functionality #153

Conversation

grdumas commented Aug 10, 2020

ctrautma left a comment

Choose a reason for hiding this comment

k-rister left a comment

Choose a reason for hiding this comment

grdumas commented Aug 10, 2020

ctrautma left a comment

Choose a reason for hiding this comment

k-rister left a comment

Choose a reason for hiding this comment

k-rister commented Sep 1, 2020

ctrautma commented Sep 3, 2020 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment