Question about running ethereum's macro benchmark using Smallbank workload

[nicodechal]

Question about smallbank driver

I'm trying to run benchmark using Smallbank workload,But I'm not sure which one is right.
I saw the comment below:

blockbench/benchmark/ethereum/start-clients.sh

Lines 18 to 19 in d722cab

	#both ycsbc and smallbank use the same driver
	nohup ./driver -db ethereum -threads $1 -P workloads/workloada.spec -endpoint $host:8000 -txrate $4 -wt 60 > $LOG_DIR/client_$host"_"$1 2>&1 &

I'm not clear about what's the meaning of this comment, because from document, it seems that YCSB and Smallbank use different drivers, but code says I only need to add -wl smallbank to test using smallbank workload.

And I tried to use the driver in src/macro/smallbank as document say, but I got a all zero results,like

In the last 2s, tx count = 0 latency = 0 outstanding request = 0
In the last 2s, tx count = 0 latency = 0 outstanding request = 395
In the last 2s, tx count = 0 latency = 0 outstanding request = 783
In the last 2s, tx count = 0 latency = 0 outstanding request = 1169
...

and tx count and latency are always 0, only the outstanding request increase. I'm still not sure about why I got this results

---

Question about code

Another question is about the true in the code below:

blockbench/src/macro/kvstore/ycsbc.cc

Lines 146 to 147 in d722cab

	actual_ops.emplace_back(async(launch::async, DelegateClient, db, &wl,
	total_ops / num_threads, true, txrate));

If I change this true to false, does this means use YCSB simulate smallbank? because this is document say:

For Ethereum, smallbank is simulated via ycsb

In this way I can get some results, but the throughput I calculate is very large, like 10 times larger than YCSB workload. Is this result reasonable?

Hope to get help!

[yolo-612]

Hello, I‘m sorry that I’m not answering the question but asking the question. I'm just starting the simulation of the ethereum macro benchmark. It is totally new for me .. from where to start .. dont know .. Is there any manual (step by step) ? can you help me

[ug93tad]

The comment in "start-client.sh" means that both the executable for YCSB and Smallbank are named "driver". You need to change the path to the correct workload.

The "true" option in ycsbc.cc is to pre-load the data, has nothing to do with Smallbank.

"smallbank is simulated via ycsb" means that it simply carries multiple updates of different keys instead of checking for account/balance/etc. The smallbank driver for Hyperledger is a more precise implementation.

[nicodechal]

@ug93tad thank you for your reply!

So in Q2, when i change the true to false, then what's the meaning of the code below(Line 51, it uses method called DoTransaction)?

blockbench/src/macro/kvstore/ycsbc.cc

Lines 47 to 52 in d722cab

	if (is_loading) {
	oks += client.DoInsert();
	utils::sleep(tx_sleep_time);
	} else {
	oks += client.DoTransaction();
	}

---

Also, I have tried to use Smallbank workload in src/macro/smallbank, but I got some results like I mentioned before, Is there any possible reason for this result?

[ug93tad]

For Ethereum, both YCSB and Smallbank use the driver in ../macro/kvstore (see the Readme). All operations are "DoInsert()". The "DoTransaction()" is relics from the YCSB code we ported (https://github.com/basicthinker/YCSB-C).

The results you showed indicate that the server didn't produce any blocks. Please check your server logs to see what went wrong.

[nicodechal]

@ug93tad So if I want to test Ethereum using smallbank, I dont need to change driver and only need to use -wl smallbank option?

[ug93tad]

No, sorry my mistake. Smallbank uses the driver in ../smallbank directory

[nicodechal]

@ug93tad Thanks.
So can I use -wl smallbank with YCSB workload to simulate smallbank workload? because the result using driver in ../smallbank still give me wrong results and no error in server logs ... And I saw the usage in code:

blockbench/src/macro/kvstore/ycsbc.cc

Lines 253 to 254 in d722cab

	cout << " -wl workload: specify the type of smart contract to run (choices: "
	"ycsb, donothing, smallbank. By default: donothing)" << endl;

Can this still work?

[ug93tad]

yes, but for simulated workload only, and only for Ethereum. The correct smallbank should be run using the driver in ../smallbank folder.