auto testing

TL;DR

Script can be installed on any Debian based Linux device on your network. Adapt the configuration-file to your requirements, start the test procedure and wait until its done. Check the logs/ for errors and summary in /reports

Privacy concerns

It runs locally in your network, no data is sent anywhere. Icons come from the internet.

Getting started

Clone, edit userconfig/configuration.sh and run the script. Script will scan your local network (if SUBNET is defined) otherwise it will cycle on IPs you have defined in HOSTS variable.

It will create logins based on the configuration file and run different tests/ to see if the board is working properly. You can start with predefined credentials or use your own.

All you need to do is configure the configuration file, hook up the power and network cable and you are ready for running the test.

That's it.

Prerequisites

Download the latest armbian image for your board(s). We recommend to use .torrent because this does the file integrity check for you automatically. Write the data to the SDcard

Put the SDcard into your device, power it up and let it sit for 1-2 minutes, to do the standard initial setup.

Prepare your system

1. Go to a folder where you want to store it. The following command will create a folder called 'autotests'. Clone the sources from Github and open the folder autotests:

apt install git
git clone https://github.com/armbian/autotests
cd autotests
./go.sh

2. Edit userconfig/configuration.sh
   • change number of passes (optional)
   • change stress time in seconds (optional)

What is your device(s) IP-Address or do you have a dedicated subnet for your devices. To find the device, check your router or use this tool, to find your boards IP-Address(es). - set IP-Address (HOSTS) or Subnet (Examle: 192.168.0.1/24) - set WLAN_SSID and password (2,4 and/or 5,0GHz) - set BLUEDEV MAC-Address of a Blueooth device (Android phone for example)

How to find the Bluetooth MAC-Address on your mobile or Linux device with BT? Put your phone's BT to discoverable mode and run:

hcitool scan

on a Linux computer that has a BT-Dongle. On an Android device you find the MAC-Address in the phones settings/about the phone/Status.

3. Run 👍 : The script will display which board gets tested

./go.sh

4. Once finished, go to /autotests/logs and check latest report.`

What this tool does?

It Connects to the host(s) or all Armbian hosts in your subnet and runs the tests found in folder tests in alphabetical order.

Example report:

	Board / Cycle	Version / distribution Kernel / variant	Lan	2.4Ghz	5Ghz	memory	storage	BT	Stress 6s	DVFS (Mhz)
			Iperf send/receive (MBits/s)			IO read/write (MBits/s)
1	Cubietruck ###.###.###.200
	1/2 19:16:37	20.02.1 (bionic) Linux 5.4.26-sunxi stable	sun7i-dwmac 550 - 728	brcmfmac		358 - 342	388 - 8			480 - 960
	2/2 19:21:22	20.02.1 (bionic) Linux 5.4.26-sunxi stable	sun7i-dwmac 521 - 744	brcmfmac		357 - 340	386 - 11			480 - 960
2	Tinkerboard ###.###.###.204
	1/2 19:25:24	20.02.1 (bionic) Linux 5.4.20-rockchip stable	rk_gmac-dwmac 941 - 908	rtl8723bs 24 mt7601u 99 rt2800usb 109		696 - 742	698 - 70			600 - 1800
	2/2 19:29:41	20.02.1 (bionic) Linux 5.4.20-rockchip stable	rk_gmac-dwmac 941 - 906	rtl8723bs 3 mt7601u 105 rt2800usb 113		692 - 740	731 - 68			600 - 1800

Which tests are executed?

test name	function
0000-read-board-data.bash	Read board OS information
0001-nigtly-stable-switch.bash	Switching between nightly and stable, odd/even
0002-update-and-upgrade.bash	Upgrade all packages
0008-connect-wireless-devices-on-2.4Ghz.bash	Connects wireless devices on 2.4G band
0013-iperf-on-all-wired-interfaces.bash	Check speed on all wired devices
0014-iperf-on-all-wireless-interfaces.bash	Check speed on all wireless devices
0015-connect-wireless-devices-on-5.0Ghz.bash	Connects wireless devices on 5G band
0017-iperf-on-all-wireless-interfaces.bash	Check speed on all wireless devices
0018-io-tests-memory.bash	Determine maximum memory random write speed
0019-io-tests-drive.bash	Determine maximum SD/eMMC random write speed
0111-bluetoth.bash	Ping Bluetooth device to check basic BT functionality
0115-strong-stressing.bash	Running heavy stressing for n seconds
0116-dvfs.bash	Get min and max CPU temperature to see if DVFS is operational
0119-7-zip-benchmark.bash.disabled	Run 7Zip benchmark
9999-reboot.bash	Reboot the board

To do's:

• parallel processing
• implement official SBC-bench with access to full logs
• improve errors catching
• implement iperf queue to prevent colisions on network testing
• create JSON data export for single board and together
• common data collecting
• support custom test board https://forum.armbian.com/topic/10841-the-testing-thread