[GSoC17] Add backend for AirOS8

[edoput]

Meta issue for tracking airos8 compatibility

• discovery section edoput/netjsonconfig#1, frozen with defaults values
• dyndns section edoput/netjsonconfig#2, frozen with defaults values
• gui section edoput/netjsonconfig#3, frozen with defaults values
• httpd section edoput/netjsonconfig#4, frozen with defaults values
• netconf section edoput/netjsonconfig#5, need netjson extension
• ntpclient section edoput/netjsonconfig#6
• pwdog section edoput/netjsonconfig#7, frozen with defaults values
• radio section edoput/netjsonconfig#8, need more work to understand
• resolv section edoput/netjsonconfig#9
• route section edoput/netjsonconfig#10
  need more work to understand how to take values from netjson
• snmp section edoput/netjsonconfig#11
• sshd section edoput/netjsonconfig#12
• Netmode edoput/netjsonconfig#15
• Aaa section edoput/netjsonconfig#16
• Syslog section edoput/netjsonconfig#17, frozen with defaults values, need netjson extension
• Telnetd section edoput/netjsonconfig#18, frozen with defaults values
• Update section edoput/netjsonconfig#19
• Users section edoput/netjsonconfig#20
• wireless section edoput/netjsonconfig#21
• wpasupplicant section edoput/netjsonconfig#22
• ntpclient section edoput/netjsonconfig#23, frozen with defaults values
• system section edoput/netjsonconfig#24, frozen with defaults values
• ebtables section edoput/netjsonconfig#26, needs work for vlan values
• igmpproxy section edoput/netjsonconfig#27
• iptables section edoput/netjsonconfig#28
• tshaper section edoput/netjsonconfig#29
• unms section edoput/netjsonconfig#30
• Add documentation for ubiquity update process to docs edoput/netjsonconfig#31

[nemesifier]

@edoput can you build a first version of your backend by the first evaluation?

We should split the work between the 3 months (we can create more issues), we could do something like:

• 1st phase: prototype backend, basic tests, basic docs
• 2nd phase: intensive testing on real devices, bugfixes, improve tests and docs
• 3rd phase: by this stage I'd like to test the backend direclty via openwisp2 interface but I'll need to do some extra work to make this possible - we can work together on this if you like, it also depends on how the situation evolves

Let me know your thoughts.

[edoput]

As of today I have the first version, I will fill more converters to see if it is solid then it is all about writing tests and simplifying what I have done

You can try it from ee19a4b, and it should work with some default values filled in for the user.

I would like to improve the schema and the converters a little bit before doing the review? Is it ok? ETA is 2/06

[nemesifier]

@edoput sure, go ahead

[edoput]

I've added the status of the issue to this meta list

There are some topics such as edoput#5 where I need a netjson extension, edoput#8 where I still have to understand what configures the values, edoput#15 where I have to use a different renderer.

Next is working with ubntcfg to understand if it can be used to validate my config and cfgmtd to persist some files to the underliying storage.

Wish me luck!

[nemesifier]

@edoput good luck!

You can add extra NetJSON attributes to the interfaces (or any other configuration you need). In the near future I want to find what is the lowest common denominator with different configuration backends and add those to the NetJSON spec.

I don't understand exactly what you mean for:

EdoPut/netjsonconfig#8 where I still have to understand what configures the values

and:

EdoPut/netjsonconfig#15 where I have to use a different renderer

Could you expand a bit?

[edoput]

In edoput#8 there are a lot of values that I still don't know what the meaning is and how to configure them from netjson
Here is the list

radio.1.ack.auto
radio.1.ackdistance
radio.1.acktimeout
radio.1.antenna.gain
radio.1.antenna.id
radio.1.cwm.enable
radio.1.cwm.mode
radio.1.dfs.status
radio.1.iee_mode
radio.1.mode
radio.1.obey
radio.1.polling
radio.1.pollingnoack
radio.1.reg_obey
radio.1.scan_list.status
radio.1.subsystemid

Right now I'm working with a lot of defaults and some help from @cappe87

In edoput#15 there is the only value that needs a different renderer.
Instead of format netmode.{{key}}={{value}} it uses the netmode=bridge/router so it is netmode={{value}}, that's it.

[edoput]

This is a diff from the "default" configuration.

Right now the "default" configuration can be obtained by setting the advanced network management in the web interface.

In red the antenna configuration

In green the output from netjsonconfig using this netjson that closely represent the default configuration

From this diff we can see that the netconf section is approaching completion while the radio section is far from completed.

There are two new section in airos v8.3, iptables and ebtables, which I have to work on.

wireless is near perfect and wpasupplicant begs for some work but I don't have the slightest idea of what it does and how to use it

--- /tmp/system.sorted
+++ /tmp/prova.sorted
@@ -1,34 +1,21 @@
-aaa.1.radius.acct.1.port=1813
-aaa.1.radius.acct.1.status=disabled
-aaa.1.radius.auth.1.port=1812
-aaa.1.status=disabled
 aaa.status=disabled
+bridge.1.comment=
 bridge.1.devname=br0
-bridge.1.port.1.devname=eth0
+bridge.1.port.1.devname=ath0
 bridge.1.port.1.status=enabled
-bridge.1.port.2.devname=ath0
+bridge.1.port.2.devname=eth0
 bridge.1.port.2.status=enabled
 bridge.1.status=enabled
-bridge.1.stp.status=disabled
+bridge.1.stp.status=enabled
 bridge.status=enabled
-discovery.cdp.status=disabled
+discovery.cdp.status=enabled
 discovery.status=enabled
-dyndns.1.servicename=dyndns.org
 dyndns.status=disabled
-ebtables.status=enabled
-ebtables.sys.fw.status=disabled
-ebtables.sys.status=enabled
-gui.language=en_US
-gui.network.advanced.status=enabled
 httpd.https.port=443
 httpd.https.status=enabled
 httpd.port=80
 httpd.session.timeout=900
 httpd.status=enabled
-igmpproxy.status=disabled
-iptables.status=disabled
-iptables.sys.portfw.status=disabled
-iptables.sys.status=enabled
 netconf.1.autoip.status=disabled
 netconf.1.devname=ath0
 netconf.1.mtu=1500
@@ -42,98 +29,44 @@
 netconf.2.mtu=1500
 netconf.2.status=enabled
 netconf.2.up=enabled
-netconf.3.autoip.status=enabled
 netconf.3.devname=br0
 netconf.3.ip=192.168.1.20
 netconf.3.mtu=1500
 netconf.3.netmask=255.255.255.0
 netconf.3.role=mlan
 netconf.3.status=enabled
+netconf.3.up=enabled
 netconf.status=enabled
 netmode=bridge
 ntpclient.1.server=0.ubnt.pool.ntp.org
-ntpclient.1.status=disabled
+ntpclient.1.status=enabled
 ntpclient.status=disabled
 pwdog.delay=300
 pwdog.period=300
 pwdog.retry=3
-pwdog.status=disabled
-radio.1.ack.auto=enabled
-radio.1.ackdistance=643
-radio.1.acktimeout=35
-radio.1.ampdu.frames=32
-radio.1.ampdu.status=enabled
-radio.1.antenna.gain=3
-radio.1.antenna.id=2
-radio.1.atpc.sta.status=enabled
-radio.1.atpc.status=disabled
-radio.1.atpc.threshold=36
-radio.1.cable.loss=0
-radio.1.center.1.freq=0
-radio.1.chanbw=0
-radio.1.cmsbias=0
-radio.1.countrycode=380
-radio.1.cwm.enable=0
-radio.1.cwm.mode=1
+pwdog.status=enabled
+radio.1.chanbw=20
 radio.1.devname=ath0
-radio.1.dfs.status=enabled
-radio.1.freq=0
-radio.1.ieee_mode=auto
-radio.1.low_txpower_mode=disabled
-radio.1.mode=managed
-radio.1.obey=enabled
-radio.1.polling=enabled
-radio.1.polling_11ac_11n_compat=0
-radio.1.polling_ff_dl_ratio=50
-radio.1.polling_ff_dur=0
-radio.1.polling_ff_timing=0
-radio.1.pollingnoack=0
-radio.1.pollingpri=2
-radio.1.ptpmode=1
-radio.1.reg_obey=enabled
-radio.1.rx_sensitivity=-96
-radio.1.scan_list.status=disabled
-radio.1.scanbw.status=disabled
 radio.1.status=enabled
-radio.1.subsystemid=0xe7f5
-radio.1.txpower=24
-radio.countrycode=380
+radio.1.txpower=20
 radio.status=enabled
-resolv.host.1.name=PowerBeam 5AC 400
-resolv.host.1.status=enabled
-resolv.nameserver.status=enabled
-route.1.devname=br0
-route.1.gateway=192.168.1.1
-route.1.ip=0.0.0.0
-route.1.netmask=0
-route.1.status=enabled
-route.status=enabled
 snmp.community=public
-snmp.status=disabled
+snmp.contact=
+snmp.location=
+snmp.status=enabled
 sshd.auth.passwd=enabled
 sshd.port=22
 sshd.status=enabled
-syslog.remote.port=514
-syslog.remote.status=disabled
-syslog.status=enabled
-system.airosx.prov.status=enabled
-system.cfg.version=65547
-system.date.status=disabled
+syslog.status=disabled
 system.external.reset=enabled
-system.timezone=GMT
 telnetd.port=23
 telnetd.status=disabled
-tshaper.status=disabled
-unms.status=disabled
 update.check.status=enabled
-users.1.name=ubnt
-users.1.password=$1$msTRdPTq$u57bwqEnMn4z2RA8KI6He/
-users.1.status=enabled
-users.2.gid=100
-users.2.shell=/bin/false
-users.2.status=disabled
-users.2.uid=100
-users.status=enabled
+users.1.name=root
+users.1.password=changeme
+users.1.status=disabled
+users.status=disabled
+vlan.status=enabled
 wireless.1.addmtikie=enabled
 wireless.1.devname=ath0
 wireless.1.hide_ssid=disabled
@@ -144,22 +77,14 @@
 wireless.1.rate.auto=enabled
 wireless.1.rate.mcs=-1
 wireless.1.security.type=none
-wireless.1.signal_led1=94
-wireless.1.signal_led2=80
-wireless.1.signal_led3=73
-wireless.1.signal_led4=65
+wireless.1.signal_led1=75
+wireless.1.signal_led2=50
+wireless.1.signal_led3=25
+wireless.1.signal_led4=15
 wireless.1.signal_led_status=enabled
 wireless.1.ssid=ubnt
 wireless.1.status=enabled
 wireless.1.wds.status=enabled
 wireless.status=enabled
-wpasupplicant.device.1.profile=AUTO
-wpasupplicant.device.1.status=enabled
-wpasupplicant.profile.1.name=AUTO
-wpasupplicant.profile.1.network.1.key_mgmt.1.name=NONE
-wpasupplicant.profile.1.network.1.priority=100
-wpasupplicant.profile.1.network.1.ssid=ubnt
-wpasupplicant.profile.1.network.2.key_mgmt.1.name=NONE
-wpasupplicant.profile.1.network.2.priority=2
-wpasupplicant.profile.1.network.2.status=disabled
-wpasupplicant.status=enabled
+wpasupplicant.device.1.status=disabled
+wpasupplicant.profile.1.network.1.ssid=your-ssid-here

[nemesifier]

@edoput wpasupplicant is to configure encryption (see encryption object in netjson), I suggest you to focus on this next.

Regarding iptables and ebtables, very cool to know AirOS supports those, we can work on those later, as a lower priority.

[nemesifier]

@edoput do you think we can start organizing a first merge somewhen soon?

[edoput]

succintly, we can do this tomorrow I would like to squash more (136 commit are too many) and code quality could use some improvements * the radio section is still a mistery, maybe it will work only on a powerbeam and we will have to come back at it later * the user section should require an unknown salt so it can't be done by me everything else should work 2017-07-12 16:54 GMT+02:00 Federico Capoano <notifications@github.com>:

…

@edoput <https://github.com/edoput> do you think we can start organizing a first merge somewhen soon? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#76 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AD4zudFftJAnkhkKOkEFu7bQ58V6qDKSks5sNN4ugaJpZM4NmQuH> .

[nemesifier]

@edoput great! No need to rush you can start creating the pull request so we can easily review it slowly together. I can suggest you some style improvements, test it, ecc. I would like to do it soonish but not in a rush.

[edoput]

I included the radio configuration as a blob and this is the first time I can use a generated configuration without having to modify anything by hand.

This is the output of the save procedure, as you can see there is only one important change, the bridge interface changed ip.

Unfortunately the ping watchdog was enabled in my configuration and I think it's gone crazy and restored the default configuration.

Other than disabling it I could investigate further with the support information I got to download. As it is a block it could be a core dump or just encrypted, it's hard to say.

WA.v8.3# /usr/etc/rc.d/rc.do.softrestart save
--- /tmp/.running.cfg.5487
+++ /tmp/.system.cfg.5487
@@ -3,6 +3,7 @@
 aaa.1.radius.auth.1.port=1812
 aaa.1.status=disabled
 aaa.status=disabled
+bridge.1.comment=
 bridge.1.devname=br0
 bridge.1.port.1.devname=eth0
 bridge.1.port.1.status=enabled
@@ -11,9 +12,8 @@
 bridge.1.status=enabled
 bridge.1.stp.status=disabled
 bridge.status=enabled
-discovery.cdp.status=disabled
+discovery.cdp.status=enabled
 discovery.status=enabled
-dyndns.1.servicename=dyndns.org
 dyndns.status=disabled
 ebtables.status=enabled
 ebtables.sys.fw.status=disabled
@@ -42,22 +42,28 @@
 netconf.2.mtu=1500
 netconf.2.status=enabled
 netconf.2.up=enabled
-netconf.3.autoip.status=enabled
 netconf.3.devname=br0
-netconf.3.ip=192.168.1.20
+netconf.3.ip=192.168.1.21
 netconf.3.mtu=1500
 netconf.3.netmask=255.255.255.0
 netconf.3.role=mlan
 netconf.3.status=enabled
+netconf.3.up=enabled
 netconf.status=enabled
 netmode=bridge
-ntpclient.1.server=0.ubnt.pool.ntp.org
-ntpclient.1.status=disabled
-ntpclient.status=disabled
+ntpclient.1.server=0.pool.ntp.org
+ntpclient.1.status=enabled
+ntpclient.2.server=1.pool.ntp.org
+ntpclient.2.status=enabled
+ntpclient.3.server=2.pool.ntp.org
+ntpclient.3.status=enabled
+ntpclient.4.server=3.pool.ntp.org
+ntpclient.4.status=enabled
+ntpclient.status=enabled
 pwdog.delay=300
 pwdog.period=300
 pwdog.retry=3
-pwdog.status=disabled
+pwdog.status=enabled
 radio.1.ack.auto=enabled
 radio.1.ackdistance=643
 radio.1.acktimeout=35
@@ -99,9 +105,9 @@
 radio.1.txpower=24
 radio.countrycode=380
 radio.status=enabled
-resolv.host.1.name=PowerBeam 5AC 400
+resolv.host.1.name=airos
 resolv.host.1.status=enabled
-resolv.nameserver.status=enabled
+resolv.status=enabled
 route.1.devname=br0
 route.1.gateway=192.168.1.1
 route.1.ip=0.0.0.0
@@ -109,7 +115,13 @@
 route.1.status=enabled
 route.status=enabled
 snmp.community=public
-snmp.status=disabled
+snmp.contact=edoput@lilik.it
+snmp.location=
+snmp.status=enabled
 sshd.auth.passwd=enabled
 sshd.port=22
 sshd.status=enabled
@@ -117,24 +129,20 @@
 syslog.remote.status=disabled
 syslog.status=enabled
 system.airosx.prov.status=enabled
-system.cfg.version=65547
+system.cfg.version=0
 system.date.status=disabled
 system.external.reset=enabled
 system.timezone=GMT
 telnetd.port=23
 telnetd.status=disabled
-tshaper.status=disabled
-unms.status=disabled
 update.check.status=enabled
 users.1.name=ubnt
 users.1.password=$1$yxUQm7Il$ViUqgyf1SvPmT4a.nTRdH/
 users.1.status=enabled
-users.2.gid=100
-users.2.shell=/bin/false
-users.2.status=disabled
-users.2.uid=100
 users.status=enabled
+vlan.status=enabled
 wireless.1.addmtikie=enabled
+wireless.1.ap=00:11:22:33:44:55
 wireless.1.devname=ath0
 wireless.1.hide_ssid=disabled
 wireless.1.l2_isolation=disabled
@@ -144,10 +152,10 @@
 wireless.1.rate.auto=enabled
 wireless.1.rate.mcs=-1
 wireless.1.security.type=none
-wireless.1.signal_led1=94
-wireless.1.signal_led2=80
-wireless.1.signal_led3=73
-wireless.1.signal_led4=65
+wireless.1.signal_led1=75
+wireless.1.signal_led2=50
+wireless.1.signal_led3=25
+wireless.1.signal_led4=15
 wireless.1.signal_led_status=enabled
 wireless.1.ssid=ubnt
 wireless.1.status=enabled
Fast system script build Success.
Fast users script build Success.
Fast resolv script build Success.
Fast wireless script build Success.
Fast iptables script build Success. Disabled plugin from startup list
Has not fast vlan script?
Fixup Startup_list ...Done.
Connection to 192.168.1.20 closed by remote host.
Connection to 192.168.1.20 closed.

[edoput]

The very same configuration, but with pwdog disabled succeded in configuring the antenna.

But at login I was presented with the very same popup, as I firstly dismissed it it may be about the previous pwdog incident.

I'm attaching the log of the second attempt

WA.v8.3# /usr/etc/rc.d/rc.do.softrestart save
--- /tmp/.running.cfg.2646
+++ /tmp/.system.cfg.2646
@@ -3,6 +3,7 @@
 aaa.1.radius.auth.1.port=1812
 aaa.1.status=disabled
 aaa.status=disabled
+bridge.1.comment=
 bridge.1.devname=br0
 bridge.1.port.1.devname=eth0
 bridge.1.port.1.status=enabled
@@ -11,9 +12,8 @@
 bridge.1.status=enabled
 bridge.1.stp.status=disabled
 bridge.status=enabled
-discovery.cdp.status=disabled
+discovery.cdp.status=enabled
 discovery.status=enabled
-dyndns.1.servicename=dyndns.org
 dyndns.status=disabled
 ebtables.status=enabled
 ebtables.sys.fw.status=disabled
@@ -42,18 +42,24 @@
 netconf.2.mtu=1500
 netconf.2.status=enabled
 netconf.2.up=enabled
-netconf.3.autoip.status=enabled
 netconf.3.devname=br0
-netconf.3.ip=192.168.1.20
+netconf.3.ip=192.168.1.21
 netconf.3.mtu=1500
 netconf.3.netmask=255.255.255.0
 netconf.3.role=mlan
 netconf.3.status=enabled
+netconf.3.up=enabled
 netconf.status=enabled
 netmode=bridge
-ntpclient.1.server=0.ubnt.pool.ntp.org
-ntpclient.1.status=disabled
-ntpclient.status=disabled
+ntpclient.1.server=0.pool.ntp.org
+ntpclient.1.status=enabled
+ntpclient.2.server=1.pool.ntp.org
+ntpclient.2.status=enabled
+ntpclient.3.server=2.pool.ntp.org
+ntpclient.3.status=enabled
+ntpclient.4.server=3.pool.ntp.org
+ntpclient.4.status=enabled
+ntpclient.status=enabled
 pwdog.delay=300
 pwdog.period=300
 pwdog.retry=3
@@ -95,13 +101,13 @@
 radio.1.scan_list.status=disabled
 radio.1.scanbw.status=disabled
 radio.1.status=enabled
-radio.1.subsystemid=0xe7f5
+radio.1.subsystemid=59381
 radio.1.txpower=24
 radio.countrycode=380
 radio.status=enabled
-resolv.host.1.name=PowerBeam 5AC 400
+resolv.host.1.name=airos
 resolv.host.1.status=enabled
-resolv.nameserver.status=enabled
+resolv.status=enabled
 route.1.devname=br0
 route.1.gateway=192.168.1.1
 route.1.ip=0.0.0.0
@@ -109,7 +115,13 @@
 route.1.status=enabled
 route.status=enabled
 snmp.community=public
-snmp.status=disabled
+snmp.contact=edoput@lilik.it
+snmp.location=
+snmp.status=enabled
 sshd.auth.passwd=enabled
 sshd.port=22
 sshd.status=enabled
@@ -117,24 +129,20 @@
 syslog.remote.status=disabled
 syslog.status=enabled
 system.airosx.prov.status=enabled
-system.cfg.version=65547
+system.cfg.version=0
 system.date.status=disabled
 system.external.reset=enabled
 system.timezone=GMT
 telnetd.port=23
 telnetd.status=disabled
-tshaper.status=disabled
-unms.status=disabled
 update.check.status=enabled
 users.1.name=ubnt
 users.1.password=$1$yxUQm7Il$ViUqgyf1SvPmT4a.nTRdH/
 users.1.status=enabled
-users.2.gid=100
-users.2.shell=/bin/false
-users.2.status=disabled
-users.2.uid=100
 users.status=enabled
+vlan.status=enabled
 wireless.1.addmtikie=enabled
+wireless.1.ap=00:11:22:33:44:55
 wireless.1.devname=ath0
 wireless.1.hide_ssid=disabled
 wireless.1.l2_isolation=disabled
@@ -144,10 +152,10 @@
 wireless.1.rate.auto=enabled
 wireless.1.rate.mcs=-1
 wireless.1.security.type=none
-wireless.1.signal_led1=94
-wireless.1.signal_led2=80
-wireless.1.signal_led3=73
-wireless.1.signal_led4=65
+wireless.1.signal_led1=75
+wireless.1.signal_led2=50
+wireless.1.signal_led3=25
+wireless.1.signal_led4=15
 wireless.1.signal_led_status=enabled
 wireless.1.ssid=ubnt
 wireless.1.status=enabled
Fast system script build Success.
Fast users script build Success.
Fast resolv script build Success.
do_radio_fast_script: rname wifi0
Unsuported change in radio.1.subsystemid for fast update
Fast radio script build failed
Fixup Startup_list ...Done.
Connection to 192.168.1.20 closed by remote host.
Connection to 192.168.1.20 closed

[edoput]

Much of the work is now done and there are the 20% little things that can be configured but also have working defaults. I've tried updating the list of issues to see what needs to be done but as always the only true measure of completion and compatibility is uploading something to the antenna to see if it works.

[nemesifier]

Great work Edoardo. I suggest to get some rest and then work on making travis-ci and coveralls happy as well as refine your code if you feel some areas can be improved.