Mesh Powersave Implementation Notes
- Abstract
- Where is it specified
- Files
- General info
- Local PS mode management
- Indication
- Tracking
- Frame buffering
- Additional IEs
- Frame release
- Doze/wakeup scheduling
- Driver extensions
- Userspace access
- Test cases
- Current limitations and bugs
Mesh powersave is designed to allow a longer uptime for battery-powered mesh devices. Therefore, in times of no activity the radio receiver is switched off to conserve power. When data passes a mesh link in powersave, the data is transmitted in bursts which allows to power down the receiver in the times of no activity.
1Mbps UDP data passing a link in active mode
1Mbps UDP data passing a link in light sleep mode
Powersave in a mesh BSS is specified in section 13.14 of the IEEE standard 802.11-2012.
Changes have been made to nl80211, cfg80211 and mac80211. net/mac80211/mesh_ps.c is the central hub for all mesh PS code if you intend to review and/or extend it. Hooks are placed in multiple places in transmit and receive paths, routing and peering code.
The last state is reflected in the ft-powersave integration branch in the open80211s repo.
The mesh powersave implementation consists of two separable parts. One part are the general powersave basics like management, indication, tracking, frame buffering and release, which allow coexistence in a network with powersaving nodes. They are also used as a basis for the second part of the implementation; the powersave doze and wakeup scheduling and its hooks into the driver.
The design goal was to have all the functionality in mac80211 for the sake of maintainability and easy adaption of drivers. This comes with the cost of imperfect scheduling timing due to stack delay (beacon RX in HW until it arrives in mac80211) and use of jiffies as time reference instead of TSF timestamps. This is compensated with increased safety margins.
The interface mac80211<->driver is a minimal version of what Johannes proposed here (adapted to the current requirements).
The basic primitive for mesh powersave is the enum nl80211_mesh_power_mode, which holds the possible modes unknown, active, light sleep, deep sleep. unknown is used only as the initial value when the own power mode is not set because no peering is established or when a peer's power mode is not known.
Each node maintains a peer-specific power mode for established peer links and a non-peer power mode for all other non-peer neighbors.
The peer-specific power mode is set up in the local_pm field of the struct sta_info on peer link establishment. It is set to the user-defined default power mode that is stored as part of the struct mesh_config. To avoid frame losses when switching to a lower power state after peering (and for possible use in a per-link dynamic powersave), the corresponding function can be called with a delay parameter that sets a per-STA timer accordingly. The local_pm is updated when the PS status if the peering status of a neighbor changes.
The non-peer power mode is set depending on the peer-specific power modes in the function ieee80211_mps_local_status_update: if there is at least one link in light or deep sleep mode, the non-peer power mode will be deep sleep mode. To allow successful peering, 802.11-2012 defines to temporarily set the non-peer power mode to active mode during peering.
Both, the peer-specific power mode and the non-peer power mode are indicated in different frame types. The peer-specific power mode is announced in all individually-addressed QoS data frames. One special case is that after a power mode change the STAs indicate the new power mode with an individually-addressed QoS Null frame to the respective peer.
The non-peer power mode is announced in management frames and group-addressed frames. In group addressed Mesh QoS Data frames the Mesh Power Save Level field in the QoS Control field indicates whether the local STA has any deep sleep peers. In beacon (and probe response) frames STAs indicate whether they have any deep sleep peers.
For performance reasons, calls to the function setting the frame flags are placed in HWMP routing routines, as there the STA pointer is already available.
For each neighbor the non-peer power mode is stored and updated on each frame receipt. Individually-addressed QoS Data frames from peers contain their peer-specific power mode towards the receiver STA. The recive path handler ieee80211_rx_h_sta_process is modified to extract information on both power modes on frame receipt of the respective frame type.
We have to PS buffer frames towards:
- peers in light/deep sleep towards us
- non-peers with non-peer deep sleep power mode
- in case the peers peer-specific power mode is unknown use its non-peer power mode
The trigger for buffering frames are the STA flag WLAN_STA_PS for individually-addressed frames towards that STA, and the counter num_sta_ps for all group-addressed frames; if there is at least one PS neighbor around, all group-addressed frames are buffered until the next DTIM beacon.
In case a neighbor changes its power mode or peering status, we update the setting of the WLAN_STA_PS flag. This flag triggers the frame buffering routines for individually-addressed frames in the tx path used for client mode. Each sta_info structure contains a frame buffer list called ps_tx_buf, which is used to buffer individually-addressed frames towards PS neighbors. Since drivers are not prepared for mesh PS routines, we do not inform the driver about the neighbor STA being in PS mode towards us (as sta_ps_start would do).
Some of the structures needed for PS previously belonged to the AP-specific structure ieee80211_if_ap. Thus, the ps_data structure has been extracted and added to ieee80211_if_mesh as well. This structure contains the TIM map, a counter how of PS neighbors and the group-addressed frame buffer. Functions using this structure are modified for mesh mode.
For frame release in mesh PS mode two additional IEs are used:
- Awake Window IE
- TIM IE
The Awake Window IE announces the awake window value that determines how long the STA stays awake after sending its beacon.
The TIM IE, similarly as in AP mode, announces buffered individually-addressed frames for a peer STA in a bitmap field. In mesh mode the LLID/PLID are used (mod by 2007, which is the max capacity of the TIM bitmap).
Group-addressed frames are released after the next DTIM beacon, similarly to AP mode. How buffered individually-addressed frames are released depends on whether the STA is a peer or not. For non-peers IEEE802.11-2012 does not define anything specifically. We chose to release a single frame per beacon interval (these should only be peering/authentication frames, that are supposed to wake up the peer). Frames towards a peer STA are released in mesh Peer Service Periods (MPSPs).
The start and format of a MPSP depends on multiple conditions:
- receipt of a neighbor beacon with Awake Window IE
- receipt of a trigger frame
- buffered data indicated in received beacon
- buffered data locally
MPSPs are frame release periods between two peer STAs. Both STAs can be owner and/or recipient. The owner is transmitting while a recipient is only receiving frames.
MPSPs start and end with a MPSP trigger frame. The setting of RSPI and EOSP frame flags determine if a MPSP starts or ends and which STA is the owner, recipient or both during the MPSP. The indication of buffered frames at the peer and the status of the local buffers determine which type of MPSP is started when a peer beacon is received. The buffered frames are transmitted only after the MPSP trigger frame has been acknowledged. To end a MPSP the owner sets the EOSP flag in the last QoS Data frame. In case the last frame is not a QoS data frame, we append a QoS Null frame after the last frame.
This example shows a MPSP where both STA are owner and recipient (both are transmitting data).
We count the number of currently active MPSPs the local STA is owner and/or recipient in. This will be used to disallow doze state during ongoing MPSPs.
At first the driver is expected to register the mps_ops to mac80211 if it is capable of mesh PS. If not the device will be able to coexist in a mesh with PS nodes and may even be configured for PS mode but will not actually conserve power itself.
ieee80211_mps_local_status_update updates the local Ps status of the individual links and calls ieee80211_mps_hw_conf If the mps_ops are registered and the current PS link state allows to put the radio to doze state, it will configure the driver with the IEEE80211_CONF_PS flag similar to client mode and set the mps_enabled flag of the struct ieee80211_local In difference to client mode PS, followingly mac80211 will control doze/wakeup explicitly by mps_ops calls to the driver.
After mps_enabled is set, ieee80211_mps_schedule_update becomes active and stores timing information of received peer beacons in the sta_info structure. This information is used to set per-STA timers for the next TBTT of the peer. A parameter margin is used to compensate delays due to contention and hardware wakeup time and also beacon losses. The margin is the time to wakeup before the calculated TBTT and to stay up after this time has passed. After that the beacon is considered missed and the next TBTT is projected, timers are set accordingly and the hardware is put to doze state until then. The margin is increased with the beacon miss count. To determine the peer's TBTT the T_offset parameter calculated in mesh_sync is used. After getting our own TSF T_offset allows calculating the peer's TSF. Each time the peer's TSF mod its beacon interval is zero, it sends its beacon. We calculate the duration until that event and set an hrtimer accordingly. To compensate for delayed beacons and other distortions, we use a margin parameter.
These plot depict interval lengths between consecutive beacons. The plots have been recorded in active mode, without any powersaving involved. The upper image shows that regularly beacons are missed and the resulting interval between consecutive beacons is two or three times the regular beacon interval. The lower image shows the beacon jitter in more detail to estimate the usual spread of beacon RX times. It should be noted, that beacons may only be delayed but are never sent earlier. The earlier beacons in the jitter graph actually refer to a previous beacon that was sent delayed.
This image shows how we deal with missing beacons and jitter. The margin parameter is chosen to compensate the usual jitter. Since beacons are missed regularly we just go to sleep after missing one. The margin is increased with each miss to compensate possible drift and reset upon successful receipt.
After mps_enabled is set also the mesh Awake Window routines are triggered. The beginning is marked by the ieee80211_beacon_get_tim call from the driver after the SWBA interrupt. ieee80211_mps_awake_window_start sets a timer, which on expiry will call ieee80211_mps_awake_window_end which issues a hw_doze call. There is no hw_wakeup call during the awake window start, as all tested hardware wakes up automatically on SWBA.
This image shows the resulting behaviour of a mesh STA with a single peer which it is in light sleep towards. The STA stays awake for the Awake Window duration after sending its beacon and wakes up for its peer's beacon.
mps_ops currently contains two ops: hw_doze and hw_wakeup. While hw_wakeup contains no additional logic, hw_doze performs a check on the local list of sta_info if any PS status flag permits doze state. That may be a scheduled beacon receipt or broadcast after DTIM (CAB). If no obstacles are found the status variables in ifmsh are updated and the doze call is issued.
##Driver extensions The design goal of the current implementation was to concentrate most functions in mac80211. Thus, the required changes to the driver are minimal.
- the driver needs to register its mps_ops on mesh interface_add.
- the driver has to implement the required calls mps_doze and mps_wakeup. These should override the driver's PS status variables that are already implemented in mac80211 (beacon and CAB). In case the driver sets any hardware wakeup scheduling for the AP, also this should be overridden.
- if the driver has a sleep/wakeup loop, it has to add a state variable that stores the last issued state from mac80211.
In case of behavioural differences to the current hardware (e.g. no wakeup on SWBA) it may be required to add some HW capability parameters to the mps_init callback.
Mesh power modes can be controlled with nl80211. Following are some examples using iw.
We bring up a mesh point ZOTAC10 with default power mode set to light sleep.
root@zotac--0:~# iw mesh0 mesh join mymesh mesh_power_mode light
It peers up successfully with ZOTAC12, which has active as default mesh power mode. After peering ZOTAC10 is in light sleep mode towards ZOTAC12, while ZOTAC12 is in active mode towards ZOTAC10.
root@zotac--0:~# iw mesh0 station dump
Station 00:03:7f:10:4e:0c (ZOTAC12) (on mesh0)
inactive time: 4 ms
rx bytes: 836333
rx packets: 21553
tx bytes: 32581
tx packets: 543
tx retries: 345
tx failed: 15
signal: -23 dBm
signal avg: -19 dBm
Toffset: -2479495726 us
tx bitrate: 54.0 MBit/s
mesh llid: 14694
mesh plid: 52785
mesh plink: ESTAB
------> mesh local PS mode: LIGHT SLEEP <------
------> mesh peer PS mode: ACTIVE <------
------> mesh non-peer PS mode: ACTIVE <------
authorized: yes
authenticated: yes
preamble: long
WMM/WME: yes
MFP: no
TDLS peer: no
ZOTAC12 sees it this way.
root@zotac--2:~# iw mesh0 station dump
Station 00:03:7f:10:4e:09 (ZOTAC10) (on mesh0)
inactive time: 179 ms
rx bytes: 764243
rx packets: 18436
tx bytes: 65242
tx packets: 1276
tx retries: 183
tx failed: 0
signal: -32 dBm
signal avg: -31 dBm
Toffset: 2479494343 us
tx bitrate: 54.0 MBit/s
mesh llid: 52785
mesh plid: 14694
mesh plink: ESTAB
------> mesh local PS mode: ACTIVE <------
------> mesh peer PS mode: LIGHT SLEEP <------
------> mesh non-peer PS mode: DEEP SLEEP <------
authorized: yes
authenticated: yes
preamble: long
WMM/WME: yes
MFP: no
TDLS peer: no
Now we set ZOTAC12 to be in deep sleep mode towards ZOTAC10.
root@zotac--2:~# iw mesh0 station set 00:03:7f:10:4e:09 mesh_power_mode deep
Now both nodes see the scenario differently.
root@zotac--0:~# iw mesh0 station dump
Station 00:03:7f:10:4e:0c (ZOTAC12) (on mesh0)
inactive time: 1013 ms
rx bytes: 875773
rx packets: 22535
tx bytes: 32581
tx packets: 543
tx retries: 345
tx failed: 15
signal: -30 dBm
signal avg: -28 dBm
Toffset: -2479495733 us
tx bitrate: 54.0 MBit/s
mesh llid: 14694
mesh plid: 52785
mesh plink: ESTAB
------> mesh local PS mode: LIGHT SLEEP <------
------> mesh peer PS mode: DEEP SLEEP <------
------> mesh non-peer PS mode: DEEP SLEEP <------
authorized: yes
authenticated: yes
preamble: long
WMM/WME: yes
MFP: no
TDLS peer: no
root@zotac--2:~# iw mesh0 station dump
Station 00:03:7f:10:4e:09 (ZOTAC10) (on mesh0)
inactive time: 1881 ms
rx bytes: 780203
rx packets: 18816
tx bytes: 65306
tx packets: 1278
tx retries: 184
tx failed: 0
signal: -32 dBm
signal avg: -32 dBm
Toffset: 2479494312 us
tx bitrate: 54.0 MBit/s
mesh llid: 52785
mesh plid: 14694
mesh plink: ESTAB
------> mesh local PS mode: DEEP SLEEP <------
------> mesh peer PS mode: LIGHT SLEEP <------
------> mesh non-peer PS mode: DEEP SLEEP <------
authorized: yes
authenticated: yes
preamble: long
WMM/WME: yes
MFP: no
TDLS peer: no
##Test cases For testing the mesh powersave code, three tests are recommended:
- peering with nodes default-configured for PS mode
- connectivity is not broken (use relaxed beacon interval and DTIM period of 400:1 to avoid problems due to HWMP timeouts)
- observe power consumption. Power drain should be considerably reduced in connected but idle state. Increasing the number of peers or the length of the awake window increases power consumption.
Following are some consumption characteristics of different USB devices:
| device | idle (down) | active (PS) | active (no PS) |
|---|---|---|---|
| TL-WN821 (ath9k_htc) | 37mA | 150mA | 232mA |
| TL-WN721 (ath9k_htc) | 37mA | 56mA | 118mA |
| SMCWUSB-N2 (carl9170) | 43mA | 62mA | 197mA |
| Fritz!WLAN USB (ath9k_htc) | 29mA | 36mA | 75mA |
| TL-MR3020 (ath9k) | 106mA | 109mA | 138mA |
| WNDR3800 (ath9k, only 2.4Ghz) | 334mA | 336mA | 388mA |
##Current limitations and bugs
-
In HT channels MPSPs behave imperfectly and show large delay spikes and frame losses. This seems to be a problem with the driver/firmware/HW trying to perform frame aggregation.
-
Authenticated mesh peering is currently not working when either node is configured to be in power save mode before the peering starts. The power mode can be changed after peer link establishment, though.
-
ath9k_htc shows bad behavior when transmitting when being in powersave. Many frames get lost and delay is irregular. Also the device gets stuck in a high power state and needs to be shut down completely to resolve this (see bug #16).
-
For carl9170 the device crashes after some time ("lost command responses/traps" -> device restart). Also it shows bad behavior when transmitting when being in powersave.
-
mesh_path_refresh_time should be equal to the DTIM interval, since PREQs won't be released until DTIM.