Security and Privacy considerations for ALS

[lknik]

I would suggest updating security/privacy considerations with the following:

Ambient Light Sensor API provides information about lighting conditions near the device environment. There are several potential privacy risks related with exposing this information on the web.

• Information leaks about the user’s surrounding and work habits.
• Profiling. Readout from Ambient Light Sensor can potentially induce information leaks about the user’s interests, web use and work habits, as well as the users’ surrounding. This information might be used to enhance the user profiling and behavioral analysis.
• Cross-device linking and tracking. Access to sufficiently precise readouts of Ambient Light sensors potentially enhance cross-device linking techniques. Such situation may arise if two different devices access web sites including same third-party scripts that compare lighting levels over time.
• Cross-device communication. Verbose readout of Ambient Light Sensor could be applied to receive messages emitted by other devices in nearby location. A simple messaging method could arise by multiple devices flashing their screens or camera LEDs and reading out responses with Ambient Light Sensors

The user agent SHOULD expose minimized lighting levels.

A verbose, high-resolution readout from the API SHOULD be subject to permissions.

The user agent SHOULD inform the user about the current and past uses of the API.

[anssiko]

(For the background, see @lknik's blog post: https://blog.lukaszolejnik.com/privacy-of-ambient-light-sensors/)

[torgo]

What is the resolution of this issue? Asking for a friend.

[tobie]

@torgo I'm catching up on this after having been out of the loop for 6 months. Will get back to you by EOY at the latest.

[tobie]

You can laugh all you want. I'm serious about the schedule.

[tobie]

So turned out this got even further delayed than I expected. :)

Plan is to include something possibly along the lines of what @lknik pasted above.

[lknik]

We might include some additional information

Cross-origin information flows. Ambient Light Sensor may be used to indirectly access information displayed on a device. In some cases this may allow information flows across origins or contexts, leaking sensitive data.

Probably reconsider making the sensor subject to permissions?

[tobie]

Probably reconsider making the sensor subject to permissions?

All sensors based on the generic sensors API rely on the permission API by virtue of subclassing the Sensor interface, so I think we should be covered.

Relying on the permission API doesn't necessarily imply actually prompting the user for permission, though. And for good reasons. "Dialog box fatigue" is a real concern, and by no means the only solution to mitigate threats.

[alexshalamov]

@lknik We are investigating different mitigation strategies for AmbientLightSensor. Could you provide details of the setup you've used, so that everyone can reproduce the results? Would be nice to know:

• type of device
• lighting conditions
• distance to walls / ceiling or nearby objects that reflect / absorb ambient or screen light

One of the solutions is to round-up readings. We could use simple JS wrapper, to find 'safe rounding zone' for ALS readings.

class RoundedAmbientLightSensor {
  constructor(frequency, threshold) {
    this.onchange_ = null;
    this.illuminance_ = null;
    this.sensor_ = new AmbientLightSensor({ frequency });
    this.sensor_.onchange = () => {
      let old_illuminance = this.illuminance_;
      if (this.sensor_.illuminance != null)
        this.illuminance_  = Math.ceil(this.sensor_.illuminance / threshold) * threshold;
      if (this.onchange_ != null 
          && this.illuminance_ != null
          && old_illuminance != this.illuminance_) {
        this.onchange_();
      }
    };
  }

  set onchange(func) {
    this.onchange_ = func;
  }

  get onchange() {
    return this.onchange_;
  }

  get illuminance() {
    return this.illuminance_;
  }

  get timestamp() {
    return this.sensor_.timestamp;
  }

  start() {
    this.sensor_.start();
  }

  stop() {
    this.sensor_.stop();
  }
}

Example of how to use wrapper:

let x = new RoundedAmbientLightSensor(20, 20);
x.onchange = () => { console.log(x.illuminance); }
x.start();

Would you have time to re-run tests that you mentioned in your blog post? For example, with different thresholds (5, 10, 20, 50)?

[lknik]

Hi,

The PoC shows that basically questions around types of devices (smartphone/laptops) / lighting condition (dim!) / distance to objects (walls, mirrors, face...) might miss the point. What matters is it's pretty difficult to actually fine-tune the settings. Any measurable difference may turn out to be useful. The question is: what precision is warranted by the use cases?

Further quantization is the way to go, the question is - is the MQ4 suitable? AFAIK Tobie is working on some (1) permission model for testing (2) idea of a high-level permission-less API?

[alexshalamov]

I made few measurements using several devices under different environmental conditions, results can be found in this table.

Normal conditions

• Auto brightness on, device sets level at 50%
• Light reflected from normal surfaces

Speculative conditions

• Auto brightness off, brightness at max
• Light reflected from highly reflective surfaces (mirror, white diffusers)

Under normal conditions, differences between ALS readings reported for ‘white’ and ‘black’ levels are in a range of 0.05-14.03 lx. Under speculative conditions, e.g., close to a mirror or white diffuse material, max delta was 127.07 lx. When two devices are attached screen-to-screen and changing brightness levels, max delta was 478 lx.

Based on the data, JS wrapper with threshold of 50 lx was used to verify that POC referred in the article is not able to provide reliable results.

Drawbacks for the 50 lx threshold is that sensitivity in 0-50 lx range would be lost. After 50 lx, illuminance values for EV or light levels describing environmental conditions (light levels, EV, daylight) are increasing logarithmically, therefore, relative precision would be increasing together with illuminance values (e.g., not important if lx value is 20.000 or 20.050).

[alexshalamov]

@lknik Do you have any comments regarding my proposal?
Would it be a good start for experimenting with ALS readings rounding?

[lknik]

Hey @alexshalamov - decreasing entropy sounds like a good way forward. But indeed the lower values in lx are the most "informative", as you say. I understand that the difference between 50 and 150 is significant.
In light of this, why not expose just a few lighting levels (permission-less) and offer the full precision behind permissions after all? Having a step=50 is fine, but:

• though it "defeats" that particular PoC, the thing is still somewhat there
• it makes the API much less useful than it is apparently supposed to be

Sounds like a tradeoff is a challenge here (assuming we only have the current permission models, as implemented in browsers).

[anssiko]

My understanding is, the light-level media query as specified cannot be implemented in an interoperable manner, and as such, is not a solution to pursue at this time. Luckily, it seems that after careful research we've identified a solution that decreases entropy without compromising known high value use cases and allows us to mitigate known attacks.

I'll let @alexshalamov expand on the proposed solution with details, since that was hand-wavy on my part.

When we have these details discussed and agreed upon here, we are in a better position to update the security and privacy consideration section with concrete guidance to implementers -- and can finally close this issue that's been open for a while.

[tobie]

My understanding is, the light-level media query as specified cannot be implemented in an interoperable manner, and as such, is not a solution to pursue at this time.

Source? The CSS WG does not seem to have an open issue about this, despite having included light-level in Mediaqueries Level 5 which it published very recently.

As an aside, the note in the light-level section (copied below) is well worth a read (and might help either find alignment if some of the use cases match or, on the contrary, clarify that they're orthogonal concerns):

Even though it is expected that User Agents will adjust the value of this media feature based on ambient light sensors, this specification intentionally refrains from defining the three levels in terms of a measurement in lux, for several reasons:

• Devices equipped with a light sensor usually adjust the brightness of the screen automatically. Depending on the level of adjustment, the thresholds for needing a low contrast or hight contrast content may vary.
• Different screen technologies wash out at very different ambient light levels; e-ink displays remain readable in bright daylight, while liquid crystal displays do not.
• Many embedded light sensors are inaccurately calibrated, making it difficult to establish useful thresholds valid across devices.

[alexshalamov]

Source?

Maybe it was me 😄. I did some research after I checked light-level related discussions.

As an aside, the note in the light-level section (copied below) is well worth a read (and might help either find alignment if some of the use cases match or, on the contrary, clarify that they're orthogonal concerns):

Note basically says, light-level is not mapped to real world lux values and should not be used (blindly) to e.g., adjust contrast of page because:

• Different schemes might be required for E-Ink and LCD displays
• Brightness of the screen is unknown
• Sensors might be inaccurate
• FOV of the sensor might be different from viewing angle of the screen

I still think it would be tricky to implement interoperable light-level feature that would work reliably for intended use-cases, e.g., to adjust contrast of the web page.

@lknik Thanks for spending your time to check the proposal.

But indeed the lower values in lx are the most "informative", as you say.

Sorry, maybe I didn't explain properly. Values in a range of [0-50] might be useful to destinguish between dark environmental conditions, for example 'Full moon' vs 'Overcast sunset'. I'm not sure whether it would be big problem if we would loose sensitivity in that range. So, those values are not that informative.

I hope after origin trials we would get more information whether 'high precision' mode is required for ALS, then we can adjust accordingly, e.g., define permission policies based on precision of required data.

[tobie]

Note basically says, light-level is not mapped to real world lux values and should not be used (blindly) to e.g., adjust contrast of page

That's not what it says. It says it doesn't specify lux values because that leaves room for implementors to set those according to the constraint listed right below it.

I still think it would be tricky to implement interoperable light-level feature that would work reliably for intended use-cases, e.g., to adjust contrast of the web page.

That's totally possible, but then the CSS WG is going to have precisely the same issues, so maybe you all should talk.

[lknik]

When we have these details discussed and agreed upon here, we are in a better position to update the security and privacy consideration section with concrete guidance to implementers -- and can finally close this issue that's been open for a while.

I guess the considerations would still stand. Also, what you mean by "known attacks" might not be that future-proof. And I reiterate that any implementation of ALS might, to some extent, be used to profiling.

Sorry, maybe I didn't explain properly. Values in a range of [0-50] might be useful to destinguish between dark environmental conditions, for example 'Full moon' vs 'Overcast sunset'. I'm not sure whether it would be big problem if we would loose sensitivity in that range. So, those values are not that informative.

What I thought is that you want to expose the following values: 0, 50, 100, 150, ... - so round the reading to the closest n*50. I guess for n<20 you'd miss out on information, as for larger n's the differences aren't that large (with respect to the outside sorrounding).

So my proposal:

• expose CSS MQ values
• if exposing more precision, ask for permission. For the use cases, e.g. calibrating a camera, permission should not be an issue... Or why not bundle a camera permission with ALS too and name it somehow (as opposed to Motion Sensor).

[anssiko]

@lknik, your proposal has two issues:

• light-level CSS MQ cannot be implemented in an interopetable manner (see above comments)
• browser vendors are not interested in adding more prompts, see prompt fatigue

I think the CSS MQ use cases that involve stylesheet swithing are better handled manually in a similar fashion as many web sites do already for font size adjustment.

[tobie]

light-level CSS MQ cannot be implemented in an interopetable manner (see above comments)

Can you actually back this claim up with technical arguments? And if so, bring it up with the CSS WG who doesn't seem to mention this concern anywhere?

[anssiko]

@alexshalamov to take an action to bring this up with CSS WG.

[tobie]

Please make sure to link to this issue when you do.

It also wouldn't hurt to hear here the technical arguments as to why enum-based values wouldn't be implementable interoperabily but discrete lux values would.

[lknik]

Could we include:

Readout of Ambient Light Sensor may potentially enable cross-origin communication or information leaks.

Or something along the lines. Feel free to modify. I'm referring to a PoC of stealing web browsing history.

[lknik]

OK @anssiko - made a separate issue for that one #37

[Joe-Palmer]

I hope this is the best place to post this (I was referred from this Chromium issue). I see there has not been much activity here recently so please let me know if there is a better place to make these comments.

I work for iProov which allows users of banks and governments services to assert and confirm their identity using secure face-based authentication. We specialise in Genuine Presence Assurance so focus on ensuring that the user is a real, live, physically present person. We do this by using the screen of the device to illuminate their face with a unique sequence of colours whilst streaming a short video back to our servers. We analysis the way the light reflects off their face in real-time to ensure we are looking at a real, 3D, skin-covered human. We also check that the unique sequence of colours are correct to ensure the user is live and we are not seeing a previous authentication attempt being replayed. And we also do face verification so ensure that the correct person is presenting themselves.

Our current live deployments use iOS and Android SDKs embedded into our customers apps which gives us great control over the device's hardware. However, we are getting increasing requests to provide a browser solution which is a much more constrained environment. So, we have have developed a WebRTC version which is working well and nearly ready for live deployment but there is once specific issue we would like to address which is why I am posting here.

Our ability to determine the Genuine Presence of user is dependent on the ability to detect the light from the screen interacting with the user's face in the video. In direct sunlight, it is too bright to detect the reflections so we do not acquire enough evidence to correctly pass real users. In our native SDKs, we have full access to the device's hardware so we can make an accurate assessment of the ambient brightness and advise the user to find somewhere less bright to prevent them from starting in conditions where they are very likely to fail.

In a browser environment, we currently have no good way to make an assessment of the ambient brightness so cannot advice the user of their current lighting conditions which increases the failure rate and frustration for the user. So, having access to the data from the Ambient Light Sensor would be a great way solve this problem for our users!

I have read this thread and understand the potential privacy concerns. The current brightness limit for our solution to work is somewhere between 1500 and 2000 lux (depending on a few things like screen size and brightness) so only having accuracy to the closest 50 lux would not be a big issue for us.

We are very encouraged by the potential of using this new API. What are the blockers for getting the Ambient Light Sensor data made available for everyone? I hope you find our use case compelling and maybe it will help justify the release of this feature. I would be very happy to provide more details or answer any specific questions if that would be helpful. If there is anything that we could do to help with the delivery of this feature, we would be very happy to contribute.

Thank you for your consideration.

[anssiko]

What are the blockers for getting the Ambient Light Sensor data made available for everyone?

It is at each browser's discretion when and whether to ship by default. The spec itself is ready for implementation: it has passed wide review and reached feature completeness (aka Candidate Rec), although some new features are being proposed (below).

To help browsers make a shipping decision at this point, the community can help:

• demonstrate there are valuable use cases and demand;
• demonstrate security and privacy considerations are addressed.

You just helped with the former. For the latter, the spec recommends implementations to limit maximum sampling frequency and reduce accuracy of sensor readings. As of now, frequency cap and reading accuracy are implementation details given different implementers might have different preferences. For example, Chromium project's earlier thinking was to cap ALS at 10 Hz with a 4-bit resolution limit -- this may have changed since.

Additional mitigation that has been proposed recently is to add an API for requesting permission to the generic sensor framework also ALS would inherit.

@Joe-Palmer, thanks for the use case. I hope this clarifies the current status.

@riju will be looking at this issues from the Chromium implementation perspective. You can follow https://crbug.com/606766 for implementation updates.