Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support scripting complex sets of mock sensor readings #371

Open
Honry opened this issue Aug 23, 2018 · 4 comments
Open

Support scripting complex sets of mock sensor readings #371

Honry opened this issue Aug 23, 2018 · 4 comments

Comments

@Honry
Copy link
Contributor

Honry commented Aug 23, 2018

This thread is separated from #369 (Introduce WebDriver Extension API) to discuss about supporting multiple HTTP requests for the use case of simulating a series of updating mock sensor readings. Thanks @jugglinmike for proposing a great idea!

Posted @jugglinmike's comments:
(#369 (review))

Readings Speaking as a web developer, think this API may not be powerful enough to thoroughly test my applications. Unless I am mistaken, the API requires a POST request for each new sensor reading I want to simulate. In many cases, I may not be able (or willing) to issue HTTP requests at an acceptable rate, especially if the remote end is running over the Internet (maybe via a third party like Sauce Labs).

Instead of sending individual readings in a series, what if I could specify a JavaScript function capable of creating those readings? The implementation could call this at the device's sampling frequency, and if it provided a timestamp, I would be able to derive an expected value in a stateless way. Alternatively, we could accept an ES2015 generator function. A generator could yield readings and receive timestamps from the result (or I could maintain the timestamp myself using the performance timing API). In either case, we could use WebDriver's "execute script" command as a guide.

(#369 (comment))

An application developer might write some code to respond to a rapid increase
in brightness. To test their application logic using this new WebDriver API,
they'd need to simulate the increase via a series of POST requests:

POST /session/c0ffee/sensor/ambient-light
{"illuminance": 0}
POST /session/c0ffee/sensor/ambient-light
{"illuminance": 0.2}
POST /session/c0ffee/sensor/ambient-light
{"illuminance": 0.4}
POST /session/c0ffee/sensor/ambient-light
{"illuminance": 0.6}
POST /session/c0ffee/sensor/ambient-light
{"illuminance": 0.8}
POST /session/c0ffee/sensor/ambient-light
{"illuminance": 1}

However, they won't have any control over when those values get applied.
Depending on network effects, this could be not very rapid at all (and not very
smooth as a result). It will also get more cumbersome for more expressive
sensors (e.g. gyroscope) and for more complex readings (e.g. a "twist"
movement). And since all WebDriver commands execute in series, developers won't
be able to interact with the browser while this is happening.

If instead, the automation API allowed developers to specify a function, they
could express an entire series of sensor readings that was arbitrarily complex
and continuous, and they could do it with a single POST request.

Here's how it might look with a "classic" function body that received the time
offset as its first argument:

POST /session/c0ffee/sensor/ambient-light
{
  "body": "const t = arguments[0]; return {illuminance: Math.min(t / 1000, 1)};"
}

And here's a fancier version using an ES2015 generator function that tracks the
time with its own internal state:

POST /session/c0ffee/sensor/ambient-light
{
  "body": "const start = performance.now(); while (true) { const t = performance.now() - start; yield {illuminance: Math.min(t / 1000, 1)}; }"
}

The latter is more verbose for this example, but the ability to maintain state
could make it much easier to script complex sets of readings.

(#369 (comment))

How about keep single set of readings as current solution, and extend this
section by using WebDriver's "execute script" command to script complex sets
of readings?

That could be a nice simplification, but I don't think the current text
supports it. There doesn't seem to be an JavaScript API to control this, so
the code that developers provide via "Execute Script" doesn't have an interface
to specify mock values.

We could define such a JavaScript API, but that might not be necessary.
Developers may stub out the JavaScript constructor of the device they wish to
mock. Their stub constructor could generate instances which emitted events in
whatever sequence they liked. This just requires injecting code prior to the
evaluation of their application. For some configurations, that could be a
little cumbersome but not impossible. I've filed an issue against WebDriver to
discuss making this easier
.

If that is a tenable solution, then maybe web developers aren't an audience for
this feature, after all! What do you think?

@foolip
Copy link
Member

foolip commented Aug 23, 2018

This is a similar problem as with pointer device input, which in https://w3c.github.io/webdriver/#actions is solved by having a notion of ticks and pausing to send a whole series of updates in one command.

Something similar might work for sensor readings, given how similar the data flow is.

@foolip
Copy link
Member

foolip commented Aug 23, 2018

A potential problem with having JS generate sensor readings is where that script will execute. To not make assumptions about where the sensor readings are being inserted, it may be necessary to isolate that script similar to how worklets are defined: https://drafts.css-houdini.org/worklets/#worklet

https://webaudio.github.io/web-audio-api/#audioworklet is a bit similar to what might be needed, although that's a web-exposed API.

@jugglinmike
Copy link
Contributor

@foolip could you comment on the need to address this in WebDriver? I'm starting to think that the "browser implementer" use case is addressed by @Honry's patch and that the "web app developer" use case is addressed by pure JavaScript mocks. Are there any situations where one of those two approaches will be insufficient?

@foolip
Copy link
Member

foolip commented Aug 27, 2018

Since Sensor doesn't have a constructor it won't be possible to perfectly mock the API for a web developer, but I believe it will be possible to mock out sensors well enough to run any tests that are verifying the behavior of a web app as opposed to the browser itself.

Whenever possible I think it'd be good to serve the testing needs of web developers with WebDriver as well, but this is a case where doing so would add significant complexity. So, my inclination would be to not tackle that unless requests from real web developers attempting to use the existing proposed API come in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants