[interest invokers] Touch inputs #1052
Comments
|
So far I think we've been talking about a long press as a touch input interest mechanism. Pros:
Potential issues to overcome:
|
Potential solutions are to encourage Devs to add user-select: none to interesttarget buttons (perhaps apply automatically but less keen on this) For elements that already trigger a context menu on long press we can add this trigger action to that context menu just like copy link is already an option.
Hopefully those users will be able to use TalkBack and that will be able to trigger this. Needs confirming.
Unfortunate but not impossible.
This will just have to be accepted, but perhaps we should add this to the platform too. |
lukewarlow
added
agenda+
|
It might be controversial to say this, but I think it might be necessary to leave some parts of the concept of "interest" to user agents to define. For example, in a Vision Pro like platform, with eye-tracking and a distinctly non-keyboard, non-mouse user interface, it likely needs to be up to the platform to decide when a user is "interested" in an element. I don't think we should (or need to) write down exactly how this works. In contrast, keyboard users and mouse users have fairly standard ways to indicate "interest", namely focusing or hovering the element. These, I feel, should be standardized so they behave in the same way across browsers and platforms. Somewhere in the middle is touch. Perhaps touch is standardized enough that we can specify it, but perhaps not? What's important, to me at least, is that the standard says, normatively, that the user must have a way to "indicate interest" in an element, no matter their input modality. Stepping back off the soap-box, per your questions about touch:
In chromium at least, I could see just adding an item to the existing context menu.
I agree this might be an issue, but I would see it as an implementation issue maybe? It must be possible for the user to do both, somehow.
I might need more context on this one. But again, the UA should provide an accessible way to indicate interest.
Agree completely.
Agree. But it's not a requirement that new features be polyfillable. Definitely nice to have though. |
|
The Open UI Community Group just discussed The full IRC log of that discussion<jarhar> Luke: we've discussed before the idea of interest invokers is that theyre somewhat agnostic to the input method. if its mouse you have hover, keyboard is focus, eye tracking you get whatever<jarhar> Luke: one of the input modalities is touch screens, we've discussed various ideas how that should work <jarhar> Luke: this is really to get the ball rolling on approaches to it, what we envisage seeing <jarhar> Luke: is this even something that we want to do? it might be that on touch screens that its up to the ua to decide how it works <jarhar> Luke: the two proposed resolutions i want are - do we think that intreest invokers should always be able to be used? <jarhar> Luke: if youre on a touch screen, does the ua have to provide some mechanism? we could say its optional, and up to the ua <jarhar> Luke: for hover invokers we could require it <gregwhitworth> q+ <jarhar> Luke: or, as mason pointed out, the ua must provide this in some mechanism depending on the input <jarhar> Luke: or we could spec some kind of how this should actually work <jarhar> Luke: personally i would like to require it for mouse keyboard and touch <jarhar> Luke: i think touch might be difficult to spec because it depends on platform specifics <jarhar> Luke: long press is a thing on android to show tooltips, if you have an icon button. i dont know if this is a generic thing, but you long press and get a description <jarhar> Luke: pro of that is that it at least matches one platform, but there are issues to overcome, it coudl be overloaded or mess with text selection <jarhar> Luke: buttons in some uas dont even have text selection enabled, links you can text select from a link very well because it loads the context menu <jarhar> Luke: some users might not be able to do long press so we need to figure out at story as well <jarhar> Luke: we could hook it into talkback somehow <jarhar> Luke: or if its a thing in ats <jarhar> Luke: we would have to handle focus differently, on mobile the prototype is a bit weird <jarhar> Luke: and the other bit thats not too big a deal is that its hard to polyfill because theres no longpress event <jarhar> Luke: you would have to impelentnt that yourself <jarhar> q? <keithamus> q+ <jarhar> gregwhitworth: i missed the last discussion. the interest invokers statement, what is the actual job to be done? i conceptually understand it with the long press <jarhar> gregwhitworth: im making an assumption that its going to occur. i guess that is an interest in the popup <jarhar> gregwhitworth: are you able to give me a one sentence heres the job its trying to fulfill <jarhar> gregwhitworth: how long its held down to invoke a popover <jarhar> keithamus: i think there are two strong use cases for the application of interest invokers <jarhar> keithamus: one is tooltips, and another is page previews, like wikipedia or github <jarhar> keithamus: on any input devices, the idea is that it should be an at a glance discovery of the element youre showing interest on <jarhar> keithamus: users avatar on twitter or facebook or whatever, <jarhar> gregwhitworth: that gives me the use case <jarhar> gregwhitworth: i guess my - i agree that if you go down this road it should be a must. you've called out the problem that the devils in the details, how do we get everyone to agree <jarhar> gregwhitworth: im curious if we're not able to - pointerevents is very generic, is doesn't define behaviors on certain elements <jarhar> gregwhitworth: if that was the resolution i would +1 it, but the devils in the details <gregwhitworth> ack gregwhitworth <gregwhitworth> ack keithamus <jarhar> keithamus: my general feelings on this is that its potentially hard or impossible to spec <jarhar> keithamus: i do think its easier with mouse and keyboard <jarhar> keithamus: then it gets very difficult when we talk about touch screens because there are many applications and they treat ui differently <sanketj_> q+ <jarhar> keithamus: keyboard and mouse are standardized, mice have 2 buttons <jarhar> keithamus: keyboard universally have a tab key <jarhar> keithamus: as luke points out, long press on android is different from ios <gregwhitworth> q+ <jarhar> keithamus: i cant imagine us being able to spec or suggest something that would fucntion idiomatically that apple would be satisfied with <jarhar> keithamus: i dont know how much leverage we would have to add some verbage in the spec that says you gotta do this <jarhar> keithamus: yeah sure they gotta do everything <jarhar> keithamus: tail call recursions in the ecmascript spec but 2/3 browsers dont do it <jarhar> keithamus: we can say it, but the rubber has to meet the road somewhere <jarhar> keithamus: not that we shouldn't do it, but whats the tractable part of this <gregwhitworth> ack sanketj_ <jarhar> sanketj_: i want to +1 what others have said. theres a lot of behaviors that are different on different platforms when it comes to touch. i dont think the ecosystem is going to get to a point where theyre consistent <jarhar> sanketj_: ipads have this feature called tap to hover, when you tap on something you get a hover preview <jarhar> sanketj_: bringint that to other touch devices has been problematic <keithamus> q+ <jarhar> sanketj_: i dont think theres going to be a standardized way to say that this kind of touch input should do this everywhere <jarhar> sanketj_: maybe theres no other way than to say its up to the ua, or we dont say anything, which feels worse <gregwhitworth> ack gregwhitworth <gregwhitworth> ack keithamus <jarhar> keithamus: theres other things to think about to spark some imagination, tvs have a very different mechanism for focus and stuff <jarhar> keithamus: usually you have a dpad, and i dont think we want to have focus follow interest in that example because the only way to navigate is with focus, and it exposes more than a tv user would want <jarhar> keithamus: and of course eye tracking - you cant have interest land on wherever the user is looking at <jarhar> keithamus: so theres a whole world of considerations there. is it a delay, a visual indicator that youre about to expose that your einterested in this element <jarhar> keithamus: theyre in the camp of this - its like mason, its an unsolved problem. if it was solved then it would be consistent, but we dont even have consistency within one as sanket points out <jarhar> Luke: question 1: do we want to want to require touch devices to have some mechanism to show interest? <jarhar> Luke: while that may seem slihgtly unimportant, the key thing is that im building an interface and im using interesst target, can i use it knowing that it wont show on all platforms? <zcorpan> q+ <keithamus> q+ <jarhar> Luke: could it be on mobile devices like wikipedias preview it doesn't show and thats ok? <jarhar> Luke: or do we want a mechanism on those platforms <jarhar> Luke: it could be like mason points out that we dont require it for keyboard or mouse, up to the ua to decide what they want to do <jarhar> Luke: they should say that you should provide interst nomatter the modality <bkardell_> "you should no matter the modality" makes sense <jarhar> Luke: i dont want to end up in a situation where its only implemented for mouse and keybard <jarhar> Luke: i want to see long pressing on a button do something <jarhar> Luke: but at the same time i dont want to put too much emphasis on uas to do stuff and delay the whole thing <gregwhitworth> ack zcorpan <jarhar> zcorpan: one possible user interface for touch devices would be to render another button next to the thing and you would touch that to get the popover <jarhar> zcorpan: maybe that could be an opt in <jarhar> zcorpan: if the author feels that its important and needs to be availble everywhere <scotto8> q+ <jarhar> zcorpan: please make it available and show another button <jarhar> zcorpan: with mouse and keyboard, its more discoverable because you hover or focus before click/enter <jarhar> zcorpan: with touch you just touch and theres no opportunity for discovering <jarhar> zcorpan: but if you have a questionmark button next to it then you have something to tap <gregwhitworth> ack keithamus <jarhar> keithamus: i think that sounds good, i would want that to be a pseudo element so its configurable for design system <jarhar> keithamus: i wonder if we could map it to existing heuristics <jarhar> keithamus: whenever focus visible is applied, then it should or must show an invoker. maybe that covers some use cases? <gregwhitworth> ack scotto <jarhar> scotto8: i was going to +1 simons idea <gregwhitworth> s/scotto8/scotto <jarhar> scotto8: dont make it an opt in, because if someone sees a bunch of these buttons show up everywhere maybe they'll rethink their life choices, and then they design it in a way where they dont have all these things showing up on touch <jarhar> scotto8: i think it makes sense to have that be the way that it does represent itself <jarhar> scotto8: if someones not happy with how that renders, then they can change their ui and make it something more consistent across devices <jarhar> Luke: alternative to that is we dont do it auitomatically at all and they can implement it themselves <jarhar> Luke: we have css media queries for input modalities, so you could hide your interest invoker and use a real invoker instead - or prevent it from doing what you want <jarhar> Luke: good ideas <jarhar> scotto8: to coutner that, if there was also a pseudo element then someone could use css media queries and then add their own and then best of both worlds <jarhar> scotto8: i just worry about not doing it by default, gives people false belief that thjey dont have to do anything <jarhar> scotto8: when working with developers, html allows you to do this, that doesnt mean you should <jarhar> gregwhitworth: we are at time, we will have to pick this up again <gregwhitworth> Zakim, end meeting |
On Chrome / Firefox, |
And drag and drop, and contextmenu of course. You can have all of these things defined and they already get in the way of each other, this is one more thing but seems fine. Some platforms have a shorter timeout plus haptic feedback for one action and a longer long press timeout for a different action. E.g. The homescreen on iOS picks up the icon for dragging before showing the menu. This is something we could explore to disambiguate. |
|
The Open UI Community Group just discussed
The full IRC log of that discussion<hdv> Luke: we didn't get to a resolution on this last time<masonf> q+ <flackr> q+ <hdv> Luke: will we require invokers to always be invokable, try our best to make it work for all modalities, or do we accept it's not necessarily possible in all modalities? <hdv> Luke: at least on Android there's buttons you can long press to show tooltips, not sure if there's similar mechanisms on other platforms like iOS <hdv> Luke: one issue around this is that interest should naturally be before invoking… if you have the long press, that's naturally more of an invokation than showing interestg <hdv> s/interestg/interest <keithamus> q+ <hdv> Luke: there's also the issue that longpress on some elements already does stuff, eg on links. So can we spec this, would UAs actually be able to implement that? <hdv> Luke: or would people get a separate toggle? <hdv> Luke: personally feel we'd rather have something buitl in if anything <hdv> ack mason <hdv> masonf: my opinion is that for keyboard and for mouse, the standard should dictate how it works <hdv> masonf: the spec should also say for other input modalities, like touch and others, there should 'be a way' to show interest <hdv> masonf: that way being up to the UA, because of the many differences between platforms today <hdv> masonf: what's most important is that all users can do the thing <hdv> masonf: and probably should be specced that interest is before invoke <masonf> ack flackr <keithamus> q- <hdv> flackr: I agree, we should have a way to access these with touch. these features should work with the standard modality of the device, on some devices maybe you have something like gaze… but on some devices you might only have touch and that should be supported <hdv> flackr: there are lots of ways people have talked about this… there are a lot of overloaded actions <hdv> flackr: there's menu actions, drag and drop… many ways this has been done in native UIs <hdv> flackr: could see this working if interest is nondestructive, you could probably do thisif the finger lingered long enough without scroll <masonf> q? <hdv> flackr: there are a lot of details around how this interacts with other featueres <hdv> s/featueres/features <hdv> flackr: another issue is disembiguating between different actions, eg you could have drag and drop and context menu at the same time that doesn't conflict <hdv> Luke: as in HTML drag and drop? does that support touch devices? <hdv> flackr: yes on iOS, there's work underway for Android too <hdv> masonf: how standardised is this? <hdv> flackr: that's literally the homescreen behaviour on iOS and Android… but on the web it's not standardised <hdv> masonf: my fear is how it is supported <hdv> flackr: even if we don't spec it you would still get a contextmenu on longpress <hdv> flackr: iOS supports this and I think Android does in some contexts <hdv> masonf: I don't know if there is a standard for contextmenu, I think there isn't detail in the spec on long press vs hard press <masonf> q? <hdv> flackr: for 3d press could look at maximum pressure or something like that <hdv> Luke: we could resolve that we should make it a MUST, for any given device/set of modalities, at least one of them should be available to show interest? <masonf> Proposed resolution: the spec should say that UAs "must" support showing interest via any input modality. <hdv> Luke: any input modality seems ambiguous <hdv> dbaron: kind of like 'any user must be able to'? <masonf> Proposed resolution: the spec should say that any user "must" be able to show interest via some input modality. <Luke> +1 <keithamus> +1 <hdv> +1 <flackr> +1 <masonf> RESOLVED: the spec should say that any user "must" be able to show interest via some input modality. <dbaron> +1 <brecht_dr> +1 |
mfreed7
removed
the
agenda+
|
Here's a new idea for touch, from an internal discussion (credit to @b1tr0t): On touch platforms (or potentially any platform where the UA determines that there's no mouse/keyboard?) the UA adds a pseudo-element button (such as ℹ️) next to the This idea gets around the problems of using long-touch, and is actually a cow-path paved by several design systems. Agenda+ to discuss. |
mfreed7
added
the
agenda+
|
Adding in here that this is a pattern that is a fairly common pattern, for example commerce checkout flows. Smashing Mag did a writeup a few years ago that covers tooltip patterns, including this one. https://www.smashingmagazine.com/2021/02/designing-tooltips-mobile-user-interfaces/ |
|
The Open UI Community Group just discussed The full IRC log of that discussion<hdv> masonf: this is about interesttarget and interest invokers<hdv> masonf: webkit folks are against this feature because it is not well defined on toch <hdv> s/toch/touch <hdv> masonf: there are solutions for touch and keyboard <gregwhitworth> q+ <hdv> masonf: we actually resolved the spec for interesttarget would say it should cover touch but doesn't say how <hdv> masonf: it's kind of nebulous which understandably the webkit folks don't particularly like <hdv> masonf: so we're looking for something that works across the board. Touch is very overloaded. When you try to set target on something that is not a button, eg to provide a definition for the text in a <span> … there is a couple of things that could happen on touch, you could long touch to copy paste, also to show a context menu… <hdv> masonf: now if we add interesttargets and devs start using them everywhere, then copy pasting and context menus are harder to use <hdv> masonf: idea: on some platforms, an extra button shows up when there is an interest target, then when you click on that button that indicates you have interest in that target <hdv> masonf: this would be a lot more discoverable and a lot more out of the way… and it could have a pseudo element that is fully stylable <hdv> masonf: could also be a popover=manual that is anchor positioned <una> q? <una> q= <una> q+ <hdv> masonf: curious what use cases would be broken by that <gregwhitworth> ack una <sanketj_> q+ <hdv> una: very interesting approach that I really like… but it seems like this means there would be different use cases in mobile vs desktop, which authors would need to separately author <hdv> una: could use a pointer media query that checks for touch input <hdv> una: you wouldnt want that on desktop if you had a pointer device, you wouldn't want that on mobile (using mobile as shorthand for no pointer) <scott> q+ <hdv> masonf: yes idea would be this button only shows up when you have a coarse pointer <hdv> una: really like the idea of handling this with some kind of click handler on certain pointer devices <hdv> una: a developer should have the ownership on when and how it appears <hdv> una: eg the github preview repo might look differrent than a preview card on social media… should prob be up to the author <hdv> masonf: agree it should be author styleable but with default that it shows up <hdv> masonf: does it sound reasonable to make it show up by default? <hdv> una: worry that developers might get things they are not expecting <hdv> masonf: if we make it a popover that is anchor positioned, that probably doesn't have weird side effects as it doesn't affect flow layout and positioning <hdv> una: still think it obscuring content would be a problem <hdv> una: can see it get really messy really quickly <hdv> gregwhitworth: is there historical precedence we can lean on with touch events moving to pointer events… as we don't know every form factor now and in the future <una> ^especially if there is a dense layout with interest invokers <una> ^people use the interest modality to help with data visualization on dense layouts <hdv> gregwhitworth: what we put forward solves the problem, but worry that setting vision aside would cause problems along the way <hdv> gregwhitworth: if we start defining what happens on hover by user agents I think we'll get pushback <hdv> gregwhitworth: I agree with una, we could end with issues <gregwhitworth> ack gregwhitworth <gregwhitworth> ack sanketj_ <hdv> gregwhitworth: you could probably not either/or any of this, eg lots of users would use touch and then also mouse, I used to on Surface <hdv> sanketj_: yes there are devices that have keyboard, mouse and touch, not just Microsoft's… so the risk of getting heuristics wrong is real <hdv> sanketj_: I know Chrome, on desktop touch, has a menu that shows up on touch, that has compat concerns <hdv> sanketj_: I too worry side effects could happen, not sure if there is a good interoperable option we could decide on <gregwhitworth> +1 to what Sanket just said <hdv> sanketj_: can we define, if interesttarget is set and long touch is done, can we define that other actions like context menu and copy/paste just aren't available <gregwhitworth> q? <hdv> masonf: I think users would find that annoying <hdv> q+ <hdv> masonf: on my iOS device, if I long press a link I want to copy/paste it, instead I get a giant preview… I don't like that… and also developers, you're removing a previous capability they had <hdv> sanketj_: I see… probably not as much as issue for a button, but would be for additional things <hdv> masonf: agree, button is prob not a problem here <masonf> q? <gregwhitworth> ack scott <hdv> scott: I like the idea, it's interesting <hdv> q- <una> q+ <hdv> scott: would be curious to see what other browsers come up with… would be interesting to allow developers to style it, but not sure if developers should be able to decide whether to show/hide it <hdv> scott: because it's ultimately about the user, they are first in everything we do <brecht_dr> q+ <hdv> scott: could imagine a bare bones user agent representation of it while we ensure users can get to the content <gregwhitworth> ack hdv <gregwhitworth> ack una <hdv> una: want to respectfully disagree with an example… am looking at an example right now that I think would be a worse user experience if automate <hdv> s/automate/automated <hdv> una: [shows GitHub home page, hovering over links where each individual link to a name of a repo as well as an image, each of these shows a popover on hover] <hdv> una: if we add an extra button to show these on mobile, there would be a lot of buttons <Penny> q+ <hdv> una: I think we'd degrade the user experience if we cluttered up the UI with lots of buttons, especially in this example where all these are links <hdv> scott: actually I agree authors should be able to turn this off but users should be able to override <hdv> una: do you envision the user control to be display settings for the user where they could force interest els to show? <hdv> scott: yes <gregwhitworth> ack brecht_dr <hdv> una: I think that'd be a good way to do it, gives the user control and allows them to see if it makes things better <hdv> brecht_dr: +1 to not wanting too much clutter <hdv> brecht_dr: as a user I don't want to think about the difference between desktop and mobile… <gregwhitworth> ack Penny <sanketj_> q+ <hdv> Penny: I think it'd be good if there is a global and a per case setting, and a user pref that overrides everything <masonf> q+ <hdv> Penny: maybe with the right combination of settings we can overcome the challenges <gregwhitworth> ack sanketj_ <sanketj_> I'll type in my question. <gregwhitworth> q+ <hdv> gregwhitworth: is there a preference for something that we are going to standardise as a user preference? <gregwhitworth> ack gregwhitworth <hdv> gregwhitworth: users and developers would have to be able to rely on the setting existing <hdv> masonf: I don't think we've ever standarised on a setting like that <hdv> s/we've/we can or have <hdv> s/standarised/standardised <bkardell_> q+ to say I suggested this for tabs <hdv> gregwhitworth: looks like no webkit folks here… if I put their hat on… they'd probably expect this to be on by default? <sanketj_> The unavailability of hover on touch is a general problem. Should we think about having recommending/mandating UAs to provide a option (ex. contextmenu option) to view the invoker target on long press? <hdv> masonf: that is kind of the backup plan I think <hdv> masonf: I did here some pushback on that too <hdv> masonf: feedback I heard was it would be a lot of clicks to show interest <gregwhitworth> ack masonf <hdv> masonf: thanks all for the feedback today… some of this changed my opinion on things. Global on/off seems to be the right thing, not per platform <hdv> masonf: little icon should be on or off across the board and developer can change this <hdv> masonf: we can show a little i-icon when folks use interest and make it very easy to turn it off <bkardell_> s/is there a preference for something/is there a precedent for something <brecht_dr> q+ <hdv> masonf: was pushing for popover to keep it out of flow before but not sure on that now <gregwhitworth> ack bkardell_ <Zakim> bkardell_, you wanted to say I suggested this for tabs <sanketj_> I think I like the "backup plan" option better. Maybe the requirement would be that the UA needs to provide some type of UI to view interest - what that looks like may vary. Agree that the context menu has its limitations. <hdv> bkardell_: it's come up a few times over the years… I don't think we have ever done that but it does keep coming up <chrishtr> I really like the idea of showing the pseudo element icon by default on every platform; developers can then remove it / restyle it as needed. <hdv> bkardell_: maybe there isn't a good answer… and UA nor author can provide it, it's really a user preference thing <hdv> bkardell_: so maybe the best thing we can do is to make it very easy… if it's not observable that's probably easy, if not it's a bit harder <gregwhitworth> ack brecht_dr <hdv> brecht_dr: to masonf's earlier point… I think if we go that route we should be able to style the icon as well. And then there could be some other solutions on touch devices <hdv> masonf: proposal definitely was that it would be a pseudo element that could be styled in all the ways pseudo els can <hdv> una: curious to hear feedback from Apple on this <hdv> una: I think it's a very interesting path forward <hdv> masonf: I think next step would be to write out the proposal and maybe have the discusson on the webkit thread that already exists on this and has a lot of discussion already <gregwhitworth> Zakim, end meeting |
|
Great discussion just now, thanks all. So I'm coming down to a more concrete proposal: <button interesttarget=foo>Button</button>
<span interesttarget=bar>some text</span>The idea would be that both of those elements (the button and span) would generate pseudo elements that could be targeted like: [interesttarget]::interest-target-icon {
content: "?";
background-color: linen;
border: 0;
}which would be presented as:
Clicking those To address the concerns raised in the meeting:
|
|
(Let's keep this marked with Agenda+ so we can discuss again next week, please.) |
|
I don't agree with this direction, for the same reasons <button interesttarget=foo>Button</button>
<button commandfor=foo>?</button>
This is a really unusual default.
What if a developer wants the interest to be triggered on focus? I think that's what most folks will want.
I doubt browsers will add this, as it's extremely hard to describe to a user. Folks thinking about this should consider how they'd implement something as common as the toolbar in Google Docs. I guess they'd have to:
If I was on that team, I'd say it sounds easier just implementing it manually with JS, rather than having to fight the browser's default behaviour. |
|
@jakearchibald just double checking you are aware this (edit: latest) conversation / proposal was being had for what to do for mobile/touch (dare i say even voice) scenarios where hover/focus may not even be something the user could do / easily do. just asking because i am not disagreeing with any of your comments in regard to a traditional desktop experience, but they aren't reading, to me, as acknowledging the use case this was trying to solve for or what to potentially do instead. |
|
I formally acknowledge the mobile use-case 😄, but I'm pointing out that the proposals in #1052 (comment) make the API hard to use in cases it was designed for, and the result will be that people will avoid this API and do what they actually want with JS instead. That seems like a bad outcome for this API. In cases where folks want a way to activate interest targets on click, that's what |
|
thanks for the acknowledgement / expansion on your thoughts, @jakearchibald, they are appreciated :) hope to see you in open ui meetings again, someday |
I'm glad you're supportive of mobile/touch. Sounds like you're also supportive of an API that tries to address these use cases (hover-triggered popovers, menus, etc.)? If so, and if you're against this recent suggestion, do you have an alternative idea? Without a concrete (and explicit) solution for touch users, WebKit is against this API. (Yes, that's a standards position for
This is a very good point. I wonder if the pseudo element could somehow be a child of the element, rather than a sibling, to avoid the problem?
Yeah, perhaps it could still be triggered on hover and focus. That was, in fact, the original proposal before the meeting. I think the hard part is providing developers a good hook to turn these extra buttons on only when the user needs them, i.e. when there's no keyboard or mouse present. Is there a good media query that will do that correctly?
I can speak for one browser that would be willing to try to describe this in a sentence or two. 😄
The last sentence is interesting to me - what would that JS do? Whatever it is, that's the behavior we're trying to build into the platform. Unless, of course, it's just to say "too bad about those touch users - they'll just have to be in the dark about what all these toolbar buttons do. Adding affordances for them would make things 'ugly'." And my fear is that this is exactly what most design systems do today, precisely because they don't have a standardized way to handle this. |
In terms of what the expected platform behavior is, on Android it is long press to show tips. On iOS where long press doesn't work I also haven't seen ℹ️ icons to tell me what buttons do so I'm not sure if iOS has a pattern other than ensuring that your interface has text or is reversible? I would prefer to see us aligning with native toolkits if possible. We do have to wrestle with the fact that this is an overloaded interaction. I think an acceptable way to deal with this might be to warn developers when they have a tooltip on an element that is also draggable or also has a contextmenu listener. |
|
One concern I'd have with having additional icons is that this is typically on devices where screen real-estate is far more restricted and so is likely to result in things no longer fitting if you add this. |
Have I done or said anything in the past that gave you the impression that I'm not, or that I wasn't?
Yes, I already mentioned it in both of my previous comments: <button interesttarget=foo>Button</button>
<button commandfor=foo>?</button>This allows the developer to put the "command" alternative wherever they want. It also allows for things like a toolbar to have a single command alternative that covers the 'help' for that whole toolbar. There are cases where you want the tooltip to appear on
That's unfortunate. Lack of interoperability is a failure case. However, if Chromium and WebKit shake hands on an API that developers won't want to use, that's still a failure. I think it's ok to have APIs that lean into advantages of a particular device. Safari released Information that's available on hover can be a nice enhancement for pointer users, but of course developers should provide an alternative way to access that information. There are many ways to do that, such as labels, and help buttons that cover multiple hover points. For example, you could have a toggle that adds labels to all or a subset of the icons, which would take up more space, but could be toggled back away. If the content is fewer actions away on desktop vs mobile, I don't think that's an issue. Just as the Safari AR experience is enhanced on mobile. Just as many desktop sites place content in-view that's a tap away on mobile, due to the space available. I don't understand why we're looking for a lowest-common-denominator approach.
That's how
I know, that's why I was complaining about the new direction. I think it's important to have use-cases in mind, and if there's a big change in direction, write down or at least think about how developers would hit those use-cases with the changes proposed.
There's
I'm sceptical, but I've been surprised before. What's the draft wording so far?
It would try to provide the best UX for all users. A lot of developers might get that wrong, but if you push developers away from your 'ideal' API by making it unsuitable for a number of use-cases, or make it feel unreliable due to footguns, then that's the consequence. This is well documented. OpenUI has a whole customisable select effort because developers badly recreate
I assumed that was still the goal. Which is why I voiced my concerns, because the goal is being missed with this new direction.
I hope this group remains a safe place to discuss and debate API changes. Specifically, I don't want to see alternative viewpoints shouted down as "you don't care about [some section of users]" when that clearly isn't the case. Also, visual design is a key factor in the user experience of any user that consumes the web visually. Web design is a major part of what makes the web work, and folks who do it well are highly skilled. Please don't dismiss it as pretty vs ugly. |
Right - that can work, but it's manual. The same for monitoring touch start/end to detect long-press. I'm hoping we can find the best and most common pattern for touch and provide an easy declarative API for that. It is starting to sound like the "add (i) buttons to everything" approach is not that common pattern. 😄
Agreed, for sure. Hence this discussion.
I think the issue is that since this is such a tricky thing to implement (get it right for mouse, keyboard, touch, AT, hook up a11y mappings correctly, etc.) many design systems simply "punt" all but the hover interaction. They ensure the information in the hovercard isn't "necessary" for the user to see, and then just don't show it to those users. Which I see as unfortunate.
Good to hear. That's still one contender - add a "show interest" item to the UA-provided long-press menu. It's more difficult to do, but at least it's available.
Welcome to web standards. 😄
Yeah, thanks for the comments. Based on your responses and a bunch of others, it does seem like the "automatically add a (i) button to everything" approach is likely not the right one. I'd like to have a discussion about this at TPAC, and I'll try to put together a presentation with use cases and potential approaches.
I don't think we can write the wording for a feature that controls something that isn't defined yet.
Agreed.
Sorry if you felt that way! I honestly didn't intend it as trying to shout anything down. (Perhaps be a little provocative?) On the contrary, your response (plus many others) have lead me to conclude that this approach is likely the wrong direction. This discussion, plus the live one we had last week, has been exactly what I was looking for. So thanks! And I hope you can keep contributing your ideas so we can find a good solution. My motivation is simply to make sure we've heard all of the ideas so that we can pick the best one. This feature is like an open field: the cows go a number of different directions, so the paths aren't well worn. But they're all trying to go in a general direction, so we need to find the best of those faint paths to pave. |
|
The Open UI Community Group just discussed The full IRC log of that discussion<jarhar> masonf: the question is how to support touch for interest target<jarhar> masonf: one suggestion we dug into was what if we make a button pop up next to everything with an interest target and touch users can tap it <jarhar> masonf: that was viewed as a not good way to do it because theres visual clutter <jarhar> masonf: and extra tab stops for keyboard users <jarhar> masonf: conclusion was that this is not the right approach <jarhar> masonf: the one approach i keep hearing is long press <jarhar> masonf: it seems like long press is the standard mechanism for this kind of thing <jarhar> masonf: the problem is that theres no easy api for that that isnt overriden by what current browsers already do for things like links <jarhar> masonf: maybe the best way forward is to spec interest target on touch screens as long press - then a context menu shows up, and the first item in that context menu should be to show interest in this element <jarhar> masonf: why is that a terrible idea? <jarhar> q? <masonf> q? <flackr> q+ <jarhar> flackr: if we don't already have something that we're doing on long press, i feel like we shouldn't be showing an unecessary menu <jarhar> flackr: it is already the case that buttons on android, long press shows your interest based thing in native android <jarhar> flackr: i think we should do the same for buttons because there is no long press menu for buttons on mobile <jarhar> flackr: where this gets challenging is cases where we do have some sort of menu for links or images <keithamus> q+ <jarhar> flackr: it might be a good idea. trying to think of what the alterantive could be. could you long press and then move your finger to indicate that you didnt want to open the menu or timers <jarhar> flackr: maybe what you proposed is the best we could do <jarhar> masonf: there has to be a way for the user to know to do this. if theyre used to doing long press and the context menu shows interest its discoverable <masonf> ack flackr <jarhar> flackr: if the timer was different, like the fact that something happened before the menu showed, that would be an indicator to the user to do something there <jarhar> flackr: timer based things are tricky <jarhar> masonf: the key thing for me is that its discoverable somehow. either it gives them a clue or some way for the user agent to tell <masonf> ack keith <jarhar> keithamus: the main use case for interest is showing some kind of popup. we can lean into that pattern a bit. i would make the case of why not show both the context menu and the popup? <jarhar> keithamus: if i long press on a link, i get a context menu and a page preview. there is nothing blocking us from taking the popover element and putting that in the page preview <jarhar> keithamus: native applications give you the same kind of thing <jarhar> keithamus: you could envisage a popup next to the context menu showing the additional stuff for a long press <flackr> +1 <jarhar> masonf: there may be issues with that. that preview on ios is the actual preview of the actual site. there may be some spoofing issues where you show a fake version of a site that you might get that isn't the same as the one if you tap on it <jarhar> keithamus: there would need to be some visual treatment to disambiguate <scott> q+ <jarhar> keithamus: to me, this is the central thing that i would like to take to a broader audience to challenge the idea that we cannot do this. the main pushback being that we can't coopt long press because of existing ui <flackr> +1 <una> +1 also <jarhar> keithamus: if we're aiming to have the web platform operate with the same power as native apps, this is functionality that native apps can handle, so its not outside the realm of possibility <una> q? <jarhar> keithamus: theres a ton of options that we can explore, thats the direction i would like to take this <jarhar> masonf: i think the pushback isn't that we can't touch long press, although i tmight be, i heard that theres no paved cowpath on the web <jarhar> masonf: no design system handles touch users, therefore we don't need to <jarhar> flackr: we did implement legacy behavior to support hover and active target on touch, so websites using hover are possible to navigate with touch because the browser says that if you touch something youre also hovering it <jarhar> masonf: links are the thing that have a very special behavior on one os which would have to be changed to support this <jarhar> flackr: i want to put this in the native ui, preview plus info thing <masonf> q? <jarhar> flackr: the ua should be able to preserve the ability to preserve the preview but also show this thing <sanketj_> q+ <masonf> ack scott <una> q+ <flackr> q+ <jarhar> scott: if we cant add things to the context menu, then it still holds some water. if they still dont like it anyway, then it sounds like theyre giving up <jarhar> masonf: the button idea didn't sound very popular <masonf> ack sanket <jarhar> sanketj_: are there touch surfaces where there is no context menu? <jarhar> flackr: what do you mean? on phones there is no context menu on most long presses except links and pictures <jarhar> masonf: there is kind of one on selection of text <jarhar> flackr: yeah that depends on selection, its sort of a context menui <jarhar> sanketj_: im not sure either. my intuition is that there is no context menu for interest targets. with lack of a concrete example its hard to have a discussion, but if there is, then do we have a proposal that works in that case? <jarhar> masonf: robs suggestions was good. if there would have been a context menu then do a special thing. if there would not have been a context menu, then just show the popover <jarhar> sanketj_: that makes sense. about the context menu: if the contextmenu event is preventdefaulted, would we still show the preview in that case? <jarhar> masonf: that seems like a case where you would fire the interest immediately because theres no context menu to show up <jarhar> sanketj_: we would have to handle that case <jarhar> flackr: in that case the context menu is interest, and the developer wants to do something else. ios also doesn't fire this <Penny> q+ <jarhar> masonf: if youve preventdefaulted the contextmenu event then we shouldn't do interest? <jarhar> masonf: what if they want to do their own thing with an interest invoker? <jarhar> sanketj_: whatever we choose here, contextmenu is going to fire for touch and non touch <jarhar> flackr: the default action for contextmenu for touch would be to show interest. for mouse we woulnd't say that it is to show intere3st <jarhar> sanketj_: what about some future type of input thats not mouse or touch? <masonf> q? <jarhar> keithamus: eye tracking would need an explicit gesture. we discussed this at tpac last year <jarhar> keithamus: theres a variety of different devices, each would need their own gesture. a playstation dpad moves to the nearest focusable element but its not like a cursor pointer, its more like an assistive cursor. doesn't change focus but makes it so you can <jarhar> keithamus: all these devices are already built around this notion of focus being the central modality for interest for that input devices <jarhar> keithamus: controller has roaming cursor that you emit focus with, visual devices woudl be the same <jarhar> keithamus: for new technologies, interest would be the same as focus, and we would think of hover as being legacy interest <jarhar> flackr: if you roam to the button, you want to know what the button will do if you activate it <jarhar> masonf: i think keith is saying that focusing the button is showing interest <jarhar> keithamus: theres an explicit action to focus the element by pressing the circle button, which is separte from click <jarhar> flackr: we need separate things for interest <masonf> ack una <jarhar> una: ive been looking at hover cards on apps and sites. facebook on safari prevent default long press and has a custom menu that pops up, and some other custom links in there <jarhar> una: its not a common pattern, its just hard to do that. other sites with hover cards dont take advantage of that, it takes a lot of scripting. its the most natural behavior, long press, which we could spec, and have that be a part of how you access interest target <jarhar> una: with apps, its a lot easier to access the long press behavior. on mobile web its harder. it highlights a gap <jarhar> una: i agree that the most seamless user experience would be long press. cool to see that theres some precedence for it on a website <masonf> ack flackr <jarhar> flackr: the two options arent exclusive - adding a button and having hover show interest <keithamus> q+ <jarhar> flackr: you could image a pseudo class that would allow for creating a button to show interest. if we cant do context menu, then we could have an interest button <jarhar> flackr: solves use cases in issue where we cant put things in toolbars, but text has more room for buttons <masonf> ack Penny <jarhar> Penny: we're saying that its up to the developer to indicate that there is something to explore via long press by coloring differently etc. <jarhar> masonf: rather than have one built in? <jarhar> Penny: have one built in, or any sort of indicator like a pseudo element or a color or anything else <jarhar> una: that should be up to the author <jarhar> masonf: you find this on your own by seeing what can be long pressed <masonf> q? <jarhar> flackr: if we make it focusable, then its better than hover today <brecht_dr> q+ <jarhar> keithamus: if its just for links and buttons then they would be focusable anyway <jarhar> keithamus: what if you did tabindex=-1? thats an author error right? <masonf> ack keith <masonf> ack brecht <jarhar> brecht_dr: +1 on long press idea <jarhar> brecht_dr: a thing that came to mind is that ive had situations where people would prevent right click on a website which is evil and has been done in the past <jarhar> brecht_dr: what if people add everything inside a button, then choose to use invokers so that context menu cannot be opened? <jarhar> brecht_dr: just want to throw that out there. its something we should think about <jarhar> masonf: they can already do that today right? by preventdefaulting the contextmenu event <jarhar> masonf: people are going to be evil <masonf> q? <scott> q+ <jarhar> masonf: what other problems would this cause? good question to ask <jarhar> masonf: seems like support is for long press being the thing, and when appropriate showing it in the context menu <masonf> ack scott <jarhar> scott: i heard something about making elements focusable, and that made me wonder what was going on. would people be putting this on otherwise not focusable elements? <jarhar> scott: that would destroy keyboard navigation <brecht_dr> q+ <jarhar> masonf: that ones still an open question. current proposal is that this only works on links and buttons, and i think that covers the majority use case ive seen. definitely use cases for non link invokers, but that needs more thought <jarhar> masonf: if we were to allow it on any element then we should make it focusable or find a way to let keyboard users find it <jarhar> masonf: the v1 thing is buttons and links. for now it doesn't work - its the kind of thing we can expand to later if we need to <jarhar> scott: maybe this can be solved in the general api for long press, but i know that for people with motor disabilities long press is not easy for them. if you have voiceover or talkback running, long press requires both someone long pressing to send their press through the screenreader and then another thing to make the long press actually work, its <jarhar> complicated <jarhar> scott: ive spoken to people about this and they hate it. if they have an actions available context menu, then that would be a way to surface that to talkback users at least <jarhar> scott: this may come up in further conversations that people have prlblems with long press <jarhar> flackr: does selecting something with talk back - could that show interest? <jarhar> scott: that would be bad. touching anything means that youre trying to read it but not that youre showing interest in it <jarhar> masonf: how does this interact with aria details? doesn't it tell you theres soemthing else you can do here? <jarhar> scott: that is a good question and is up to the screenreader. on desktop, its not always even exposed if you are using the virtual cursor to navigate. it is consistently exposed on windows <jarhar> scott: on ios and android, swiping - i think that ios doesn't fully support aria details at this point <masonf> q? <masonf> ack brecht <jarhar> brecht_dr: for input type submit, and certainly image maps, there might be something where people want to create an interest target. out of scope right now, but we should keep it in mind <jarhar> masonf: button-like things counted <jarhar> keithamus: invokers is just the button element, not input type button <jarhar> masonf: image maps also not supported? <jarhar> masonf: custom elements are even harder because we have the issue of making them button like <scott> clarification for minutes above: 'consistently exposed on windows if navigating using the tab key' <jarhar> sanketj_: that is one thing we've heard with popover <jarhar> keithamus: theres a big looming discussion about custom elements and these mixins <jarhar> keithamus: right now the only way to do a custom element is with scripting. what does a declarative custom element look like? that might unlock some of these. a lot of desire but not a lot of concrete suggestions <jarhar> masonf: the currently fully supported way is to do a button <jarhar> keithamus: theres semantic delegate which could forward all things to that button <masonf> q? <jarhar> masonf: im happy to leave it here with no resolutions |
|
I'd like to keep this on the agenda again for next week, to potentially discuss a keyboard-support idea. (Yes, this issue is called "touch inputs" but it's likely better titled "non-mouse inputs", since that's what we've been discussing here.) A common complaint I've heard about simply showing interest based on keyboard focus (either immediately or after a delay) is that it's noisy and distracting. Simply navigating the site via keyboard should not indicate interest in anything. A common pattern (used on Github actually!) is to use a special hotkey combination (Alt-ArrowUp for Github) to show interest and get hovercards to show up. The issue with that is that a) the hotkey isn't standardized, and b) it isn't very discoverable. For a UA-provided solution, what if the UA, after an element with
Thoughts? |
|
I know there's some push-back on putting stuff into top-layer with CSS, but if there was a CSS way to activate popovers, would that solve (or at least sidestep) the issues here? .some-parent:has(.some-button:hover) .some-popover {
appearance: show-popover; /* I'm sure there's a better syntax for this */
}
/* Or in many cases just: */
.some-button:hover .some-popover {
appearance: show-popover;
}Developers could choose if they want to activate the popover on This means there isn't a feature being added specifically for hover, it's just a CSS way of activating popovers. |
So I don't think it sidesteps the issues, does it? Or more accurately, it sidesteps them and leaves them as problems. This would make the the keyboard case (still) only doable via focus (and not something else), and would provide no solution for touchscreen. Is that right? Or perhaps I missed something about the proposal. |
That's really the definition of sidestepping.
Again: <button commandfor=foo>?</button>Or: .some-button:is(:hover, :focus) .some-popover {
appearance: show-popover;
}Yes, it does leave the solution to the developer, although much easier than it is today. I think it's clear that there are no cowpaths to pave here. The above is flexible, it makes some of the hard parts easier, but it doesn't force developers into an untested solution (like the long press, like the extra pseudo-element). It gives developers freedom to implement the best solution for their given situation, rather than having to go with the one dreamed up by a standards group. You can then observe where the cowpaths form, and make those easier too. |
FWIW the design of this is being iterated out from developers (for example GitHub) to build on the problem areas they currently have when building tooltips or richer interest based popover elements (for example GitHub's "Hovercard"s, Wikipedia Page Previews) so I think it's somewhat uncharitable to say we're going with a solution dreamed up by a standards group. Our intent is to make it easier to solve the kinds of problems developers (like GitHub) face in building a portable solution. |
Ok, but setting in stone a pattern that worked for GitHub in 2024 is problematic too. My point is: There doesn't seem to be cowpaths to pave in terms of one-true-way to handle focus and touch interaction here. So instead, make the common bits easier, while maintaining freedom to solve the focus & touch issues. My concern here is based that on new things being invented in this group (forced pseudo-elements, long-press) that don't seem to be solutions in the wild. Tooltips are pretty common bits of UI, so it seems like the right thing to do is make the common parts easier, without forcing new behaviours that don't seem to be what developers in general want. |
Yep I agree. I don't want us to design something here, I want to pave a cowpath. The cowpath is long press on mobile. Websites have been doing this for decades; Wikipedia has been offering Page Previews since 2014, GitHub started working on "Hovercards" in 2018:
So the pattern has existed in the web space for a long time. We're talking a fairly well trodden path for mouse and keyboard. And for good reason. It aids in usability and, per Wikipedias own documentation, can drive clicks:
On the mobile web it's difficult to do because it requires compromise. We can add long press event listeners and prevent the default operation, but that also means users lose the context menu, and users really like context menus, so we don't, despite wanting to. But native apps do this a lot. To wit; here's the wikipedia iOS app demonstrating the long-press preview card: 
Wikipedia the website doesn't do this, because - I presume - for the same reasons that GitHub doesn't, it would break context menus. You might think "well, this is a native app, they can do what they want, it's not a pre-ordained pattern from apple" but here's Apple's Contacts and Apple's Maps apps doing the same thing:
So, in my opinion, these are solutions in the wild. They're doable via Native apps, just not the web for some reason. Much like many discussions this is another one revolving around unlocking capabilities that native apps already easily do. |
|
Ah, I thought long-press wasn't a thing on iOS. That said, the interactions above do not look equivalent to hover, they look more like a decorated context menu. I'd expect those to be activated on right click on desktop rather than hover. |
|
They seem to serve double duty as both tooltip-reveal and context menu. This is what makes it very challenging to implement an equivalent pattern on iOS specifically. |
This is one interesting argument I've heard a couple times about touch. Is it your opinion that it's totally possible to support long-press on the web today, and developers just choose not to use it? What I've heard from multiple developers now is that it really isn't possible. You can take over long-press, but that removes the built-in context menu, and users (and therefore developers) don't like doing that. So at least for the folks I've spoken to, long-press really isn't an option, even though it's the desired option. Said another way, it's unfair to say long-press isn't a cow path, when there's a big fence blocking the cows from going in that direction. |
I agree with the premise that developers can't do this today because of the reason you mentioned, but I disagree with the conclusion that this means long-press isn't an option, when it is the option that is used in native UI. I don't think we should make developers on the web have to choose an alternate UI to do a thing that is established in native with long press. I thought that we were moving towards some idea where for links we could present the popover hint in the context menu similar to the examples shown in #1052 (comment) However, many of the use cases where developers want tooltips or other hover UI is on buttons where there is no long-press menu to compete with. In general, I think we should warn developers when they try to attach more than one long press behavior. |
Sorry if I wasn't clear - I meant it isn't an option today on the web. I do think we should provide an API ( |
|
The Open UI Community Group just discussed The full IRC log of that discussion<bkardell_> masonf: We've done this a few times, I keep bringing it back - I've added it to the agenda for TPAC as well<bkardell_> masonf: the one I wanted to bring today is about keyboard, but there has been some general discussion on the issue maybe we can talk about those things too - whether there should be an api <bkardell_> masonf: we have been talking about using focus itself as a trigger for keyboard and I've heard that this is not great from a noise standpoint because you're just tabbing through sequentially but the thing is reading to you or talking, or it's adjusting - making a tooltip with new things in it available <flackr> q+ <bkardell_> masonf: I've talked with some people who have done this in systems with like a hot key - the main downside I see is that it isn't discoverable, nor standard... the idea I wanted to throw out there is "what if it is a hotkey" and "what if the user provided a hint that there is a hotkey to activate that thing" -- maybe... it's not the best idea, but another tooltip... wdyt? <sanketj_> q+ <bkardell_> flackr: I don't have a good idea, but I wanted to ask if there are established patterns - any UI anywhere that has something like this <keithamus> q+ <bkardell_> masonf: GitHub, but it's not discoverable -- I didn't know about it, but I learned it and then wow, yes <bkardell_> flackr: I'd be curious if it is part of a larger pattern, or it's a one off. We shouldn't standardize if it is not a well established pattern <gregwhitworth> q+ scottohara <bkardell_> masonf: the more general question even is there is a chicken and egg problem - you can't establish a cowpath because there are fences in the way (??? poorly miinuted) <gregwhitworth> ack flackr <gregwhitworth> ack sanketj_ <bkardell_> sanketj_: I agree we need to find something that is a precedent to a certain extent. Keyboard things, as I understand, are for more power users. Even with hints to educate users, from what I have seen in other example - keyboards tend to have low use among regular users, mainly for power users <bkardell_> masonf: the keyboard is probably a kind of power user thing in general, isn't it? It seems almost like if we provide an api that allows you to show interest, and you have a keyboard, it either has to be tab or another key/hotkey, right? what else can you do? <bkardell_> sanketj_: space, enter, tab, yeah those are the common ones - it would be hard, I guess <bkardell_> masonf: the two ways this is done today is a hot key that doesn't get much use... or not at all <gregwhitworth> ack keithamus <bkardell_> keithamus: I can speak to Github's use case... we went through many iterations of how to make this discoverable. We tried focus, users found it distracting. We don't want focus to change the order of focus, there is trickiness to make sure that things that pop up don't become a potential sequential target unless you really want it to <bkardell_> keithamus: one way we've tried to help is by adding an aria-describedby - every link you landed on would say 'press blah to see more' and users kind of didn't like that too - you need to enable without being persistently annoying... <bkardell_> keithamus: how do you walk the fine l line of making something discoverable without being annoying? <gregwhitworth> ack scottohara <keithamus> q+ <gregwhitworth> q+ <bkardell_> scotto: Keith did a really good job summarizing - here at Microsoft we are hot-key happy. We have all of the different product teams encountering this issue and wanting to do this - and everyone does it differently because there is no standard. We don't want it inserting focus where you aren't intending to focus, you're just moving your interest around... you don' want to be bombarded... I would love a standard control that would allow <bkardell_> interest to move around <bkardell_> scotto: we're all using different keyboard commands in all of these places. Even the feedback we've talked about in issues is that we can only follow platform conventions, but there are none... I don't know what to do here <bkardell_> flackr: My point is not that we shouldn't do this - but rather not to settle on the first thing we think of or just because GitHub did it a way, as Scott said MS does it all different ways <bkardell_> masonf: You said ms has hot keys and GitHub has it, how do you handle discoverability? <flackr> q+ <bkardell_> scotto: The way this is commonly handled in projects I work with is, as Keith said, you don't want to be overbearing - it shouldn't be on every single instance/control. commonly this is part of the documentation for products. at least following wag guidelines the only thing that actually matters is that it be keyboard accessible - it is intended that you document it and let people know, but there aren't requirements <gregwhitworth> ack keithamus <bkardell_> scotto: it often goes into documentation, people file issues, we point them at the documentation and close the issue. repeat <bkardell_> keithamus: we removed the described by - we don't announce at all to screen readers, but if you press ctrl+? on any GitHub page you get a cheatsheet with how all of this works. I wish we had declarative keyboard shortcuts <masonf> shift-? --> how is that discoverable? It's discoverability all the way down. <bkardell_> keithamus: general comments if we are having a crisis of faith about tackling it: we should not run away from it, we should develop a pattern that makes this less custom. We should take the learnings from GitHub and Microsoft and make that better. I take issue with that we have to have figured this out, we don't do this because it's easy - we do it because it is hard <gregwhitworth> ack flackr <keithamus> q+ <bkardell_> flackr: access keys on browsers are a similar sort of thing. The majority of users don't know about them. They vary by OS and browser. I think it is ok, but it is a pattern we can look to maybe there is ... something. Where possible I'd love to build on some pattern used elsewhere <gregwhitworth> ack keithamus <bkardell_> keithamus: part of the shortcut work, I tried very hard to use Accesskey - but on the same OS/different browser they can vary. There is something to pull on here, I'm just not sure what <bkardell_> gregwhitworth: I feel like we need a meaningful proposal and research. Personally I found that historically if you come with the research and a proposal about where we should put the gate, it is harder to argue that we need to keep the fence in place <gregwhitworth> https://www.researchgate.net/publication/234800029_What_can_a_mouse_cursor_tell_us_more_Correlation_of_eyemouse_movements_on_web_browsing <bkardell_> gregwhitworth: I think this reminds me of skip to main before we solved it. <bkardell_> gregwhitworth: I remember if I call ms office having something where if you land in the head, if you are on AT, it will help you find the shortcuts. What I am trying to lead with the research is that you said interest isn't focus - this shows a strong correlation between mouse movement and eyes - is there something we can say here about 'interest' <bkardell_> q+ <bkardell_> gregwhitworth: we should spec 'interest' <bkardell_> gregwhitworth: I feel like 'skip to main' is a similar approach that we could adopt iteratively <masonf> q+ <masonf> q+ scotto <gregwhitworth> ack greg <gregwhitworth> ack bkardell_ <gregwhitworth> bkardell_: I know I brought this up prior, but David Bokham came up with a proposal called interest in 2018 that was presented at CSS F2F in Toronto <gregwhitworth> bkardell_: it's related to a bunch of different things that use on your TV, 2D to move your interest around but not focus <gregwhitworth> bkardell_: it's used in lots of simpler sorts of UI that their not touch enabled but they have a wheel or dpad, etc <gregwhitworth> bkardell_: I really think there are a lot of documentation and proposals and research that already exists and UI that has sequential order of the doc is not completely straight forward and you don't want to move in a single way <gregwhitworth> bkardell_: there is a lot of prior art to link up <gregwhitworth> bkardell_: I'd be supportive of some type of "interest" idea that isn't focus <gregwhitworth> ack masonf <gregwhitworth> 100% <gregwhitworth> ack scotto <bkardell_> masonf: I do like the idea, of maybe a separate feature, to say your page provides these features and your browser can provide those.. If we did something like that we could provide all of the things for discoverability. cool idea I'll think more about that <bkardell_> scotto: That is kind of where I was going to go too masonf - I like the idea of saying not what user agents must do, but that they must make it discoverable - it could be in AT, it might have something visual, there could also be a user setting that is like "I don't need those". <bkardell_> scotto: I think you or someone else suggested something like that. The big issue is that I do think there were will users who want to turn that off. If we can do that we can turn the other ones off too <bkardell_> scotto: I would still worry a bit about the combination of hover and focus - like especially with people who use voice dictation software, for example. Solving for lingering mouse hover is a hard problem I don't even want to add to this |
|
If folks are convinced that the lack of control over long-press is the blocker here, it's probably better to solve that before creating a higher level feature that solves some of the long-press issues in a particular way. [taps the sign] https://extensiblewebmanifesto.org/ |
|
To expand on the above: The limitations around long-press do not exist in native apps, but in those cases it seems like long-press is used as an enhanced context menu, so it's more like right click than hover. So, if there's a "big fence" in the way, and that is perceived to be the thing blocking developer innovation, just remove that big fence. Don't remove the fence and build a city on it. |
We've discussed this before but I don't think there's a dedicated issue to talk about what could constitute interest on a touch device (e.g. phone, tablet, certain laptops).
We should also discuss what potential issues there are with this and potential work arounds.
The text was updated successfully, but these errors were encountered: