Interop Lack of Transparency & Accountability

[mtom55]

Description

The voting process for Interop proposals is done in secret and each browser vendor involved can veto any feature without any transparency. Browser Vendors can sift through interop proposals and exclude items they believe are too difficult or don’t match their internal corporate goals. This is bad for the web and against the principle of openness and transparency that web standardization efforts rely on.

We propose that any votes or vetoes are made public with attribution, and any reasons for not proceeding with a particular feature is posted in the github issue along with that attribution. A summary page should be published for each future Interop with the complete set of information.

This lack of transparency has already been negatively received by developers and has been a source of mistrust and dissatisfaction with the interop process:

      "Fans of the spec bemoan lack of transparency in Interop 2024 process"
      Thomas Claburn - The Register

      "That's it? That's the response to what is, by several times, the most requested feature in Interop 2024?
      No explanation for rejecting the feature that got 4.5x more support than the runner-up? Really?"
      Tibet Tornaci

This problem is also discussed in this issue which discusses the lack of transparency of Interop.

      “My assumption and intention was that the process would be transparent, making per-proposal decisions by consensus,
      and documenting those decisions in meeting notes or equivalent. I made this assumption because almost everything in
      web standards and WPT is public by default, and the Interop 2022 positions were made public via GitHub issues.”
      Philip Jägenstedt

There are many reasons why keeping the Interop process for choosing which features to work on a secret is problematic. Here are the key issues:

1. Lack of Transparency

   Why it matters: Transparency is fundamental to maintaining trust in any process, especially when it's shaping the future of widely used technologies. If the interop voting process is kept secret, it becomes unclear who is influencing the decisions, how the priorities are set, and why certain features are prioritized over others.

   Impact: This lack of visibility can lead to suspicion or the perception that decisions are being made behind closed doors or based on undisclosed factors, such as personal or corporate interests.

2. Bias or Corporate Influence

   Why it matters: If the voting process is secret, there's a risk that powerful stakeholders could exert disproportionate influence over the feature selection, even if those features don't align with the broader needs of the web community.

   Impact: Smaller companies, independent developers, or underrepresented groups might not have their voices heard, which could skew development toward the interests of the most powerful players rather than benefiting the web ecosystem as a whole.

3. Accountability Issues

   Why it matters: Keeping votes secret removes accountability. It makes it difficult to trace back to the individuals or groups responsible for the decision. Without accountability, it’s harder to make improvements to the decision-making process.

   Impact: If features that are unpopular or less relevant to most developers are prioritized, it could delay or block the advancement of features that are more universally beneficial. Without accountability, it’s harder to rectify such issues.

4. Excluding Community Feedback

   Why it matters: Public visibility into the voting process allows community members to understand why certain features are chosen and provides them an opportunity to give feedback on the prioritization. Keeping the process secret means that community feedback may be excluded.

   Impact: This lack of engagement could result in features that don't align with real-world use cases, or miss critical input from people actually working with the web technologies on a day-to-day basis.

5. Inability to Audit the Process

   Why it matters: When the voting process is secret, it becomes very difficult to audit how decisions were made. For example, if a particular decision seems to have been made under unclear circumstances, there’s no way to verify if the process was fair and unbiased.

   Impact: An opaque process undermines confidence in the decision-making system and can erode the legitimacy of the project or initiative.

6. Potential for Conflicts of Interest

   Why it matters: Secret voting processes might mask potential conflicts of interest that could influence the decision-making. For example, a particular vendor or organization may be pushing for a feature that benefits their product or service but isn't necessarily the best choice for the web as a whole.

   Impact: This could lead to the prioritization of features that serve narrow interests rather than the broader needs of the web ecosystem, possibly undermining the long-term health and sustainability of the web.

7. Gaming the Process to Maintain High Interop Scores

   Why it matters: In a secret voting system, there’s a risk that browser vendors might strategically avoid committing to difficult or complex features in order to maintain or improve their interop score. This could happen if vendors prioritize "easy wins" or less challenging features that are more likely to pass the interop tests with minimal effort, rather than tackling more complex, nuanced features that could be harder to implement across all browsers.

   Impact: By gaming the process, vendors could prioritize features that are less impactful but more easily interoperable, while neglecting or delaying more important innovations that require significant effort or cross-vendor collaboration. This would ultimately result in a less innovative web, where difficult but valuable features are delayed or ignored in favor of easier, less meaningful improvements. Such behavior could also discourage long-term investment in true interoperability and lead to stagnation in web technology development.

Conclusion

Overall, secrecy in the voting process undermines many of the principles that made the open web what it is today—transparency & community involvement are critical.

A more open voting process ensures that decisions are made in the best interest of the broader developer and user community, helping to ensure that the web evolves in a way that is fair, inclusive, and sustainable.

Specification

Interop Team

Additional Signals

https://www.theregister.com/2024/02/03/jpeg_xl_interop_2024/

[o-t-w]

The Igalia podcast has a useful episode explaining the process https://podcasts.apple.com/gb/podcast/igalia-chats/id1533323588?i=1000647686844

[mtom55]

@nt1m why have you removed this issue and 891 from the list to be discussed for Interop 2025?

[nt1m]

This Github project is used purely for practical purposes, it only contains potential developer focus areas scored by WPTs. We copy this Github project into spreadsheets for evaluating developer proposals for focus areas, so it's not great for us having to trim down rows every time we run through the different spreadsheets.

#891 is an investigation (not a focus area), it's in fact one that we're already doing this year (see the Interop 2024 dashboard in the investigations and https://github.com/web-platform-tests/interop-mobile-testing). If you look at the list of issues, none of the investigations are automatically put in the Github project.

This issue isn't really a focus area in the classic sense either.

[mtom55]

This Github project is used purely for practical purposes, it only contains potential developer focus areas scored by WPTs. We copy this Github project into spreadsheets for evaluating developer proposals for focus areas, so it's not great for us having to trim down rows every time we run through the different spreadsheets.

#891 is an investigation (not a focus area), it's in fact one that we're already doing this year (see the Interop 2024 dashboard in the investigations and https://github.com/web-platform-tests/interop-mobile-testing). If you look at the list of issues, none of the investigations are automatically put in the Github project.

This issue isn't really a focus area in the classic sense either.

Fair enough, however, given the significant developer interest in both of these issues, particularly this transparency issue, It's reasonable for the web community to expect them to be discussed in detail as part of Interop 2025 and to see the detailed results of those discussions.

We're asking for the details of each voting round to be made publicly available along with reasons for vetos.

[lilnasy]

I am not involved with the interop people, I am just expressing my own opinion as a sole developer.

This is an interesting criticism. I found the interop process more opaque than tc39's as well, but after reading through this post, I see it as natural.

The motivation behind interop is to get browser vendors on the same page, so it would make sense why a veto exists: there is no interop with less than all the players. Developers are not going to adopt a feature before it's cross browser so it's natural for a browser vendor to not invest resources in implementing/improving it all by itself.

And yes, they may use the veto because of corporate interests, lack of resources, willingness to prioritize - but isn't that just business as usual? I may be wrong about the nature of the process but I don't think it's about governance or regulation. Instead, all parties participate of their own accord and due to their selfish and corporate motivations - including the developers. Browser vendors are hoping to get the best bang for their engineering buck, while developers are hoping to reduce the tech burden borne by their agencies.