Features Success: Easily collect qualitative feedback specific to a new feature

[liyiy]

Is your feature request related to a problem?

Qualitative user feedback is an important part of rolling out new features. We usually have to manually gather this feedback per new feature.

Describe the solution you'd like

Automate the feedback system when releasing new features so once you've started to roll out a feature flag, within the same feature flag, you can opt in with minimal setup (like pick which url to show the feedback box to and set up the questions you want to ask) to "collect feedback automatically" about it without having to implement or install anything new

Things to consider

Right now the PostHog ecosystem has a number of places where we're collecting feedback. For example, our feedback widgets and user interview apps, on our posthog.com website, and even a one-off NPS survey for correlation analysis on funnel insights. We've also had historical NPS survey components in the app that has now been removed.

The ideal solution here is to be able to re-use our feedback app and extend it out to the feature flags environment so that it's easier to get the feedback you need specific to your new feature.

Describe alternatives you've considered

In parallel to this work, it would be nice if we can get some more data points and research into how our customers like to gather qualitative feedback.

Thank you for your feature request – we love each and every one!

[liyiy]

Implementation details + design for a first iteration

Feedback box

• Limit the feature to client library posthog-js
  • Why? It's our biggest library, the feedback apps are already based in JS, visible new features are the easiest to get feedback on compared to backend
• Feedback box displays based on which page url/html identifier a user chooses AND if the feature flag is true for the user
  • Why? We want to get feedback on the specific feature that's being launched or tested
• Allow questions to be customizable (text based ones only) but limited to only 1-2
  • Why? Longer feedback in survey or form styles is outside of scope because it requires slightly different design and formatting
• NPS scoring ?
  • Why? Have some way to gather numerical feedback data to aggregate by
• Nice to have: basic color customization for feedback box.
  • Why? If it's easily implementable and helps users blend the feedback box to their site's design better, then we should do it~

Collected feedback display

• Basic table of responses
  • Why? We'll only need a table because we only want short responses for our first iteration
• Important to be able to aggregate and filter feedback based on user information
• Would be nice for posthog to be able to display which users "interact" with the feature the most and their corresponding feedback
• Ability to share or export this data

Answering @neilkakkar's questions

How do we reconcile different modes of gathering feedback? Is this only an us-problem, because posthog has all these different ways of gathering feedback, and others are fine, so we can disregard this issue? Or is it more prevalent?

This is hard to answer for users without doing more research into how our users collect their own feedback. This probably also varies widely based on what "type" of company you run, and for some companies it probably doesn't even make sense at all to collect user feedback in this way. For PostHog specifically, I think it's okay to have different ways of gathering feedback as long as we're intentional about it and that it's funneled to the same storage location. Which is neither the case right now. That should be a separate problem to address 😁 I am highlighting it to show that adding a qualitative feedback tool for new feature releases should not be used as a solution to that existing problem

What kind of targeting makes sense? I see you’ve mentioned URL-level. We talked about this, but would be good to mention why only URL level for now, and why not, for example, what the feedback app currently does, which is target on all URLs, but based on a flag.

Our feedback tools currently exist "site-wide", so they are not unique to a page or a component + the user, which you probably don't want if you're trying to test based on a new feature!

If it’s survey style feedback, we’d want to do aggregations that make sense of this data. Raw numbers per user wouldn’t be that useful here - does any insight type already support this? Can we build it as a custom insight via hogql?

Will skip surveys for now as we don't even use that internally ourselves, but eventually it would make a lot of sense to display charts and insights based on quantitative user feedback! Right now the biggest problem to solve is to make sure we can make the feedback collection process as painless as possible and as useful as possible since it's still qualitative data at the end of the day so we want quality vs quantity in responses

[annikaschmid]

Linking this issue here from way back when, it's about creating a user interview product. No actions at this point, just for reference. #2880