Which metrics should we track?

[iteles]

Moved these questions from @nelsonic to their own issue from #5 (comment) so that we are able to discuss each topic separately:

Todo

• Get access to GA for the existing site/MVP (@jussi-club-soda to provide via Email. please/thanks!)
• Review existing stats/data for pageviews and any "completed actions"
  • summarise the data in a "report" format for others to read.
    Note: this does not need to be more than a couple of pages. And should be done as an .md file with screenshots and a few paragraphs of copy/discussion/summary.
    • The reason we need to summarise existing metrics is as a "baseline" everyone can understand. So that we all know what we are (need to be) focussing on in the project.
      If the existing metrics are meaningless, just write that.
      If nobody checked the GA metrics on a daily (or even hourly) basis, write that. (and Why...)
• Give Recommendations on which metrics should be tracked for each Persona and Content Type
  Specifically:
• What completed action would constitute "success" for each Content Type What are the primary content types? #94 and User Persona Who are the users of the website? > Create Personas #93 viewing/interacting with that Content. (example below)

For example:
The Venue (Place) Content Type will be rendered as a "place page".
Tracking "page views" for the venue page is a good baseline metric
however it's "crude" and does not inform us what the "success" of the page is.
Questions we need to ask (and thus collect metrics on) include:
Did the user (Consumer) viewing the place page:

• Scroll down the page to view more detail about the venue?
  • What content was "in view" when they stopped scrolling? i.e. what did they interact with (read)?
• Click "view all drinks" (for this venue) link?
• Click on the Map ("pin") icon or address copy in order to find the venue on the Map?
• Re-visit the page after a set amount of time? (i.e. returning visitor?)

We need to be proactive about describing the specific metrics we want to track for each Content Type and Person interacting with that content in order to determine "success".
Only by actively tracking the "action completion" metrics will we
(the Product/Project Owner and Developers) be able to know that what we are building
is "what people want". i.e. is it helping the people (Persona) to achieve their goal?

In order to track the Consumer "User Journey", we need to store a "fingerprint" of the device
on the server -> see/example: https://github.com/dwyl/hits#implementation-detail
We can then lookup that device in the Log when they return to the application
and continue to track their "progress" in their user journey.
If they had previously viewed a Venue and then closed their web browser,
but then returned after an amount of time e.g. 3h (exact time elapsed TBD)
we can display a new piece of UI to them asking:
"did you visit this venue? would you like to rate it?" (also copy TBD)
And display a simple ("low friction") rating input so they can easily rate the venue.
We should expand on the Ratings/Reviews content types and UX in: #95
but our reasoning for everything we do on this project needs to be:
Does this help the user (Persona) to achieve their goal?

Questions we need to Answer:

If we can answer the following questions (we will have a much greater chance of success):

What metrics do ....

• Consumer need in order to modify their behaviour and achieve their goal?
  (and thus find happiness and fulfilment in life)

• Venue Managers need to improve their page?
  (and thus make more money)

• Brand Managers want to see about their own (or their competitors) products?
  (and thus sell more product and get a bonus)

• What can we do to make the whole team more "Data Driven"?
  (so that we can decide how to invest our time/efforts into refining/building
  the right features/UX to help the users achieve their goals)

---

5. Metrics?

5. Which Metrics to Track? Page Views vs. Engagement and Action?

Is the objective of the Website to get more page views? or to help people change their behaviour?
Viewing the page is the first step, but it's not in itself a good metric to focus on.

A much better metric would be to track "completed actions" of the user journey through the website. e.g:

• a prospective mindful drinker (MD) "lands" on a specific product ("landing") page after searching for a specific beverage on Google: "Peach Bellini Mocktail" (insert name of real drink here!)
• The website detects that the MD is on a mobile device and asks: "would you like to find a venue near you that stocks this drink?"
  • Request access to device location http://diveinto.html5doctor.com/geolocation.html
  • Show postcode lookup input. (Dan has built this for TC before)
• Display list of venues which stock/serve this drink.
  • Bonus Level: show [view on map] link next to each venue in list.
• User clicks on [view on map] link
• User visits venue
• User leaves a review for the product and/or venue

Do we have stats for the annual expenditure of mindful (or non-alcoholic) drink consumers? e.g:
https://www.statista.com/statistics/597628/average-total-spend-per-year-in-pubs-uk/

What are the "biggest concerns" for the Pubs? (what keeps them "up at night"?)
And how can we help them to be successful instead of going out of business?
https://www.bbc.com/news/business-43492043

Note: I'm not suggesting that "success" for a pub is selling people more alcohol.

We should seek to understand the following metrics:

• Who are the most loyal customers ("regulars") and do they drink alcohol more often than non-alcoholic drinks?
  • Are they tracking the increase/decline in these regulars?
• Who are the new customers driving new demand/growth?
• What is the percentage of sales of Alcoholic drinks, Non-Alcoholic drinks, food (meals) and snacks?
  • What is the profitability of each product category?
  • How are these product categories growing/shrinking?
• Do people drinking non-alcoholic drinks spend more or less on snacks/food in the venue?

Feedback Loop?

• How many of the various types of users are signed up to receive the CS newsletter?
• How many of the users are regular/repeat visitors to the website?
  • What is the percentage of people who visit once and do not click on any call-to-action?
  • How many visits per month is the average?
• Do users give feedback on products which can be "re-packaged" and passed on to the brands?

Note: this is a non-exhaustive list; it's a "starting point" of the questions we should be seeking clarity on so that we have a "SMART" goal for the project (beyond "build a CMS website").

[jussi-club-soda]

Objectives of the site:

1. nudge venues to be better (stock more low/no alcohol drinks, display and serve them better, train their staff about them, treat all customers well)
2. provide information to consumers about venues and drinks; and direct them to buy drinks from our partner online shops (DryDrinker, WiseBartender, Amazon) to provide Club Soda with revenue
3. provide drink brands with a marketing opportunity

Any usage is good news. For venues, clicks on drinks and where to buy them means they are searching for the right information (and hopefully taking action based on it, but we can't track things outside of the site). For consumers, searching and browsing means they are finding information. So journeys are useful to track.

We have a bunch of statistics here (sales in terms of ££ are not usually public information and often hard to find): https://mindfuldrinkingfestival.com/media-stats/

For pubs, money is the number 1 relevant concern for us (they also have many other worries but we can't keep UK in the EU with this website...). Any publicity = more customers = more sales that we can get them is good news. We don't have any of the metrics Nelson lists above on venue sales etc. They will vary form venue to venue. But we do have a decent sales pitch to them on why they should do better on low/no drinks (our ongoing research project will hopefully gives us more here, but this type of data is really hard to get hold of...)

Feedback loop:
Club Soda individual members: over 20k in total, some on our website, some on Facebook, some on MeetUp, some on two or three platforms.
Venues listed on clubsodaguide.com now: 302 (maybe 30 of them not signed up, just a listing created by Club Soda)
Drinks: 162 brands and 452 products on our Airtable database; 500 people on our email list.

Would it be useful to give you access to our Google Analytics?

[nelsonic]

@jussi-club-soda yes, access to GA would be useful for us to establish baseline key metrics.
Then we should regularly review them once a new version of the site/app is launched.
Thanks.

[nelsonic]

@Cleop I have updated the issue description above with a "Todo List".
We need @jussi-club-soda's help with getting access to the existing GA data.
Familiarising ourselves with and summarising the existing metrics should (ideally) not take more than a couple of hours.
Once that is done, we need to review each of the (3) primary content types #94
and determine what the Metrics that we need to collect are for each one.
Specifically what constitutes a "goal achievement" for the person "consuming" that content?

Armed with that insight we will be in a much better position to ensure that we are capturing the right data for each feature we build and thus helping the team (and stakeholders) to be more "Data Driven".

Please let me know if you have any questions (by posting comments on this issue).
Thanks! ✨

[jussi-club-soda]

@nelsonic send me the emails and I'll get you access to our GA!
I'm at jussi@joinclubsoda.co.uk

[iteles]

@Cleop @Danwhy @nelsonic You should all now have access to GA, but please ignore the latest two months of data as there has been a problem with these!

Thanks @jussi-club-soda 😊

[Cleop]

Metrics for the 3 primary content types:

Venues

Basic metric: venue page views
Goal: Reach the venue page and then to visit the venue.
Goal metric:

• Check the funnel to the venue page to see if there are any pages with a large drop off.
• As we can't measure whether the user does actually visit the venue, it would be valuable to use interactions with the venue page as an indicator of interest. E.g. scrolling activity, clicks on the venue website/phone number/get directions, drinks etc.

Drinks

Basic metric: drinks page views
Goal: reach a drinks page and then buy the drink from a partner website.
Goal metric:

• Check the funnel to the drinks page to see if there are any pages with a large drop off.
• Clicks on one of the 'Where to Buy' links.

Goal: reaches a drinks page and then goes and tries that drink at a venue.
Goal metric:

• Check the funnel to the drinks page to see if there are any pages with a large drop off.
• Clicks from a drinks page to a venue that serves the drink (followed by the measures on the venue page for engagement as above).

People (User personas)

Bradley, Drinks Brand Manager

Goal: evidence that it's worth paying for a premium brand listing
Goal metric:

• Page view duration and click events on premium listings vs free.

Goal: to know more about the users that are interested in my product
Goal metric:

• Geographic and demographic data of users who view the page

Vicky, Venue Manager

Goal: to be able to simply upload my venue to the site
Goal metric:

• Check the funnel of venue sign up process to see if there are any pages with a large drop off.

Goal: to know which non / low alcohol drinks are the most popular so they know what to start stocking
Goal metric:

• Most visited drinks pages

Goal: to know if a lot of people in my area have visited CS so I know if the low / no alcohol trend may be present in my area
Goal metric:

• Area search terms in the search functionality

Nisha, Consumer

Goal: to buy some non / low alcoholic drinks for an upcoming party
Goal metric:

• Check the funnel to the drinks page to see if there are any pages with a large drop off.
• Clicks on one of the 'Where to Buy' links.

Goal: to be able to recommend a venue serving great non alcoholic drinks to go to with my friends after work
Basic metric: venue page views
Goal metric:

• Check the funnel to the venue page to see if there are any pages with a large drop off.
• As we can't measure whether the user does actually visit the venue, it would be valuable to use interactions with the venue page as an indicator of interest. E.g. scrolling activity, clicks on the venue website/phone number/get directions, drinks etc.

Jussi, Club Soda

Goal: for the website to build a sustainable community of people who use it
Goal metric:

• Retention and community engagement levels ie. signing up venues conversion and reviews (once implemented)

Some of these metrics cross over between content type and personas so have been repeated

[nelsonic]

Traffic (Customer Acquisition) Sources

The success of the project/product depends on our ability to:

• Track the in-bound traffic sources

Thankfully this is done reasonably well by Google Analytics (GA), see below.

• Monitor the effectiveness of each source in "converting" a specific type of user.
  • Bounce Rate?
  • Conversion Rate ?

In-bound traffic source metrics are essential for understanding
the effectiveness of customer acquisition "strategy".

Our goal in reviewing, summarising and analysing this data is to answer the question:

Where Should CS Focus Customer Acquisition Efforts?

The biggest segment (42%) of in-bound traffic (people visiting) to: https://clubsodaguide.com
is from referrals: (links from other websites)


Referrals are also the lowest "Bounce Rate":

Which makes sense given that they are usually the highest relevance
as people are intentionally clicking on a link from a page that itself had relevant content,
i.e. people are clicking a link in order to "read more".
Even so they are "bouncing" around 50% of the time when arriving from joinclubsoda.co.uk ...

Of the referral traffic, 77% is from joinclubsoda.co.uk and 7.7% is mindfuldrinkingfestival.com

The union of the two top sources is (just under) 85%:

The next 6% of traffic is from "SEO Service" and has a 100% bounce rate:

So it's not relevant as a traffic source and does not warrant further analysis (at this stage!)

In-bound Traffic from "External" Sources (Not "Controlled" by CS)

The next sources from real (external) websites are:

• nudgingpubs.uk (23 users. 0.94%)
• girlandtonic.co.uk (17 users. 0.73%)
  

http://nudgingpubs.uk

Nudging Pubs appears to be a purpose-built "link back" site (owned/operated by CS):

The content appears to be good, and referrals have a low bounce rate.

https://girlandtonic.co.uk

A blog by Laurie Mcallister, Freelance Writer & Blogger (and "influencer")
Working to make not drinking cool:

https://twitter.com/LVMcAllister

[jussi-club-soda]

Just to note that since our pilot we have not made much of an effort to publicize the site - it gets mentioned by us on our own websites so I'm not surprised most of the referral traffic comes from us ourselves. This should change once we relaunch to the public in January 2019!

[nelsonic]

@jussi-club-soda thanks for confirming. 👍
I just wanted to summarise the existing traffic sources analytics data so that we have a reference.

[jessyclubsoda]

How do we go about implementing some of the metrics that @Cleop suggested here?