Add opt-in usage tracking for blocks added to post content #2140

nylen · 2017-08-01T22:27:26Z

This PR asks users for permission to enable usage tracking:

When the user clicks "Yes" or "No", the preference is stored using the existing WordPress method of saving user preferences (the setUserSetting function used elsewhere in wp-admin; props @swissspidy for telling us about this function in #1948). This preference is saved as a cookie and eventually persisted as a user option.

If the user clicks "Yes", tracking events will be recorded using existing WP.com infrastructure every time a block is added to the editor. The approach taken here is very similar to Calypso's event tracking code.

We can use the data added in this PR to inform various decisions such as default order for blocks and whether some blocks are less suitable for core, and more generally this is a very useful technique to collect user experience data.

nylen · 2017-08-01T22:28:07Z

components/button/index.js

+ isToggled,
+ className,
+ disabled,
+ ...additionalProps, // eslint-disable-line comma-dangle


Without the eslint-disable-line here, eslint reports this line as an error. This seems like a bug.

Found quite a bit of prior discussion, which consensus seeming as though the specification does not allow a trailing comma in these cases:

comma-dangle always-multiline incorrectly reports error for object rest spread eslint/eslint#7463

Breaking change in eslint@3.8.0 with babel-eslint parser (comma-dangle and rest object destructuring) eslint/eslint#7422

comma-dangle incorrectly requires commas after rest properties eslint/eslint#7297

Fix: Don't require commas after rest properties (fixes #7297) eslint/eslint#7298

eslint/eslint@52da71e

Version 3.8.0 comma-dangle breaks Flowtype annotations when a object rest parameter is used eslint/eslint#7370 (comment)

I guess that sort of makes sense, though it would be nice to have a better error message. I'll incorporate this change into #2205.

nylen · 2017-08-01T22:28:47Z

editor/enable-tracking-prompt/index.js

+
+export default connect(
+ undefined,
+ { dispatchRemoveNotice: removeNotice }


I'm renaming this variable here because otherwise eslint reports an error under the no-shadow rule.

An alternative (used elsewhere) would be to avoid destructuring: props.removeNotice

codecov · 2017-08-01T22:35:57Z

Codecov Report

Merging #2140 into master will increase coverage by 0.68%.
The diff coverage is 75.6%.

@@            Coverage Diff             @@
##           master    #2140      +/-   ##
==========================================
+ Coverage    22.3%   22.98%   +0.68%     
==========================================
  Files         137      141       +4     
  Lines        4291     4415     +124     
  Branches      722      743      +21     
==========================================
+ Hits          957     1015      +58     
- Misses       2815     2874      +59     
- Partials      519      526       +7

Impacted Files	Coverage Δ
editor/index.js	`0% <0%> (ø)`	⬆️
editor/inserter/index.js	`0% <0%> (ø)`	⬆️
editor/modes/visual-editor/block-list.js	`0% <0%> (ø)`	⬆️
editor/enable-tracking-prompt/index.js	`100% <100%> (ø)`
components/button/index.js	`90.9% <100%> (ø)`	⬆️
editor/actions.js	`39.28% <100%> (+2.24%)`	⬆️
editor/utils/tracking.js	`91.3% <91.3%> (ø)`
blocks/library/button/index.js	`26.66% <0%> (ø)`	⬆️
editor/sidebar/table-of-contents/index.js	`0% <0%> (ø)`	⬆️
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a1b1886...d10b59c. Read the comment docs.

nylen · 2017-08-01T22:41:13Z

To test this PR, you can use the following commands in the browser console:

window.getUserSetting( 'gutenberg_tracking' ) - Returns the current value of the gutenberg_tracking user setting. Should be '' if unset or deleted; on immediately after clicking "Yes" and on subsequent page loads; and off immediately after clicking "No" and on subsequent page loads.
window.setUserSetting( 'gutenberg_tracking', '', true ); - Deletes the saved value of the gutenberg_tracking user setting. Should immediately stop all tracking events/messages and should cause the prompt to be shown again on the next page load.

youknowriad · 2017-08-02T10:25:03Z

editor/utils/tracking.js

@@ -0,0 +1,61 @@
+/* eslint-disable no-console */


Maybe the tracking could be its own module, it could be useful outside of the editor (and other WP feature plugins later)

Do we have a path forward for making this work? (Working build system, publishing to npm)

I was thinking, just moving this to a root folder in gutenberg for now. And once we have a WP.org infrastructure we could move to the packages repository

I don't think this belongs in a new root folder. Maybe as part of utils, but the prompt component should stay in editor.

Then later we can move it over to https://github.com/WordPress/packages (I thought that was what you meant).

I don't think this belongs in a new root folder. Maybe as part of utils

For me a root folder is a separate module just like the packages repo, and I don't think we should add a lot of things to the utils module, I prefer smaller separate modules (like the ongoing PR for wordCounter)

We should avoid adding a lot of small packages in this repo. This is what WordPress/packages is designed for; I'm fine with moving stats tracking over there once we get the build system working well. We will probably want to make it a bit more generic (remove the gutenberg_ prefix and modify the opt-in logic).

youknowriad · 2017-08-02T10:26:20Z

editor/enable-tracking-prompt/index.js

+
+export default connect(
+ undefined,
+ { dispatchRemoveNotice: removeNotice }


An alternative (used elsewhere) would be to avoid destructuring: props.removeNotice

youknowriad · 2017-08-02T10:27:38Z

Longer term (or not), WP.org needs its own tracking infrastructure, this could be very useful to enhance WordPress.

nylen · 2017-08-03T12:52:37Z

I plan to rebase and merge this later today. I expect it will provide a lot of good data that we can use to inform decisions.

Making the data available to the public is also doable, but will require some separate work on the WP.com side.

swissspidy · 2017-08-03T12:54:24Z

editor/utils/tracking.js

+ + '=' + encodeURIComponent( name )
+ + '&t=' + Math.random();
+
+ if ( process.env.NODE_ENV === 'development' ) {


Seems like this should be unified with the process.env.NODE_ENV !== 'production' check below.

Especially since the console message says "non-production build".

Right, good point.

I wrote it this way so the URL-building logic can be unit-tested (NODE_ENV === 'test'). However, we only want to display the URL message in development. I'll add those tests and change the message to say development build as well.

aduth · 2017-08-03T13:03:34Z

editor/enable-tracking-prompt/index.js

+/**
+ * WordPress dependencies
+ */
+import { __ } from 'i18n';


Note with #2172, these need to be updated to prefix the dependencies with @wordpress/. You will need to perform a rebase against the latest version of master and apply your changes:

git fetch origin git rebase origin/master

nylen · 2017-08-03T15:59:14Z

editor/utils/test/tracking.js

+ // There are a couple of pieces of the URL where we don't care about
+ // testing the specific value. Replace them with placeholders.
+ const urlMatch = url
+ .replace( /^[a-z]+:/, 'PROTOCOL:' )


In actual usage this will either be http: or https:. During test suite runs it is about:. This is difficult to change (jsdom/jsdom#1700) and it doesn't seem worth testing for a specific value.

Alternatively, we could run assertions against individual pieces from url.parse.

I've incorporated this change into #2205.

aduth · 2017-08-04T02:42:08Z

editor/enable-tracking-prompt/test/index.js

+/**
+ * External dependencies
+ */
+import { mount } from 'enzyme';


In this case I think we could have used shallow rendering which tends to be more performant (doesn't mount into DOM, doesn't render descendants).

shallow doesn't work here because we reach inside a Button element and grab its text. Open to other suggestions.

shallow doesn't work here because we reach inside a Button element and grab its text. Open to other suggestions.

Remarked at #2140 (comment). It wasn't entirely as simple, but close:

diff --git a/editor/enable-tracking-prompt/test/index.js b/editor/enable-tracking-prompt/test/index.js index 686fa0ec..098b95e0 100644 --- a/editor/enable-tracking-prompt/test/index.js +++ b/editor/enable-tracking-prompt/test/index.js @@ -1,7 +1,7 @@ /** * External dependencies */ -import { mount } from 'enzyme'; +import { shallow } from 'enzyme'; /** * Internal dependencies @@ -29,13 +29,13 @@ describe( 'EnableTrackingPrompt', () => { } ); it( 'should render a prompt with Yes and No buttons', () => { - const prompt = mount( + const prompt = shallow( <EnableTrackingPrompt /> ); - const buttons = prompt.find( '.button' ); + const buttons = prompt.find( 'Button' ); expect( buttons.length ).toBe( 2 ); - expect( buttons.at( 0 ).text() ).toBe( 'Yes' ); - expect( buttons.at( 1 ).text() ).toBe( 'No' ); + expect( buttons.at( 0 ).children().text() ).toBe( 'Yes' ); + expect( buttons.at( 1 ).children().text() ).toBe( 'No' ); expect( window.setUserSetting ) .not.toHaveBeenCalled(); @@ -46,11 +46,11 @@ describe( 'EnableTrackingPrompt', () => { } ); it( 'should enable tracking when clicking Yes', () => { - const prompt = mount( + const prompt = shallow( <EnableTrackingPrompt removeNotice={ removeNotice } /> ); - const buttonYes = prompt.find( '.button' ) - .filterWhere( node => node.text() === 'Yes' ); + const buttonYes = prompt.find( 'Button' ) + .filterWhere( node => node.children().text() === 'Yes' ); buttonYes.simulate( 'click' ); expect( window.setUserSetting ) @@ -62,11 +62,11 @@ describe( 'EnableTrackingPrompt', () => { } ); it( 'should disable tracking when clicking No', () => { - const prompt = mount( + const prompt = shallow( <EnableTrackingPrompt removeNotice={ removeNotice } /> ); - const buttonNo = prompt.find( '.button' ) - .filterWhere( node => node.text() === 'No' ); + const buttonNo = prompt.find( 'Button' ) + .filterWhere( node => node.children().text() === 'No' ); buttonNo.simulate( 'click' ); expect( window.setUserSetting )

I am still having problems getting this to work, specifically with the it( 'should show and hide a popover when clicking More info' ) test not covered in the above diff. See 861218d.

aduth · 2017-08-04T02:43:03Z

editor/enable-tracking-prompt/test/index.js

+ const prompt = mount(
+ <EnableTrackingPrompt removeNotice={ removeNotice } />
+ );
+ const buttonNo = prompt.find( '.button' )


To point above, this selector would not work with shallow rendering, but then again, it is not the place of the EnableTrackingPrompt to know that an element of class .button is rendered by the Button component. Replacing this with 'Button' should work just as well.

I found that we couldn't look inside a shallow-rendered Button element to find its text:

console.log( { text: prompt.find( 'Button' ).at( 0 ).text() } ); { text: '<Button />' }

I've incorporated the change to Button instead of .button into #2205.

aduth · 2017-08-04T02:46:16Z

editor/utils/test/tracking.js

+ // There are a couple of pieces of the URL where we don't care about
+ // testing the specific value. Replace them with placeholders.
+ const urlMatch = url
+ .replace( /^[a-z]+:/, 'PROTOCOL:' )


Alternatively, we could run assertions against individual pieces from url.parse.

#2140 (comment)

nylen commented Aug 1, 2017

View reviewed changes

nylen added this to the Beta 0.7.0 milestone Aug 1, 2017

youknowriad reviewed Aug 2, 2017

View reviewed changes

youknowriad mentioned this pull request Aug 2, 2017

Collect stats for block usage and interaction #1230

Closed

swissspidy reviewed Aug 3, 2017

View reviewed changes

aduth reviewed Aug 3, 2017

View reviewed changes

nylen added 4 commits August 3, 2017 09:46

Add mechanism for opt-in usage tracking

441b0de

Track which blocks are added to post content

2cab1fa

Use a different way of avoiding eslint warning

b725515

Update WordPress dependencies after #2172

16b882b

nylen force-pushed the add/opt-in-usage-tracking branch from 7f4d5be to 16b882b Compare August 3, 2017 13:47

nylen added 2 commits August 3, 2017 09:50

Update stats URL message

32d3e31

Add unit tests

88d24cd

nylen commented Aug 3, 2017

View reviewed changes

nylen added 5 commits August 3, 2017 13:44

Clarify tracking prompt to indicate that post content is not collected

d12951a

Add tests for new Button features

07755f6

Move notice ID into a reusable constant

e0b18e2

Add tests for EnableTrackingPrompt component

be2bd49

Track usage data opt-in events

d10b59c

nylen merged commit 51466b2 into master Aug 3, 2017

nylen deleted the add/opt-in-usage-tracking branch August 3, 2017 19:02

nylen added a commit that referenced this pull request Aug 3, 2017

Design tweaks to tracking opt-in prompt (#2140)

085088a

nylen added a commit that referenced this pull request Aug 3, 2017

Another design tweak to tracking opt-in prompt (#2140)

1b322e9

nylen mentioned this pull request Aug 3, 2017

Track stats for automatic block conversion #2205

Merged

aduth reviewed Aug 4, 2017

View reviewed changes

afercia mentioned this pull request Aug 4, 2017

Opt-in usage tracking notice improvements #2215

Closed

nylen added a commit that referenced this pull request Aug 4, 2017

Remove eslint exception (per spec, this is a syntax error)

ccf3906

#2140 (comment)

nylen added a commit that referenced this pull request Aug 4, 2017

Test bumpStat URL via url.parse

1c179a2

#2140 (comment)

nylen mentioned this pull request Aug 7, 2017

Remove stats tracking code #2278

Merged

4 tasks

swissspidy mentioned this pull request Aug 21, 2017

Add user preference api #1948

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add opt-in usage tracking for blocks added to post content #2140

Add opt-in usage tracking for blocks added to post content #2140

nylen commented Aug 1, 2017

nylen Aug 1, 2017

aduth Aug 4, 2017

nylen Aug 4, 2017

nylen Aug 1, 2017

youknowriad Aug 2, 2017

codecov bot commented Aug 1, 2017 •

edited

Loading

nylen commented Aug 1, 2017

youknowriad Aug 2, 2017

nylen Aug 2, 2017

youknowriad Aug 2, 2017

nylen Aug 2, 2017

nylen Aug 2, 2017

youknowriad Aug 2, 2017

nylen Aug 3, 2017

youknowriad Aug 2, 2017

youknowriad commented Aug 2, 2017

nylen commented Aug 3, 2017

swissspidy Aug 3, 2017

nylen Aug 3, 2017

aduth Aug 3, 2017

nylen Aug 3, 2017

aduth Aug 4, 2017

nylen Aug 4, 2017

aduth Aug 4, 2017

nylen Aug 4, 2017

aduth Aug 4, 2017

nylen Aug 7, 2017

aduth Aug 4, 2017

nylen Aug 4, 2017

nylen Aug 4, 2017

aduth Aug 4, 2017

Add opt-in usage tracking for blocks added to post content #2140

Add opt-in usage tracking for blocks added to post content #2140

Conversation

nylen commented Aug 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Aug 1, 2017 • edited Loading

Codecov Report

nylen commented Aug 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

youknowriad commented Aug 2, 2017

nylen commented Aug 3, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Aug 1, 2017 •

edited

Loading