Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add telemetry to companion #1117

Merged
merged 32 commits into from
Jan 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
dbde5bd
feat: add types for state
SgtPooki Dec 16, 2022
3e3dc16
tmp
SgtPooki Dec 16, 2022
829283e
feat: ipfs-companion tracks views and sessions
SgtPooki Dec 17, 2022
2ab0bbe
Update add-on/src/lib/telemetry.js
SgtPooki Jan 12, 2023
1934425
Update add-on/src/lib/telemetry.js
SgtPooki Jan 12, 2023
faf72a8
Update add-on/src/lib/telemetry.js
SgtPooki Jan 12, 2023
3b83c17
Update add-on/_locales/en/messages.json
SgtPooki Jan 12, 2023
f6b6081
Update add-on/_locales/en/messages.json
SgtPooki Jan 12, 2023
e372e74
Update add-on/_locales/en/messages.json
SgtPooki Jan 12, 2023
11d3b9b
Update add-on/_locales/en/messages.json
SgtPooki Jan 12, 2023
4a0af8f
chore: fix options and state typings
SgtPooki Jan 12, 2023
a256e05
chore: use debug logger
SgtPooki Jan 12, 2023
798c5fe
fix(lint): remove unused method
SgtPooki Jan 12, 2023
8d4d6c8
fix(lint): run 'npm run fix:lint'
SgtPooki Jan 12, 2023
4058da2
chore: build and lint success
SgtPooki Jan 12, 2023
0e66ea9
chore(types): fix type errors
SgtPooki Jan 12, 2023
090d700
chore: add docs/telemetry/COLLECTED_DATA.md
SgtPooki Jan 12, 2023
d83de1a
chore: update old metric group names in logConsent
SgtPooki Jan 12, 2023
ceb65b6
chore: clean up UI
SgtPooki Jan 13, 2023
9db9b6e
chore: use ignite-metrics from npm
SgtPooki Jan 13, 2023
15ab207
chore: update ignite-metrics and some types
SgtPooki Jan 14, 2023
fa915c3
fix(tests): tests dont fail on countly-sdk-web import
SgtPooki Jan 18, 2023
c19ba88
fix: build
SgtPooki Jan 18, 2023
d334ec0
Merge branch 'main' into 1115-feat-getting-basic-metrics-in-ipfs-comp…
SgtPooki Jan 18, 2023
162aaa8
Merge branch 'main' into 1115-feat-getting-basic-metrics-in-ipfs-comp…
SgtPooki Jan 25, 2023
089b0d0
chore: temporarily use updated ignite-metrics
SgtPooki Jan 20, 2023
ee060c2
chore: use deployed ignite-metrics version
SgtPooki Jan 20, 2023
ea798df
chore: address PR comments
SgtPooki Jan 27, 2023
dad1bfa
chore: pin ignite-metrics dependency
SgtPooki Jan 27, 2023
d2f9cab
chore(lint): fix lint errors
SgtPooki Jan 27, 2023
0810499
fix: use browser.runtime.sendMessage
SgtPooki Jan 27, 2023
7160856
Merge branch 'main' into 1115-feat-getting-basic-metrics-in-ipfs-comp…
SgtPooki Jan 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,6 @@
/coverage
/.nyc_output
/add-on/manifest.json

.DS_Store
.vscode
40 changes: 40 additions & 0 deletions add-on/_locales/en/messages.json
Original file line number Diff line number Diff line change
Expand Up @@ -710,5 +710,45 @@
"page_landingWelcome_projects_title": {
"message": "Related Projects",
"description": "Projects section title (page_landingWelcome_projects_title)"
},
"option_header_telemetry": {
"message": "Telemetry",
"description": "A section header on the Preferences screen (option_header_telemetry)"
},
"option_telemetry_disclaimer": {
"message": "We're collecting minimal telemetry data to improve and prioritize our work. Please consent to the collection of these metrics to assist in our efforts!",
"description": "Disclaimer about telemetry collection in the telemetry section on the Preferences screen (option_telemetry_disclaimer)"
},
"option_telemetryGroupMinimal_title": {
"message": "Feature Telemetry",
"description": "A title for the 'minimal' grouping of metrics we collect (option_telemetryGroupMinimal_title)"
},
"option_telemetryGroupMinimal_description": {
"message": "Collect basic feature and usage metrics to help maintainers to prioritize work on the most useful features.",
"description": "A description for the 'minimal' grouping of metrics we collect (option_telemetryGroupMinimal_description)"
},
"option_telemetryGroupMarketing_title": {
"message": "Marketing title",
"description": "A title for the 'marketing' grouping of metrics we collect (option_telemetryGroupMarketing_title)"
},
"option_telemetryGroupMarketing_description": {
"message": "Marketing description",
"description": "A description for the 'marketing' grouping of metrics we collect (option_telemetryGroupMarketing_description)"
},
"option_telemetryGroupPerformance_title": {
"message": "Performance title",
"description": "A title for the 'performance' grouping of metrics we collect (option_telemetryGroupPerformance_title)"
},
"option_telemetryGroupPerformance_description": {
"message": "Performance description",
"description": "A description for the 'performance' grouping of metrics we collect (option_telemetryGroupPerformance_description)"
},
"option_telemetryGroupTracking_title": {
"message": "Tracking title",
"description": "A title for the 'tracking' grouping of metrics we collect (option_telemetryGroupTracking_title)"
},
"option_telemetryGroupTracking_description": {
"message": "Tracking description",
"description": "A description for the 'tracking' grouping of metrics we collect (option_telemetryGroupTracking_description)"
Comment on lines +730 to +752
Copy link
Member Author

@SgtPooki SgtPooki Jan 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SgtPooki to address

Need to remove these since we wont be using them for now

}
}
1 change: 1 addition & 0 deletions add-on/src/background/background.html
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
<meta charset="utf-8">
<script src="/dist/bundles/ipfs.bundle.js"></script>
<script src="/dist/bundles/backgroundPage.bundle.js"></script>
<body></body>
lidel marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 2 additions & 0 deletions add-on/src/background/background.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,14 @@ browser.runtime.setUninstallURL(getUninstallURL(browser))

// init add-on after all libs are loaded
document.addEventListener('DOMContentLoaded', async () => {
browser.runtime.sendMessage({ telemetry: { trackView: 'background' } })
// setting debug namespaces require page reload to get applied
const debugNs = (await browser.storage.local.get({ logNamespaces: optionDefaults.logNamespaces })).logNamespaces
if (debugNs !== localStorage.debug) {
localStorage.debug = debugNs
window.location.reload()
}
// init inlined to read updated localStorage.debug
// @ts-expect-error - TS does not know about window.ipfsCompanion
window.ipfsCompanion = await createIpfsCompanion()
})
2 changes: 1 addition & 1 deletion add-on/src/landing-pages/welcome/page.js
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ const renderCompanionLogo = (i18n, isIpfsOnline) => {

return html`
<div class="mt4 mb2 flex flex-column justify-center items-center transition-all ${stateUnknown && 'state-unknown'}">
${logo({ path: logoPath, size: logoSize, isIpfsOnline: isIpfsOnline })}
${logo({ path: logoPath, size: logoSize, isIpfsOnline })}
lidel marked this conversation as resolved.
Show resolved Hide resolved
<p class="montserrat mt3 mb0 f2">${i18n.getMessage('page_landingWelcome_logo_title')}</p>
</div>
`
Expand Down
1 change: 1 addition & 0 deletions add-on/src/landing-pages/welcome/store.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ export default function createWelcomePageStore (i18n, runtime) {
state.webuiRootUrl = null
let port
emitter.on('DOMContentLoaded', async () => {
browser.runtime.sendMessage({ telemetry: { trackView: 'welcome' } })
emitter.emit('render')
port = runtime.connect({ name: 'browser-action-port' })
port.onMessage.addListener(async (message) => {
Expand Down
66 changes: 33 additions & 33 deletions add-on/src/lib/ipfs-client/brave.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,39 +16,39 @@ const waitFor = (f, t) => pWaitFor(f, { interval: tickMs, timeout: t || Infinity
// wrapper for chrome.ipfs.* that gets us closer to ergonomics of promise-based browser.*
export const brave = hasBraveChromeIpfs()
? Object.freeze({
// This is the main check - returns true only in Brave and only when
// feature flag is enabled brave://flags and can be used for high level UI
// decisions such as showing custom node type on Preferences
getIPFSEnabled: async () =>
Boolean(await promisifyBraveCheck(chrome.ipfs.getIPFSEnabled)),

// Obtains a string representation of the resolve method
// method is one of the following strings:
// "ask" uses a gateway but also prompts them to install a local node
// "gateway" uses a gateway but also prompts them to install a local node
// "local" uses a gateway but also prompts them to install a local node
// "disabled" disabled by the user
// "undefined" everything else (IPFS feature flag is not enabled, error etc)
getResolveMethodType: async () =>
String(await promisifyBraveCheck(chrome.ipfs.getResolveMethodType)),

// Obtains the config contents of the local IPFS node
// Returns undefined if missing for any reason
getConfig: async () =>
await promisifyBraveCheck(chrome.ipfs.getConfig),

// Returns true if binary is present
getExecutableAvailable: async () =>
Boolean(await promisifyBraveCheck(chrome.ipfs.getExecutableAvailable)),

// Attempts to start the daemon and returns true if finished
launch: async () =>
Boolean(await promisifyBraveCheck(chrome.ipfs.launch)),

// Attempts to stop the daemon and returns true if finished
shutdown: async () =>
Boolean(await promisifyBraveCheck(chrome.ipfs.shutdown))
})
// This is the main check - returns true only in Brave and only when
// feature flag is enabled brave://flags and can be used for high level UI
// decisions such as showing custom node type on Preferences
getIPFSEnabled: async () =>
Boolean(await promisifyBraveCheck(chrome.ipfs.getIPFSEnabled)),

// Obtains a string representation of the resolve method
// method is one of the following strings:
// "ask" uses a gateway but also prompts them to install a local node
// "gateway" uses a gateway but also prompts them to install a local node
// "local" uses a gateway but also prompts them to install a local node
// "disabled" disabled by the user
// "undefined" everything else (IPFS feature flag is not enabled, error etc)
getResolveMethodType: async () =>
String(await promisifyBraveCheck(chrome.ipfs.getResolveMethodType)),

// Obtains the config contents of the local IPFS node
// Returns undefined if missing for any reason
getConfig: async () =>
await promisifyBraveCheck(chrome.ipfs.getConfig),

// Returns true if binary is present
getExecutableAvailable: async () =>
Boolean(await promisifyBraveCheck(chrome.ipfs.getExecutableAvailable)),

// Attempts to start the daemon and returns true if finished
launch: async () =>
Boolean(await promisifyBraveCheck(chrome.ipfs.launch)),

// Attempts to stop the daemon and returns true if finished
shutdown: async () =>
Boolean(await promisifyBraveCheck(chrome.ipfs.shutdown))
})
Comment on lines -19 to +51
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint fixes only. Willing to do separately

: undefined

export async function init (browser, opts) {
Expand Down
26 changes: 22 additions & 4 deletions add-on/src/lib/ipfs-companion.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import createRuntimeChecks from './runtime-checks.js'
import { createContextMenus, findValueForContext, contextMenuCopyAddressAtPublicGw, contextMenuCopyRawCid, contextMenuCopyCanonicalAddress, contextMenuViewOnGateway, contextMenuCopyPermalink, contextMenuCopyCidAddress } from './context-menus.js'
import { registerSubdomainProxy } from './http-proxy.js'
import { runPendingOnInstallTasks } from './on-installed.js'
import { handleConsentFromState, startSession, endSession, trackView } from './telemetry.js'
const log = debug('ipfs-companion:main')
log.error = debug('ipfs-companion:main:error')

Expand All @@ -33,6 +34,7 @@ export default async function init () {
// INIT
// ===================================================================
let ipfs // ipfs-api instance
/** @type {import('../types.js').CompanionState} */
let state // avoid redundant API reads by utilizing local cache of various states
let dnslinkResolver
let ipfsPathValidator
Expand All @@ -55,8 +57,11 @@ export default async function init () {
runtime = await createRuntimeChecks(browser)
state = initState(options)
notify = createNotifier(getState)
// ensure consent is set properly on app init
handleConsentFromState(state)

if (state.active) {
startSession()
// It's ok for this to fail, node might be unavailable or mis-configured
try {
ipfs = await initIpfsClient(browser, state)
Expand Down Expand Up @@ -167,6 +172,15 @@ export default async function init () {
const result = validIpfsOrIpns(path) ? resolveToPublicUrl(path) : null
return Promise.resolve({ pubGwUrlForIpfsOrIpnsPath: result })
}
if (request.telemetry) {
return Promise.resolve(onTelemetryMessage(request.telemetry, sender))
}
}

function onTelemetryMessage (request, sender) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏽

if (request.trackView) {
return trackView(request.trackView)
}
}

// PORTS (connection-based messaging)
Expand Down Expand Up @@ -366,11 +380,11 @@ export default async function init () {
// https://github.com/ipfs-shipyard/ipfs-companion/issues/398
if (runtime.isFirefox && ipfsPathValidator.isIpfsPageActionsContext(url)) {
if (sameGateway(url, state.gwURL) || sameGateway(url, state.apiURL)) {
await browser.pageAction.setIcon({ tabId: tabId, path: '/icons/ipfs-logo-on.svg' })
await browser.pageAction.setTitle({ tabId: tabId, title: browser.i18n.getMessage('pageAction_titleIpfsAtCustomGateway') })
await browser.pageAction.setIcon({ tabId, path: '/icons/ipfs-logo-on.svg' })
await browser.pageAction.setTitle({ tabId, title: browser.i18n.getMessage('pageAction_titleIpfsAtCustomGateway') })
} else {
await browser.pageAction.setIcon({ tabId: tabId, path: '/icons/ipfs-logo-off.svg' })
await browser.pageAction.setTitle({ tabId: tabId, title: browser.i18n.getMessage('pageAction_titleIpfsAtPublicGateway') })
await browser.pageAction.setIcon({ tabId, path: '/icons/ipfs-logo-off.svg' })
await browser.pageAction.setTitle({ tabId, title: browser.i18n.getMessage('pageAction_titleIpfsAtPublicGateway') })
}
await browser.pageAction.show(tabId)
}
Expand Down Expand Up @@ -554,6 +568,8 @@ export default async function init () {
await registerSubdomainProxy(getState, runtime)
shouldRestartIpfsClient = true
shouldStopIpfsClient = !state.active
// Any time the extension switches active state, start or stop the current session.
state.active ? startSession() : endSession()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If state.active is changed, and state is true-ish, start session. Otherwise, end session.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@whizzzkid @lidel is there a better place to do this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for now this should be ok, in the future we might wanna have a message being broadcast and such listeners can subscribe to those.

break
case 'ipfsNodeType':
if (change.oldValue !== braveNodeType && change.newValue === braveNodeType) {
Expand Down Expand Up @@ -620,6 +636,8 @@ export default async function init () {
break
}
}
// ensure consent is set properly on state changes
handleConsentFromState(state)

if ((state.active && shouldRestartIpfsClient) || shouldStopIpfsClient) {
try {
Expand Down
4 changes: 2 additions & 2 deletions add-on/src/lib/notifier.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ export default function createNotifier (getState) {
return await browser.notifications.create({
type: 'basic',
iconUrl: browser.runtime.getURL('icons/ipfs-logo-on.svg'),
title: title,
message: message
title,
message
lidel marked this conversation as resolved.
Show resolved Hide resolved
})
} catch (err) {
log.error('failed to create a notification', err)
Expand Down
12 changes: 10 additions & 2 deletions add-on/src/lib/options.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
'use strict'

import { isIPv4, isIPv6 } from 'is-ip'
import isFQDN from 'is-fqdn'
import { isIPv4, isIPv6 } from 'is-ip'
lidel marked this conversation as resolved.
Show resolved Hide resolved

/**
* @type {Readonly<import('../types.js').CompanionOptions>}
*/
export const optionDefaults = Object.freeze({
active: true, // global ON/OFF switch, overrides everything else
ipfsNodeType: 'external',
Expand Down Expand Up @@ -31,7 +34,12 @@ export const optionDefaults = Object.freeze({
importDir: '/ipfs-companion-imports/%Y-%M-%D_%h%m%s/',
useLatestWebUI: false,
dismissedUpdate: null,
openViaWebUI: true
openViaWebUI: true,
telemetryGroupMinimal: true,
telemetryGroupPerformance: false,
telemetryGroupUx: false,
telemetryGroupFeedback: false,
telemetryGroupLocation: false
})

function buildDefaultIpfsNodeConfig () {
Expand Down
28 changes: 19 additions & 9 deletions add-on/src/lib/state.js
Original file line number Diff line number Diff line change
@@ -1,28 +1,38 @@
'use strict'
/* eslint-env browser, webextensions */

import { safeURL, isHostname } from './options.js'
// @ts-check
import { isHostname, safeURL } from './options.js'

export const offlinePeerCount = -1

/**
*
* @param {import('../types.js').CompanionOptions} options
* @param {Partial<import('../types.js').CompanionOptions>} [overrides]
* @returns {import('../types.js').CompanionState}
*/
export function initState (options, overrides) {
// we store options and some pregenerated values to avoid async storage
// reads and minimize performance impact on overall browsing experience
/**
* @type {Partial<import('../types.js').CompanionState & import('../types.js').CompanionOptions>}
*/
const state = Object.assign({}, options)
// generate some additional values
state.peerCount = offlinePeerCount
state.pubGwURL = safeURL(options.publicGatewayUrl)
state.pubGwURLString = state.pubGwURL.toString()
state.pubGwURLString = state.pubGwURL?.toString()
delete state.publicGatewayUrl
state.pubSubdomainGwURL = safeURL(options.publicSubdomainGatewayUrl)
state.pubSubdomainGwURLString = state.pubSubdomainGwURL.toString()
state.pubSubdomainGwURLString = state.pubSubdomainGwURL?.toString()
delete state.publicSubdomainGatewayUrl
state.redirect = options.useCustomGateway
delete state.useCustomGateway
state.apiURL = safeURL(options.ipfsApiUrl, { useLocalhostName: false }) // go-ipfs returns 403 if IP is beautified to 'localhost'
state.apiURLString = state.apiURL.toString()
state.apiURLString = state.apiURL?.toString()
delete state.ipfsApiUrl
state.gwURL = safeURL(options.customGatewayUrl, { useLocalhostName: state.useSubdomains })
state.gwURLString = state.gwURL.toString()
state.gwURLString = state.gwURL?.toString()
delete state.customGatewayUrl
state.dnslinkPolicy = String(options.dnslinkPolicy) === 'false' ? false : options.dnslinkPolicy

Expand All @@ -32,9 +42,9 @@ export function initState (options, overrides) {
try {
const hostname = isHostname(url) ? url : new URL(url).hostname
// opt-out has more weight, we also match parent domains
const disabledDirectlyOrIndirectly = state.disabledOn.some(optout => hostname.endsWith(optout))
const disabledDirectlyOrIndirectly = state.disabledOn?.some(optout => hostname.endsWith(optout))
// ..however direct opt-in should overwrite parent's opt-out
const enabledDirectly = state.enabledOn.some(optin => optin === hostname)
const enabledDirectly = state.enabledOn?.some(optin => optin === hostname)
return !(disabledDirectlyOrIndirectly && !enabledDirectly)
} catch (_) {
return false
Expand All @@ -55,5 +65,5 @@ export function initState (options, overrides) {
})
// apply optional overrides
if (overrides) Object.assign(state, overrides)
return state
return /** @type {import('../types.js').CompanionState} */(state)
}
43 changes: 43 additions & 0 deletions add-on/src/lib/telemetry.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
import MetricsProvider from '@ipfs-shipyard/ignite-metrics/vanilla'
import debug from 'debug'

const log = debug('ipfs-companion:telemetry')

const metricsProvider = new MetricsProvider({
appKey: '393f72eb264c28a1b59973da1e0a3938d60dc38a',
autoTrack: false,
storageProvider: null
})

/**
*
* @param {import('../types.js').CompanionState} state
* @returns {void}
*/
export function handleConsentFromState (state) {
const telemetryGroups = {
minimal: state?.telemetryGroupMinimal || false,
performance: state?.telemetryGroupPerformance || false,
ux: state?.telemetryGroupUx || false,
feedback: state?.telemetryGroupFeedback || false,
location: state?.telemetryGroupLocation || false
}
for (const [groupName, isEnabled] of Object.entries(telemetryGroups)) {
if (isEnabled) {
log(`Adding consent for '${groupName}'`)
metricsProvider.addConsent(groupName)
} else {
log(`Removing consent for '${groupName}'`)
metricsProvider.removeConsent(groupName)
}
}
}

const ignoredViewsRegex = []
export function trackView (view) {
log('trackView called for view: ', view)
metricsProvider.trackView(view, ignoredViewsRegex)
}

export const startSession = (...args) => metricsProvider.startSession(...args)
export const endSession = (...args) => metricsProvider.endSession(...args)
Loading