Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sprint - Dec 9 - Dec 20 #26634

Open
pauldambra opened this issue Dec 4, 2024 · 14 comments
Open

Sprint - Dec 9 - Dec 20 #26634

pauldambra opened this issue Dec 4, 2024 · 14 comments
Labels
sprint Sprint planning

Comments

@pauldambra
Copy link
Member

Global Sprint Planning

3 things that might take us down

Urgent incident follow-ups

https://github.com/orgs/PostHog/projects/103/views/2

Team sprint planning

For your team sprint planning copy this template into a comment below for each team.

# Team ___

**Support hero:** ___

## Retro

<!-- Grab the high and low priority items from last time and add whether that item was completed or not -->

- 

## Hang over items from previous sprint

<!-- For each item, decide to re-prioritise (and add below) or deprioritise -->

- Item 1. prioritised/deprioritise

## OKR

1. OKR, status (red/yellow/green) and action points if yellow/red


### High priority

-

### Low priority / side quests

-
@pauldambra pauldambra added the sprint Sprint planning label Dec 4, 2024
@pauldambra
Copy link
Member Author

pauldambra commented Dec 4, 2024 •
edited by marandaneto
Loading

Team And They All Rolled Over And One Fell Out

xmas party sprint

Support hero: @pauldambra

Retro
  • @pauldambra safari bug was horrible, but at least fixed, good but tiring to make progress as well as fight bugs
  • @ioannisj progress on understanding long sessions but requires a bit of refactoring on how we rotate session and manage timestamps. Number of small fixes and improvements
  • @marandaneto fun conference, trying to sell to huge companies while he's there
  • changing teams, and new people - sad but exciting

## what we shipped

Mobile

Replay

  • ✅ replay 4000 low hanging fruit @pauldambra
    • got a chunk of this done
    • meeting adam in london to talk ux

Low priority / side quests

OKRs 🟡
  1. OKR, status (red/yellow/green) and action points if yellow/red

🟡📱Goal 1: People think of PostHog as a mobile solution
🟡 📹 Goal 2: Replay 4000
- this is now so excellent it's Replay 4000
- #26098
🟡 🎤 Goal3: Ship the things users keep asking for

  • queryable network data
  • start replay on trigger e.g. event

High priority

Mobile

Replay

  • replay 4000 UI
    • improve overview tab using sessions data
  • a brief diversion before querying network data - wrap websockets as well as fetch @pauldambra

Low priority / side quests

  • 🔄 get people using iOS autocapture @ioannisj and @marandaneto
  • (can we|do we) measure watch time and buffer time? @pauldambra
  • go through support and find people who wanted block list and event trigger @pauldambra

@pauldambra pauldambra pinned this issue Dec 4, 2024
@jurajmajerik
Copy link
Contributor

jurajmajerik commented Dec 4, 2024 •
edited
Loading

Team Experiments 🧪

Retro

Sprint goals

Support hero: @danielbachhuber

  • Continue with onboarding tasks and get familiar with the codebase. @andehen
    • Low hanging UI improvements.
    • Improve the error response from the results calculation for better diagnostics of "no results".
    • Familiarize himself with our statistical methods.
  • Test the Data Warehouse integration with 1-2 customers (Freepik, potentially ElevenLabs). @danielbachhuber
  • Finish support for multiple experiment metrics. @jurajmajerik
    • Add UI to support this
  • Plug improved statistical calculation for Trends + add docs @danielbachhuber, @andehen, @jurajmajerik
    • Clean up our current 3 statistical methods and run everything by our statistician contractor once more.
    • Implement the method for Trend counts.
    • Enable it for newly created experiments.

@daibhin
Copy link
Contributor

daibhin commented Dec 4, 2024 •
edited
Loading

Team Error Tracking

@neilkakkar off (1 week)
@oliverb123 CDP support rotation (2 weeks)

Retro

✅ Cymbal stack frame data quality @oliverb123 @daibhin
Stack trace resolution bugs solved
Symbol set management (deleting / uploading source maps)

✅ Cymbal Python support @oliverb123
Redesigned stack trace view. Uploading source maps in-app is on track
Investigated adding and built code context to each frame

🟢 Update frontend querying and UI @daibhin
Writing fingerprint overrides to ClickHouse
Aggregating exceptions based on overrides WIP but will be finished this week

✅ Migrate to Argo for deployments and external secret management (ft. infra) @oliverb123

🔴 First instance alerting via Hog functions @daibhin
PR with one possible approach ready. Need to figure out how this fits in with threshold based alerting / CDP

🟠 Investigate build time source map uploading @daibhin
Understand how it's done by Sentry. Holding off taking things further until a customer needs it

OKR

✅ Human readable stack traces for Javascript
🟢 Ensure scalability of query time issue grouping
🟢 Alerting via CDP
🟢 Launch

High priority

Launch closed beta @daibhin

  • Calls with 5 customers and make sure they get setup
  • Implement quick win feedback (e.g. clientside fingerprinting) @daibhin
  • Reimplement error tracking in billing repo

Cymbal improvements @oliverb123

  • Performance / reliability for scaling load (e.g. event batching, column materialisation)
  • Alerting & monitoring for core metrics @daibhin
  • Fix EU deployment (e.g. cleanup symbol sets)
  • Investigate potential memory leak

Alerting @oliverb123

  • Implement "first instance" alerting to unblock beta testers
  • RFC on possible long term approaches (e.g. integrating with threshold based alerting)

Low priority / side quests

  • Implement panels based UI for issue page @daibhin
  • Improve issue creation (remove foreign key or use deterministic uuid)

@Phanatic
Copy link
Contributor

Phanatic commented Dec 4, 2024 •
edited by marandaneto
Loading

Team Surveys

@Phanatic offboarding December 23rd.

Retro

✅ Shipped Adaptive sampling for survey responses.
✅ Bug fixes for NPS recurring surveys.
✅ Customer support

Sprint goals

  • Fixing bugs reported by Annika and a few customers that we're getting more survey responses than allowed when using Adaptive Sampling.
  • Fixing bugs reported by customers of NPS recurring surveys who are confused about when the survey is shown or hidden.
  • Handing off no-code experiments to the Experiments team.
  • Handing off Surveys to @marandaneto and @ioannisj
  • Customer support
  • Q1 Planning

@MichaelKutsch-ph
Copy link

MichaelKutsch-ph commented Dec 4, 2024 •
edited
Loading

Team Infrastructure

(WIP)

Retro

  • 🔴 100% do the reservation changes ASAP - @danielxnj
    • we want to wait until the clickhouse migration is done so we see the actual feasible spend
    • discussions about reseller / PPA with AWS has a huge impact on this decision, we could do some wrong decisions here if this is not finalized first
  • 🟢 Optimize the LB/envoy deployments to remove all cross AZ traffic (cost optimization) @MichaelKutsch-ph
    • has positive effects on the bill, but not as impactful as expected -> loads of work on every tiny part, therefore better focus on pricing agreements
  • 🟡 ArgoCD - move lowest priority posthog services over (cdp, temporal-workers, cymbal) @bciaraldi
    • Cymbal is deployed 👍
    • we had some other issues that were more important to resolve, therefore prioritized down
    • A bunch of stuff missing on our internal cluster to make this stuff super robust
  • updating VM to latest to resolve alerting issues
    • challenging and time consuming task - but we prioritized this as we have issues with alerting
    • we have to talk to VM to figure out some dependencies
  • initial discussions with nOps, Pump and DoIT

OKR

  1. 🏃 Continuous Delivery
  2. 💰 Cost visualization
  3. 🧪 Continuous Integration

High priority

  • install nOps agent to detect x-az traffic issues @danielxnj
  • dive into details and decide about AWS Pricing @MichaelKutsch-ph
  • Lock in WS @timgl
  • Continue w/ internal cluster into the same state as our prod clusters (monitoring stack, SSO access etc.) @danielxnj
  • ArgoCD @bciaraldi
    • some prep stuff to do for the cluster
    • start moving apps to argo in parallel @MichaelKutsch-ph
      • Get PostHog web (and workers) onto ArgoCD
  • when Frank is back: @benjackwhite
    • plan move to Warpstream for capture/events
  • exhaustion of IPs on prod: looking into it @bciaraldi

Low priority / side quests

@robbie-c
Copy link
Member

robbie-c commented Dec 4, 2024 •
edited by rafaeelaudibert
Loading

Team web analytics

Support hero: @rafaeelaudibert

Retro

Robbie
🟢 Onboard Rafael
🟢 SDK Support hero exists
🟠 Finish cookieless prototype
🟢 Wrote RFC about marketing analytics: https://github.com/PostHog/product-internal/pull/684

Rafael
🟢 Was onboarded (and updated some onboarding docs)
🟢 Languages & Timezones queries
🟢 Comparison to previous period on all queries
🟡 Conversion goal

OKR

🟠 Continue to improve performance
🟢 Build a small amount of more advanced marketing features, to measure interest
🟡 Cookieless tracking prototype
🔴 Improve the NextJS experience
🔴 More page-level insights
🇬🇦 Improve the documentation, and link to it more from the app

High priority

Rafael

  • Get comfortable doing web analytics support
  • Conversion goals
  • Add more options for previous period comparison (use product analytics UI)

Robbie

Stretch goals

Rafael

  • Help out SDK Support with next/react in mind

Robbie

  • Page level insights

@benjackwhite
Copy link
Contributor

benjackwhite commented Dec 4, 2024 •
edited
Loading

Team Conversion Drivers & Planners (CDP)

Retro

  • 🔴 @benjackwhite / @mariusandra client->posthog->destination event mapping system for ad destinations + infra
    • Swapped to Marius' todo - not a huge amount of progress due to sickness + incident work
  • 🟢 @mariusandra plumbing for allowing custom JS (GTM replacement anybody??)
    • Pretty much there although waiting on the Mappings + stateless asset delivery to be production ready
  • 🟡 @MarconLP building and testing client side destinations, E2E tests
    • E2E testing was a bust due to captchas etc so something to come back to in the future
    • Client side destinations weren't ready but major improvements to the Google Ads integration that is now basically there
  • @benjackwhite also distracted with continous onboarding of Infra team as well as planning ahead to new hires

OKRs

  • 🟢 Widespread usage of the new CDP
  • 🔴 Messaging V1
  • 🟡 Ingestion is rock solid

High priority

  • @benjackwhite gradually roll out RemoteConfigs to everyone (providing the code for site-functions)
  • @mariusandra finish Mappings work with focus on TikTok Ads
  • @MarconLP build out and test TikTok Ads client side integration

Side quests

  • Ideas for growth hacking of destinations (big list of all known SaaS we could work with that people can search for to upvote etc.
  • Continue investigations into Warpstream events capture (utilize return of Frank perhaps)

@Twixes
Copy link
Member

Twixes commented Dec 4, 2024

Team AI PM

Retro

High priority

  • 🟢 Fix response streaming in production @skoob13 with @PostHog/team-infra
  • 🔴 Make summarizer self-aware when we couldn't fully answer the question, and why @Twixes
  • 🔴 All other classic insights types (order of importance: retention, paths, stickiness, lifecycle) @Twixes
  • 🔴 Max AI rolled out as feature preview @PostHog/team-ai-product-manager
  • 🍋‍🟩 Memory, prewarmed by analyzing existing data and definitions @skoob13
  • 🔴 SQL generation MVP @skoob13

Low priority / side quests

  • 🔴 In your product's website, out a complete description of the product for the agent's context
  • 🟢 Environments out as a feature preview @Twixes

Hang over items from previous sprint

  • Make summarizer self-aware when we couldn't fully answer the question, and why @Twixes
  • All other classic insights types (order of importance: retention, paths, stickiness, lifecycle) @Twixes
  • Max AI rolled out as feature preview @PostHog/team-ai-product-manager

OKR

High priority

  • Make summarizer self-aware when we couldn't fully answer the question, and why @Twixes
  • All other classic insights types (order of importance: retention, paths, stickiness, lifecycle) @Twixes
  • Max AI rolled out as feature preview @PostHog/team-ai-product-manager
  • Human-in-the-loop for clarifying taxonomy @skoob13

@timgl
Copy link
Collaborator

timgl commented Dec 4, 2024

Team Movers of Data

Support hero: Tomas/Ross

Retro

  • We managed to get to the bottom of missing events in batch exports
  • Could have better alerts

Hang over items from previous sprint

  • Item 1. prioritised/deprioritise

OKR

Coming soon!

High priority

@fuziontech
Copy link
Member

fuziontech commented Dec 4, 2024 •
edited
Loading

Team ClickHouse

OKR Q2 2024

Objective

Goals for Q4:

  • Improve elasticity and flexibility of our data store by putting all our data in Iceberg
    • Work with Altinity effectively to ship read path for Iceberg on ClickHouse - Brett Hoerner
    • Setup infrastructure to ship all of our data to Iceberg on S3 - James Greenhill
    • Shipping Query logs to Iceberg - James Greenhill
  • Continue improving CH operational expertise
    • Upgrade to a later version of Clickhouse - James Greenhill
    • Capacity planning - James Greenhill
    • Automation - Daniel Escribano
    • Put some of the basic mitigation operations in runbooks - Daniel Escribano
  • Schema management
    • Tool for schema migration (coordinator schemas) - Daniel Escribano
    • Tool for long running mutations - Daniel Escribano
  • Continued investment in performance
    • Tooling for other teams to understand which queries are slow and why - Ted Kaemming
    • Investigate variability of queries - Ted Kaemming
    • Per-team limits on queries/query complexity (needs product work) - Ted Kaemming

Board

https://github.com/orgs/PostHog/projects/85/views/2

Retro

🔥 Incidents 🔥

Last Sprint's Goals

  • Validate cluster data parity between old and new clusters @fuziontech ⏳
  • Upgrade clusters to 24.8 (US, EU, new-US, Dev) @fuziontech @Daesgar ✅ 🎉
  • Anticipating new 24.8 cluster w/ real lightweight deletes - fix deletes @fuziontech ⏳
  • Migrate to new 8 shard lower cost cluster w/ coordinator nodes @Daesgar ⏳
  • New mechanism for managing Mat Cols @tkaemming ✅
  • Create new users for different workloads and isolate resources based on users @fuziontech ⏳
  • (low priority) Expose Iceberg table for testing @fuziontech
    • Current version this will be quite slow
    • New 24.8 version will be quite fast
  • Better caching of object store bits for Iceberg (Working in CH codebase) @bretthoerner ❌
    • Code does a HEAD request for every file even if it's in cache for files that don't change

This Sprint's Goals

Holiday plans

@Daesgar 💒 out from Dec 5 -> Dec 18

TL;DR

- Review increasing cost on all AWS @fuziontech 
~~~ Validate and move over to new cluster quickly ~~~
- Validate cluster data parity between old and new clusters @fuziontech 
- Fix deletes @fuziontech 
- Benchmark to new 8 shard lower cost cluster w/ coordinator nodes @fuziontech  
~~~ Mat Col optimizations ~~~
- Investigate how to speed up index dropping for Mat Cols @tkaemming 
- Yeet unused mat cols @tkaemming 
~~~ Below the fold ~~~
- Investigate what needs to be done for queue metering / QoS @fuziontech 
- Get query logs into iceberg. @fuziontech 
- (low priority) Expose Iceberg table for testing @fuziontech 
	- Current version this will be quite slow
	- New 24.8 version will be quite fast

@dmarticus
Copy link
Contributor

Team Flags

Support

Retro

Things we got done

  • Track feature flag called events in usage reports so we can track up who set up flags successfully @dmarticus
  • generally ironing out service (centralizing error handling, reducing memory usage overhead with smarter copying/cloning)
    • found a bug in the following two paths (1) feature flag payloads, and (2) group targeting without group overrides. PR to come later today.
  • land v0 of flag staleness feature @havenbarnes
  • Fixed issue with feature flag payloads SDK methods on the server-side (Node, Python) not correctly sending feature_flag_response payloads @dmarticus
  • Show feature flag payload as official PostHog property @havenbarnes
  • Adding some debugging event properties to $feature_flag_called to help indicate when SDKs are using bootstrapping

Things we didn't get done and why

  • testing/productionizing new /flags service @dmarticus
    • functional testing via django-compat integration tests
    • "real" testing via mirroring traffic to new service (meeting with ingestion team 11-20 to go through this.

As I was writing more integration tests, I came across some bugs, and needed to fix them.

  • Start work on feature management UI @havenbarnes
    • needed realignment on the correct UI abstractions for this feature; made an initial PR to start that conversation, and are iterating on it.

Sprint

High priority

  • testing/productionizing new /flags service @dmarticus
    • fixing P0 bugs along /flags hotpath
    • functional testing via django-compat integration tests
    • "real" testing via mirroring traffic to new service

Side Quests

  • Continue to make sentry more actionable and useful @havenbarnes
  • Figure out how to cut off the flags bit of /decide (and, in future /flags) when folks go over – right now we don't stop serving flags when overage happens @dmarticus
  • Continue to look at growth metrics and see what other properties we can add to better understand our growth @dmarticus

Things to chew on

  • Can we know when people are starting to make new features?

OKRs

  1. Ship new feature flags in production and accomplish the 10x scale + 2x speed boost performance targets (ideally decreases infra spend by N% as well, will try to figure out a more specific target going forward)
  2. Maintain current /decide service and make optimizations.

@zlwaterfield
Copy link
Contributor

zlwaterfield commented Dec 4, 2024 •
edited by patricio-posthog
Loading

Team Growth

Retro

Retro items
  • @zlwaterfield
    • RBAC
      • Get middleware PR tested and shipped
      • Get RBAC UI shipped for 1 product
    • Improve how we show value to teams addon customers when unsub'd or cancelling trial (think about how to do this for all products)
    • Talk to DW about materialzed views
    • Talk to Mine/Joe about better startup plan labels
    • Continue talking to Stripe PM about billing problems we are running into
    • Continue move to c.io transactional emails
    • extra
      • Personal API @ current endpoint for Swiftly
      • 2FA - move into our UI and get backup codes working (WIP)
      • Increased billing postgres size + adding in pgbouncer and read replica (WIP)
  • @surbhi-posthog
    • Review exiting ICP research
    • Connect with Mine about ICP data being sync'd into PostHog
    • Do some analysis on our metrics through signup, onboarding, converting to paid based on our ICP scoring data
    • Suggest (and maybe implement) changes to the onboarding product experience based on learnings from the analysis.
  • @patricio-posthog
    • Forecasting: 1 - Pato: 0. Still fighting this battle.
    • New pricing for queries (API) and mobile

Q4 Goals

✅=finished 🟡=in progress 🔴=won't finish ⚪=not started

  1. 🟡 Get to a 4/10 in Enterprise platform support (Zach)
    • 🟡 Release Enterprise addon and iterate with sales to find a pricing model that makes sense
    • 🟡 Build RBAC
  2. 🟡 Make revenue & forecasting more reliable (Patricio)
    • ✅ Rejig how we do sync_invoices so we don’t see big drops - respond to webhooks instead of large syncs
    • ⚪ Evaluate temporal vs celery for usage reports and billing tasks so we can adhere to rate limits better
    • ⚪ Side hustle, if time permits: Rejig how anonymous / identified events are billed, as the current implementation is confusing.
  3. 🟡 Activation: reverse trials, analytics onboarding flow, understand CC activation (Raquel)
    • ⚪ Dig into the data, watch replays to figure out why CC activation rate has been falling
    • ⚪ Add reverse trial to onboarding
    • ✅ Add reverse trial to billing page for teams

Stretch goals:

  • 🟡 API query pricing (Raquel & Patricio)
  • ⚪ Move the annual credits flow out of Zapier

This sprint

Time off: @zlwaterfield off 2 days and working 1/2 days for 2 days.

  • @zlwaterfield

    • Support hero week 1
    • Misc fixes from RBAC middleware changes
    • RBAC UI behind a feature flag
    • Finish the 2FA improvements
    • Block usage reporting when on a trial (request from sales)
    • Finish billing postgres db improvements - pgbouncer and read replica

  • @surbhi-posthog

    • Support her week 2
    • custom sql analysis on ICP data
    • load icp properties into Posthog?
    • Add events for Team's features in the backend

  • @patricio-posthog

    • Keep monitoring the new forecasting, check that everything runs smoothly
    • Update /billing endpoint to include the new forecasting for customers
    • API query pricing
    • Start working on the new pricing for mobile RFC

@EDsCODE
Copy link
Member

EDsCODE commented Dec 4, 2024

Team Data Storehouse

OKR Q4 2024

Objective

Query 3000

  • Key Results:
    • Autocomplete
    • Increase general BI experience/product BI meta#157
    • Declutter the data warehouse UI and make the features intuitive to find

Data Modeling MVP

  • Key Results:
    • Infrastructure decided and implemented
    • Integrating external data with feature flags
    • External data everywhere in insights/persons/cohorts
    • Get billing team to use modeling in posthog for their invoices_with_annual table

Retro

Team split in 2

  • Data import rebuild @Gilbert09
    • release internally by end of week
  • SQL editor 3000 finish MVP section + move to new priorities @EDsCODE
    • release internally be end of week

Priority

Tom off for most of sprint

@aspicer
Copy link
Contributor

aspicer commented Dec 4, 2024 •
edited by thmsobrmlr
Loading

Team Product Analytics

🎉 Peter Joining Dec 9th!

Q4 2024 objectives

Legacy 0x - Thomas owning

  • All insights are query-based

  • rm -rf posthog/queries/

  • Insight query is stored in the URL for instant sharing of unsaved insights

First-class environments - Michael owning

  • Environments are rolled out to everyone

Threshold-based alerts - Anirudh owning

  • All users get to set up alerts on PostHog metrics
  • Alerts are launched with Marketing, and positioned into paid plans
  • We're using alerts ourselves for key trends and funnels

Flexible funnels - Sandy owning

  • UDF funnels rolled out to everyone, unlocking arbitrary complexity
  • Allowing parts of funnels to be unordered. No more separation between regular, unordered, and strict funnels
  • Optional funnel steps

10x onboarding and ease of discovering features - Anirudh and Sandy owning

  • We run an exercise in implementing PostHog in a hobby project. What stands out as painful or unclear?

User delight 2x - support rotation owning

B2B analytics research - Anna owning

  • Understand the market and user needs better in order to improve our current Group analytics product next quarter

Retro

Support hero @thmsobrmlr @anirudhpillai

Anirudh (Support second week)

  • 🟢 Alert Comms / Launch
    • Official docs out, monitoring
  • 🟢 Support for launch / fast-follow features for alerts
  • 🟡 Cached dashboards (Cached dashboards #26309)
    • Remainder of sprint - rolling to more customers

Sandy

  • 🟢 Finish Cohort Migration
    • Passing local tests
    • Need to write testing framework for production data
  • 🟡 Spec out funnel improvements for December Optional funnel steps #17313
  • 🟢 Plan Peter's Onboarding, find tasks
  • [Side Quest] Hi-pri customer query issues
    • Person overrides table large
    • Removing join from actor query

Anna

  • 🟢 Group Analytics Research, generate plan for Q1
    • Survey sent out and user interviews in progress

Thomas (Support First Week)

  • 🔴 Migrate api endpoints to use hogql queries
  • 🟡 Look into filters removal broadly
    • PRs up to stop creating new filter based insights from templates
  • 🔴 Fix lifecycle charts with personless events
  • was ill for a chunk of the sprint and more work on customizable insight colors / themes happened

Upcoming Sprint

Support hero: @aspicer @thmsobrmlr

SDK Support hero: @anirudhpillai @aspicer

  • Alerts [Anirudh]

    • Scaling with the rollout
      • Improving execution system for running alerts
      • Bugfixes and deploy monitoring

  • Carryover: Cached Dashboards, further rollout [Anirudh]

    • Thinking about API concurrency and scaling

  • Peter Onboarding [Sandy]

  • Carryover: Cohort migration production monitoring and bug fixing [Sandy]

  • Preparing for Planning [Anna]

    • Collect build better notes and user interviews for Q1 Planning

  • Migrate api endpoints to use hogql queries [Thomas]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
sprint Sprint planning
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests