Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(rethinkdb): OrganizationUser: Phase 3 #9965

Merged
merged 74 commits into from
Jul 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
bfa5c21
chore: read ReflectionGroups from PG
mattkrick May 29, 2024
7ac086e
Merge branch 'master' into chore/retrogroups3
mattkrick May 29, 2024
9844960
Merge branch 'master' into chore/retrogroups3
mattkrick May 30, 2024
62e123c
chore: add PG RetroReflection table
mattkrick May 30, 2024
c38f86c
chore: write to PG RetroReflection table
mattkrick Jun 3, 2024
d318560
fixup: inserts and updates
mattkrick Jun 4, 2024
16843dc
Merge branch 'master' into retroReflection-phase1e
mattkrick Jun 4, 2024
abb3bdf
fix: move to record literal types
mattkrick Jun 4, 2024
9911448
fix types
mattkrick Jun 4, 2024
af03b31
chore: migrate reflections to PG
mattkrick Jun 6, 2024
2a3c0ca
Merge branch 'master' into retroReflection-phase1e
mattkrick Jun 6, 2024
8dfa870
fix: remove hard delete of inactive groups
mattkrick Jun 6, 2024
3b48d22
Merge branch 'retroReflection-phase1e' into chore/retroReflection-phase2
mattkrick Jun 6, 2024
24da04b
fixup: remove unused var
mattkrick Jun 6, 2024
c60e502
Merge branch 'retroReflection-phase1e' into chore/retroReflection-phase2
mattkrick Jun 6, 2024
0d2c7d7
handle escape chars and commas
mattkrick Jun 6, 2024
9ef9346
chore: begin building equality checker
mattkrick Jun 6, 2024
76aeff9
fixup: account for lots of formatting in content
mattkrick Jun 7, 2024
0830c8e
Merge branch 'retroReflection-phase1e' into chore/retroReflection-phase2
mattkrick Jun 24, 2024
a12b08a
fix: rename extra spaces for conversion from plaintext to content
mattkrick Jun 24, 2024
a732657
fix: rename Reactji composite type attributes
mattkrick Jun 24, 2024
954503d
fix: constructor prop in lookup table
mattkrick Jun 25, 2024
c7775ad
handle reactji migration
mattkrick Jun 25, 2024
d53aca4
change dataloaders to pg
mattkrick Jun 25, 2024
6689e39
fix: self-review
mattkrick Jun 25, 2024
a6fd0d3
Merge branch 'master' into chore/retroReflection-phase2
mattkrick Jun 25, 2024
00964a8
Merge branch 'chore/retroReflection-phase2' into chore/retroReflectio…
mattkrick Jun 25, 2024
50ca56b
fix: migration order rename
mattkrick Jun 25, 2024
b220f22
Merge branch 'chore/retroReflection-phase2' into chore/retroReflectio…
mattkrick Jun 25, 2024
22ca4f3
chore: write to PG
mattkrick Jun 25, 2024
9c55acd
chore: migrate old records to PG
mattkrick Jun 25, 2024
9dfd35c
fix: add teamid FK
mattkrick Jun 25, 2024
654aa42
Merge branch 'chore/timeline-phase1' into chore/timeline-phase2
mattkrick Jun 25, 2024
ebbea29
chore: read from pg
mattkrick Jun 25, 2024
7dbea0d
Merge branch 'master' into chore/timeline-phase2
mattkrick Jun 26, 2024
3fd2236
Merge branch 'chore/timeline-phase2' into chore/timeline-phase3
mattkrick Jun 26, 2024
c552ea6
fix bad merge
mattkrick Jun 26, 2024
50d9e77
Merge branch 'chore/timeline-phase2' into chore/timeline-phase3
mattkrick Jun 26, 2024
b7c092d
add migration
mattkrick Jun 26, 2024
fe9efe9
write to pg
mattkrick Jun 27, 2024
4d838f3
Merge branch 'master' into chore/organization-phase1
mattkrick Jun 27, 2024
34b05bf
Merge branch 'master' into chore/organization-phase1
mattkrick Jul 1, 2024
02aa23d
Merge branch 'master' into chore/organization-phase1
mattkrick Jul 1, 2024
dfba81e
Merge branch 'master' into chore/organization-phase1
mattkrick Jul 1, 2024
81d68b7
Merge branch 'master' into chore/organization-phase1
mattkrick Jul 3, 2024
f4f5ede
fix: array append/remove
mattkrick Jul 3, 2024
dcbcd78
chore: migrate existing orgs to PG
mattkrick Jul 3, 2024
b329fd3
remove RethinkDB.Organization
mattkrick Jul 4, 2024
0a549d6
Merge branch 'master' into chore/organization-phase2
mattkrick Jul 4, 2024
5f6bdfc
Merge branch 'chore/organization-phase2' into chore/organization-phase3
mattkrick Jul 4, 2024
21876d9
Merge branch 'master' into chore/organization-phase3
mattkrick Jul 4, 2024
af856f2
fix: isRequestToJoin tests
mattkrick Jul 8, 2024
129d1a9
fix: isOrgVerified
mattkrick Jul 8, 2024
6e368a4
fix deleteUser test
mattkrick Jul 8, 2024
27b0cf2
fix: comment
mattkrick Jul 9, 2024
b5422dd
Merge branch 'master' into chore/organization-phase3
mattkrick Jul 9, 2024
3c061e2
fix: remove newUserUntil
mattkrick Jul 9, 2024
1cdf979
chore: replace half of OrganizationUser instances
mattkrick Jul 9, 2024
473c0c5
remove newUserUntil on client
mattkrick Jul 10, 2024
49be1bc
fix: update all instances of OrganizationUser
mattkrick Jul 10, 2024
65614c6
Merge branch 'master' into chore/organizationUser-phase1
mattkrick Jul 10, 2024
ca01882
fix: bad merge
mattkrick Jul 10, 2024
44d153e
re-add rethinkdb to server tests
mattkrick Jul 10, 2024
303f493
chore: migrate existing data
mattkrick Jul 10, 2024
3c3b46c
fix tests
mattkrick Jul 10, 2024
3e3634c
fix: use test schema
mattkrick Jul 10, 2024
8d33999
fix: safer queries
mattkrick Jul 10, 2024
78dcb46
Merge branch 'chore/organizationUser-phase1' into chore/organizationU…
mattkrick Jul 10, 2024
4fd2139
remove RethinkDB.OrganizationUser
mattkrick Jul 11, 2024
24a2698
Merge branch 'master' into chore/organizationUser-phase2
mattkrick Jul 11, 2024
0b893a5
Merge branch 'chore/organizationUser-phase2' into chore/organizationU…
mattkrick Jul 11, 2024
ef0b549
fix tests
mattkrick Jul 11, 2024
8a1b2e0
fix: quit DB in jest
mattkrick Jul 15, 2024
e8b960b
fix: quit DB in jest
mattkrick Jul 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion codegen.json
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@
"NotifyTaskInvolves": "../../database/types/NotificationTaskInvolves#default",
"NotifyTeamArchived": "../../database/types/NotificationTeamArchived#default",
"Organization": "./types/Organization#OrganizationSource",
"OrganizationUser": "../../database/types/OrganizationUser#default as OrganizationUser",
"OrganizationUser": "../../postgres/types/index#OrganizationUser",
"PokerMeeting": "../../database/types/MeetingPoker#default as MeetingPoker",
"PokerMeetingMember": "../../database/types/MeetingPokerMeetingMember#default as PokerMeetingMemberDB",
"PokerTemplate": "../../database/types/PokerTemplate#default as PokerTemplateDB",
Expand Down
6 changes: 5 additions & 1 deletion packages/server/__tests__/globalTeardown.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
import getRethink from '../database/rethinkDriver'
import getKysely from '../postgres/getKysely'
import getRedis from '../utils/getRedis'

async function teardown() {
const r = await getRethink()
return r.getPoolMaster()?.drain()
await r.getPoolMaster()?.drain()
await getKysely().destroy()
await getRedis().quit()
}

export default teardown
51 changes: 12 additions & 39 deletions packages/server/billing/helpers/adjustUserCount.ts
Original file line number Diff line number Diff line change
@@ -1,15 +1,12 @@
import {sql} from 'kysely'
import {InvoiceItemType} from 'parabol-client/types/constEnums'
import getRethink from '../../database/rethinkDriver'
import {RDatum} from '../../database/stricterR'
import OrganizationUser from '../../database/types/OrganizationUser'
import generateUID from '../../generateUID'
import {DataLoaderWorker} from '../../graphql/graphql'
import isValid from '../../graphql/isValid'
import getKysely from '../../postgres/getKysely'
import insertOrgUserAudit from '../../postgres/helpers/insertOrgUserAudit'
import {OrganizationUserAuditEventTypeEnum} from '../../postgres/queries/generated/insertOrgUserAuditQuery'
import {getUserById} from '../../postgres/queries/getUsersByIds'
import updateUser from '../../postgres/queries/updateUser'
import IUser from '../../postgres/types/IUser'
import {Logger} from '../../utils/Logger'
import {analytics} from '../../utils/analytics/analytics'
Expand Down Expand Up @@ -45,39 +42,25 @@ const maybeUpdateOrganizationActiveDomain = async (
}

const changePause = (inactive: boolean) => async (_orgIds: string[], user: IUser) => {
const r = await getRethink()
const pg = getKysely()
const {id: userId, email} = user
mattkrick marked this conversation as resolved.
Show resolved Hide resolved
inactive ? analytics.accountPaused(user) : analytics.accountUnpaused(user)
analytics.identify({
userId,
email,
isActive: !inactive
})
await Promise.all([
updateUser(
{
inactive
},
userId
),
getKysely()
.updateTable('OrganizationUser')
.set({inactive})
.where('userId', '=', userId)
.where('removedAt', 'is', null)
.execute(),
r
.table('OrganizationUser')
.getAll(userId, {index: 'userId'})
.filter({removedAt: null})
.update({inactive})
.run()
])
await pg
.with('User', (qb) => qb.updateTable('User').set({inactive}).where('id', '=', userId))
.updateTable('OrganizationUser')
.set({inactive})
.where('userId', '=', userId)
.where('removedAt', 'is', null)
.execute()
}

const addUser = async (orgIds: string[], user: IUser, dataLoader: DataLoaderWorker) => {
const {id: userId} = user
const r = await getRethink()
const [rawOrganizations, organizationUsers] = await Promise.all([
dataLoader.get('organizations').loadMany(orgIds),
dataLoader.get('organizationUsersByUserId').load(userId)
Expand All @@ -90,19 +73,18 @@ const addUser = async (orgIds: string[], user: IUser, dataLoader: DataLoaderWork
)
const organization = organizations.find((organization) => organization.id === orgId)!
// continue the grace period from before, if any OR set to the end of the invoice OR (if it is a free account) no grace period
return new OrganizationUser({
id: oldOrganizationUser?.id,
return {
id: oldOrganizationUser?.id || generateUID(),
orgId,
userId,
tier: organization.tier
})
}
mattkrick marked this conversation as resolved.
Show resolved Hide resolved
})
await getKysely()
.insertInto('OrganizationUser')
.values(docs)
.onConflict((oc) => oc.doNothing())
.execute()
await r.table('OrganizationUser').insert(docs, {conflict: 'replace'}).run()
await Promise.all(
orgIds.map((orgId) => {
return maybeUpdateOrganizationActiveDomain(orgId, user.email, dataLoader)
Expand All @@ -111,22 +93,13 @@ const addUser = async (orgIds: string[], user: IUser, dataLoader: DataLoaderWork
}

const deleteUser = async (orgIds: string[], user: IUser) => {
const r = await getRethink()
orgIds.forEach((orgId) => analytics.userRemovedFromOrg(user, orgId))
await getKysely()
.updateTable('OrganizationUser')
.set({removedAt: sql`CURRENT_TIMESTAMP`})
.where('userId', '=', user.id)
.where('orgId', 'in', orgIds)
.execute()
await r
.table('OrganizationUser')
.getAll(user.id, {index: 'userId'})
.filter((row: RDatum) => r.expr(orgIds).contains(row('orgId')))
.update({
removedAt: new Date()
})
.run()
}

const dbActionTypeLookup = {
Expand Down
12 changes: 0 additions & 12 deletions packages/server/billing/helpers/teamLimitsCheck.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,6 @@ const enableUsageStats = async (userIds: string[], orgId: string) => {
.where('userId', 'in', userIds)
.where('removedAt', 'is', null)
.execute()
await r
.table('OrganizationUser')
.getAll(r.args(userIds), {index: 'userId'})
.filter({orgId})
.update({suggestedTier: 'team'})
.run()
await pg
.updateTable('User')
.set({featureFlags: sql`arr_append_uniq("featureFlags", 'insights')`})
Expand Down Expand Up @@ -101,12 +95,6 @@ export const maybeRemoveRestrictions = async (orgId: string, dataLoader: DataLoa
.where('userId', 'in', billingLeadersIds)
.where('removedAt', 'is', null)
.execute(),
r
.table('OrganizationUser')
.getAll(r.args(billingLeadersIds), {index: 'userId'})
.filter({orgId})
.update({suggestedTier: 'starter'})
.run(),
removeTeamsLimitObjects(orgId, dataLoader)
])
dataLoader.get('organizations').clear(orgId)
Expand Down
18 changes: 2 additions & 16 deletions packages/server/billing/helpers/updateSubscriptionQuantity.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import getRethink from '../../database/rethinkDriver'
import getKysely from '../../postgres/getKysely'
import insertStripeQuantityMismatchLogging from '../../postgres/queries/insertStripeQuantityMismatchLogging'
import RedisLockQueue from '../../utils/RedisLockQueue'
Expand All @@ -10,7 +9,6 @@ import {getStripeManager} from '../../utils/stripe'
* @param logMismatch Pass true if a quantity mismatch should be logged
*/
const updateSubscriptionQuantity = async (orgId: string, logMismatch?: boolean) => {
const r = await getRethink()
const pg = getKysely()
const manager = getStripeManager()

Expand All @@ -35,29 +33,17 @@ const updateSubscriptionQuantity = async (orgId: string, logMismatch?: boolean)
return
}

const [orgUserCountRes, orgUserCount, teamSubscription] = await Promise.all([
const [orgUserCountRes, teamSubscription] = await Promise.all([
pg
.selectFrom('OrganizationUser')
.select(({fn}) => fn.count<number>('id').as('count'))
.where('orgId', '=', orgId)
.where('removedAt', 'is', null)
.where('inactive', '=', false)
.executeTakeFirstOrThrow(),
r
.table('OrganizationUser')
.getAll(orgId, {index: 'orgId'})
.filter({removedAt: null, inactive: false})
.count()
.run(),
manager.getSubscriptionItem(stripeSubscriptionId)
])
if (orgUserCountRes.count !== orgUserCount) {
sendToSentry(new Error('OrganizationUser count mismatch'), {
tags: {
orgId
}
})
}
const {count: orgUserCount} = orgUserCountRes
if (
teamSubscription &&
teamSubscription.quantity !== undefined &&
Expand Down
5 changes: 0 additions & 5 deletions packages/server/database/rethinkDriver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ import NotificationResponseReplied from './types/NotificationResponseReplied'
import NotificationTaskInvolves from './types/NotificationTaskInvolves'
import NotificationTeamArchived from './types/NotificationTeamArchived'
import NotificationTeamInvitation from './types/NotificationTeamInvitation'
import OrganizationUser from './types/OrganizationUser'
import PasswordResetRequest from './types/PasswordResetRequest'
import PushInvitation from './types/PushInvitation'
import RetrospectivePrompt from './types/RetrospectivePrompt'
Expand Down Expand Up @@ -112,10 +111,6 @@ export type RethinkSchema = {
| NotificationMentioned
index: 'userId'
}
OrganizationUser: {
type: OrganizationUser
index: 'orgId' | 'userId'
}
PasswordResetRequest: {
type: PasswordResetRequest
index: 'email' | 'ip' | 'token'
Expand Down
41 changes: 0 additions & 41 deletions packages/server/database/types/OrganizationUser.ts

This file was deleted.

55 changes: 2 additions & 53 deletions packages/server/dataloader/__tests__/isOrgVerified.test.ts
Original file line number Diff line number Diff line change
@@ -1,56 +1,19 @@
/* eslint-env jest */
import {Insertable} from 'kysely'
import {r} from 'rethinkdb-ts'
import {createPGTables, truncatePGTables} from '../../__tests__/common'
import getRethinkConfig from '../../database/getRethinkConfig'
import getRethink from '../../database/rethinkDriver'
import OrganizationUser from '../../database/types/OrganizationUser'
import generateUID from '../../generateUID'
import getKysely from '../../postgres/getKysely'
import {User} from '../../postgres/pg'
import getRedis from '../../utils/getRedis'
import isUserVerified from '../../utils/isUserVerified'
import {OrganizationUser} from '../../postgres/types'
import RootDataLoader from '../RootDataLoader'
jest.mock('../../database/rethinkDriver')
jest.mock('../../utils/isUserVerified')

jest.mocked(getRethink).mockImplementation(() => {
return r as any
})

jest.mocked(isUserVerified).mockImplementation(() => {
return true
})

const TEST_DB = 'getVerifiedOrgIdsTest'

const config = getRethinkConfig()
const testConfig = {
...config,
db: TEST_DB
}

type TestUser = Insertable<User>
const addUsers = async (users: TestUser[]) => {
return getKysely().insertInto('User').values(users).execute()
}

const createTables = async (...tables: string[]) => {
for (const tableName of tables) {
const structure = await r
.db('rethinkdb')
.table('table_config')
.filter({db: config.db, name: tableName})
.run()
await r.tableCreate(tableName).run()
const {indexes} = structure[0]
for (const index of indexes) {
await r.table(tableName).indexCreate(index).run()
}
await r.table(tableName).indexWait().run()
}
}

type TestOrganizationUser = Partial<
Pick<OrganizationUser, 'inactive' | 'joinedAt' | 'removedAt' | 'role' | 'userId'>
>
Expand Down Expand Up @@ -90,35 +53,21 @@ const addOrg = async (
await pg.insertInto('Organization').values(org).execute()
}

await r.table('OrganizationUser').insert(orgUsers).run()
return orgId
}

beforeAll(async () => {
await r.connectPool(testConfig)
const pg = getKysely(TEST_DB)

try {
await r.dbDrop(TEST_DB).run()
} catch (e) {
//ignore
}
await pg.schema.createSchema(TEST_DB).ifNotExists().execute()

await r.dbCreate(TEST_DB).run()
await createPGTables('Organization', 'User', 'SAML', 'SAMLDomain', 'OrganizationUser')
await createTables('OrganizationUser')
})

afterEach(async () => {
await truncatePGTables('Organization', 'User')
await r.table('OrganizationUser').delete().run()
await truncatePGTables('Organization', 'User', 'OrganizationUser')
})

afterAll(async () => {
await r.getPoolMaster()?.drain()
await getKysely().destroy()
getRedis().quit()
})

test('Founder is billing lead', async () => {
Expand Down
Loading
Loading