Skip to content

Commit

Permalink
enhance(backend): 通知がミュート・凍結を考慮するようにする (#13412)
Browse files Browse the repository at this point in the history 
* Never return broken notifications #409

Since notifications are stored in Redis, we can't expect relational
integrity: deleting a user will *not* delete notifications that
mention it.

But if we return notifications with missing bits (a `follow` without a
`user`, for example), the frontend will get very confused and throw an
exception while trying to render them.

This change makes sure we never expose those broken notifications. For
uniformity, I've applied the same logic to notes and roles mentioned
in notifications, even if nobody reported breakage in those cases.

Tested by creating a few types of notifications with a `notifierId`,
then deleting their user.

(cherry picked from commit 421f8d4)

* Update Changelog

* Update CHANGELOG.md

* enhance: 通知がミュートを考慮するようにする

* enhance: 通知が凍結も考慮するようにする

* fix: notifierIdがない通知が消えてしまう問題

* Add tests (通知がミュートを考慮しているかどうか)

* fix: notifierIdがない通知が消えてしまう問題 (grouped)

* Remove unused import

* Fix: typo

* Revert "enhance: 通知が凍結も考慮するようにする"

This reverts commit b1e57e5.

* Revert API handling

* Remove unused imports

* enhance: Check if notifierId is valid in NotificationEntityService

* 通知作成時にpackしてnullになったらあとの処理をやめる

* Remove duplication of valid notifier check

* add filter notification is not null

* Revert "Remove duplication of valid notifier check"

This reverts commit 239a695.

* Improve performance

* Fix packGrouped

* Refactor: 判定部分を共通化

* Fix condition

* use isNotNull

* Update CHANGELOG.md

* filterの改善

* Refactor: DONT REPEAT YOURSELF
Note: GroupedNotificationはNotificationの拡張なのでその例外だけ書けば基本的に共通の処理になり複雑な個別の処理は増えにくいと思われる

* Add groupedNotificationTypes

* Update misskey-js typedef

* Refactor: less sql calls

* refactor

* clean up

* filter notes to mark as read

* packed noteがmapなのでそちらを使う

* if (notesToRead.size > 0)

* if (notes.length === 0) return;

* fix

* Revert "if (notes.length === 0) return;"

This reverts commit 22e2324.

* 🎨

* console.error

* err

* remove try-catch

* 不要なジェネリクスを除去

* Revert  (既読処理をpack内で行うものを元に戻す)

* Clean

* Update packages/backend/src/core/entities/NotificationEntityService.ts

* Update packages/backend/src/core/entities/NotificationEntityService.ts

* Update packages/backend/src/core/entities/NotificationEntityService.ts

* Update packages/backend/src/core/entities/NotificationEntityService.ts

* Update packages/backend/src/core/NotificationService.ts

* Clean

---------

Co-authored-by: dakkar <dakkar@thenautilus.net>
Co-authored-by: kakkokari-gtyih <daisho7308+f@gmail.com>
Co-authored-by: かっこかり <67428053+kakkokari-gtyih@users.noreply.github.com>
Co-authored-by: tamaina <tamaina@hotmail.co.jp>
Co-authored-by: syuilo <4439005+syuilo@users.noreply.github.com>
  • Loading branch information
@tai-cha @dakkar
@kakkokari-gtyih @tamaina @syuilo
6 people authored Feb 28, 2024
1 parent 29350c9 commit 5f43c2f
Show file tree
Hide file tree
Showing 8 changed files with 356 additions and 129 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
## 202x.x.x (unreleased)

### General
- 通知がミュート、凍結を考慮するようになりました
- Enhance: サーバーごとにモデレーションノートを残せるように
- Enhance: コンディショナルロールの条件に「マニュアルロールへのアサイン」を追加
- Enhance: 通知の受信設定に「フォロー中またはフォロワー」を追加
Expand All @@ -30,6 +31,8 @@

### Server
- Fix: nodeinfoにenableMcaptchaとenableTurnstileが無いのを修正
- Fix: 破損した通知をクライアントに送信しないように
* 通知欄が無限にリロードされる問題が改善する可能性があります
- エンドポイント`flash/update`の`flashId`以外のパラメータは必須ではなくなりました
- Fix: 禁止キーワードを含むノートがDelayed Queueに追加されて再処理される問題を修正
- Fix: 自分がフォローしていないアカウントのフォロワー限定ノートが閲覧できることがある問題を修正
Expand Down
2 changes: 2 additions & 0 deletions packages/backend/src/core/NotificationService.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,8 @@ export class NotificationService implements OnApplicationShutdown {

const packed = await this.notificationEntityService.pack(notification, notifieeId, {});

if (packed == null) return null;

// Publish notification event
this.globalEventService.publishMainStream(notifieeId, 'notification', packed);

Expand Down
269 changes: 152 additions & 117 deletions packages/backend/src/core/entities/NotificationEntityService.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@ import type { MiNote } from '@/models/Note.js';
import type { Packed } from '@/misc/json-schema.js';
import { bindThis } from '@/decorators.js';
import { isNotNull } from '@/misc/is-not-null.js';
import { FilterUnionByProperty, notificationTypes } from '@/types.js';
import { FilterUnionByProperty, groupedNotificationTypes } from '@/types.js';
import { CacheService } from '@/core/CacheService.js';
import { RoleEntityService } from './RoleEntityService.js';
import type { OnModuleInit } from '@nestjs/common';
import type { UserEntityService } from './UserEntityService.js';
import type { NoteEntityService } from './NoteEntityService.js';

const NOTE_REQUIRED_NOTIFICATION_TYPES = new Set(['note', 'mention', 'reply', 'renote', 'quote', 'reaction', 'pollEnded'] as (typeof notificationTypes[number])[]);
const NOTE_REQUIRED_GROUPED_NOTIFICATION_TYPES = new Set(['note', 'mention', 'reply', 'renote', 'renote:grouped', 'quote', 'reaction', 'reaction:grouped', 'pollEnded']);
const NOTE_REQUIRED_NOTIFICATION_TYPES = new Set(['note', 'mention', 'reply', 'renote', 'renote:grouped', 'quote', 'reaction', 'reaction:grouped', 'pollEnded'] as (typeof groupedNotificationTypes[number])[]);

@Injectable()
export class NotificationEntityService implements OnModuleInit {
Expand All @@ -41,6 +41,8 @@ export class NotificationEntityService implements OnModuleInit {
@Inject(DI.followRequestsRepository)
private followRequestsRepository: FollowRequestsRepository,

private cacheService: CacheService,

//private userEntityService: UserEntityService,
//private noteEntityService: NoteEntityService,
) {
Expand All @@ -52,138 +54,61 @@ export class NotificationEntityService implements OnModuleInit {
this.roleEntityService = this.moduleRef.get('RoleEntityService');
}

@bindThis
public async pack(
src: MiNotification,
/**
* 通知をパックする共通処理
*/
async #packInternal <T extends MiNotification | MiGroupedNotification> (
src: T,
meId: MiUser['id'],
// eslint-disable-next-line @typescript-eslint/ban-types
options: {

checkValidNotifier?: boolean;
},
hint?: {
packedNotes: Map<MiNote['id'], Packed<'Note'>>;
packedUsers: Map<MiUser['id'], Packed<'UserLite'>>;
},
): Promise<Packed<'Notification'>> {
): Promise<Packed<'Notification'> | null> {
const notification = src;
const noteIfNeed = NOTE_REQUIRED_NOTIFICATION_TYPES.has(notification.type) && 'noteId' in notification ? (
hint?.packedNotes != null
? hint.packedNotes.get(notification.noteId)
: this.noteEntityService.pack(notification.noteId, { id: meId }, {
detail: true,
})
) : undefined;
const userIfNeed = 'notifierId' in notification ? (
hint?.packedUsers != null
? hint.packedUsers.get(notification.notifierId)
: this.userEntityService.pack(notification.notifierId, { id: meId })
) : undefined;
const role = notification.type === 'roleAssigned' ? await this.roleEntityService.pack(notification.roleId) : undefined;

return await awaitAll({
id: notification.id,
createdAt: new Date(notification.createdAt).toISOString(),
type: notification.type,
userId: 'notifierId' in notification ? notification.notifierId : undefined,
...(userIfNeed != null ? { user: userIfNeed } : {}),
...(noteIfNeed != null ? { note: noteIfNeed } : {}),
...(notification.type === 'reaction' ? {
reaction: notification.reaction,
} : {}),
...(notification.type === 'roleAssigned' ? {
role: role,
} : {}),
...(notification.type === 'achievementEarned' ? {
achievement: notification.achievement,
} : {}),
...(notification.type === 'app' ? {
body: notification.customBody,
header: notification.customHeader,
icon: notification.customIcon,
} : {}),
});
}

@bindThis
public async packMany(
notifications: MiNotification[],
meId: MiUser['id'],
) {
if (notifications.length === 0) return [];

let validNotifications = notifications;

const noteIds = validNotifications.map(x => 'noteId' in x ? x.noteId : null).filter(isNotNull);
const notes = noteIds.length > 0 ? await this.notesRepository.find({
where: { id: In(noteIds) },
relations: ['user', 'reply', 'reply.user', 'renote', 'renote.user'],
}) : [];
const packedNotesArray = await this.noteEntityService.packMany(notes, { id: meId }, {
detail: true,
});
const packedNotes = new Map(packedNotesArray.map(p => [p.id, p]));

validNotifications = validNotifications.filter(x => !('noteId' in x) || packedNotes.has(x.noteId));
if (options.checkValidNotifier !== false && !(await this.#isValidNotifier(notification, meId))) return null;

const userIds = validNotifications.map(x => 'notifierId' in x ? x.notifierId : null).filter(isNotNull);
const users = userIds.length > 0 ? await this.usersRepository.find({
where: { id: In(userIds) },
}) : [];
const packedUsersArray = await this.userEntityService.packMany(users, { id: meId });
const packedUsers = new Map(packedUsersArray.map(p => [p.id, p]));

// 既に解決されたフォローリクエストの通知を除外
const followRequestNotifications = validNotifications.filter((x): x is FilterUnionByProperty<MiGroupedNotification, 'type', 'receiveFollowRequest'> => x.type === 'receiveFollowRequest');
if (followRequestNotifications.length > 0) {
const reqs = await this.followRequestsRepository.find({
where: { followerId: In(followRequestNotifications.map(x => x.notifierId)) },
});
validNotifications = validNotifications.filter(x => (x.type !== 'receiveFollowRequest') || reqs.some(r => r.followerId === x.notifierId));
}

return await Promise.all(validNotifications.map(x => this.pack(x, meId, {}, {
packedNotes,
packedUsers,
})));
}

@bindThis
public async packGrouped(
src: MiGroupedNotification,
meId: MiUser['id'],
// eslint-disable-next-line @typescript-eslint/ban-types
options: {

},
hint?: {
packedNotes: Map<MiNote['id'], Packed<'Note'>>;
packedUsers: Map<MiUser['id'], Packed<'UserLite'>>;
},
): Promise<Packed<'Notification'>> {
const notification = src;
const noteIfNeed = NOTE_REQUIRED_GROUPED_NOTIFICATION_TYPES.has(notification.type) && 'noteId' in notification ? (
const needsNote = NOTE_REQUIRED_NOTIFICATION_TYPES.has(notification.type) && 'noteId' in notification;
const noteIfNeed = needsNote ? (
hint?.packedNotes != null
? hint.packedNotes.get(notification.noteId)
: this.noteEntityService.pack(notification.noteId, { id: meId }, {
detail: true,
})
) : undefined;
const userIfNeed = 'notifierId' in notification ? (
// if the note has been deleted, don't show this notification
if (needsNote && !noteIfNeed) return null;

const needsUser = 'notifierId' in notification;
const userIfNeed = needsUser ? (
hint?.packedUsers != null
? hint.packedUsers.get(notification.notifierId)
: this.userEntityService.pack(notification.notifierId, { id: meId })
) : undefined;
// if the user has been deleted, don't show this notification
if (needsUser && !userIfNeed) return null;

// #region Grouped notifications
if (notification.type === 'reaction:grouped') {
const reactions = await Promise.all(notification.reactions.map(async reaction => {
const reactions = (await Promise.all(notification.reactions.map(async reaction => {
const user = hint?.packedUsers != null
? hint.packedUsers.get(reaction.userId)!
: await this.userEntityService.pack(reaction.userId, { id: meId });
return {
user,
reaction: reaction.reaction,
};
}));
}))).filter(r => isNotNull(r.user));
// if all users have been deleted, don't show this notification
if (reactions.length === 0) {
return null;
}

return await awaitAll({
id: notification.id,
createdAt: new Date(notification.createdAt).toISOString(),
Expand All @@ -192,14 +117,19 @@ export class NotificationEntityService implements OnModuleInit {
reactions,
});
} else if (notification.type === 'renote:grouped') {
const users = await Promise.all(notification.userIds.map(userId => {
const users = (await Promise.all(notification.userIds.map(userId => {
const packedUser = hint?.packedUsers != null ? hint.packedUsers.get(userId) : null;
if (packedUser) {
return packedUser;
}

return this.userEntityService.pack(userId, { id: meId });
}));
}))).filter(isNotNull);
// if all users have been deleted, don't show this notification
if (users.length === 0) {
return null;
}

return await awaitAll({
id: notification.id,
createdAt: new Date(notification.createdAt).toISOString(),
Expand All @@ -208,8 +138,14 @@ export class NotificationEntityService implements OnModuleInit {
users,
});
}
// #endregion

const role = notification.type === 'roleAssigned' ? await this.roleEntityService.pack(notification.roleId) : undefined;
const needsRole = notification.type === 'roleAssigned';
const role = needsRole ? await this.roleEntityService.pack(notification.roleId) : undefined;
// if the role has been deleted, don't show this notification
if (needsRole && !role) {
return null;
}

return await awaitAll({
id: notification.id,
Expand All @@ -235,15 +171,16 @@ export class NotificationEntityService implements OnModuleInit {
});
}

@bindThis
public async packGroupedMany(
notifications: MiGroupedNotification[],
async #packManyInternal <T extends MiNotification | MiGroupedNotification> (
notifications: T[],
meId: MiUser['id'],
) {
): Promise<T[]> {
if (notifications.length === 0) return [];

let validNotifications = notifications;

validNotifications = await this.#filterValidNotifier(validNotifications, meId);

const noteIds = validNotifications.map(x => 'noteId' in x ? x.noteId : null).filter(isNotNull);
const notes = noteIds.length > 0 ? await this.notesRepository.find({
where: { id: In(noteIds) },
Expand All @@ -269,17 +206,115 @@ export class NotificationEntityService implements OnModuleInit {
const packedUsers = new Map(packedUsersArray.map(p => [p.id, p]));

// 既に解決されたフォローリクエストの通知を除外
const followRequestNotifications = validNotifications.filter((x): x is FilterUnionByProperty<MiGroupedNotification, 'type', 'receiveFollowRequest'> => x.type === 'receiveFollowRequest');
const followRequestNotifications = validNotifications.filter((x): x is FilterUnionByProperty<T, 'type', 'receiveFollowRequest'> => x.type === 'receiveFollowRequest');
if (followRequestNotifications.length > 0) {
const reqs = await this.followRequestsRepository.find({
where: { followerId: In(followRequestNotifications.map(x => x.notifierId)) },
});
validNotifications = validNotifications.filter(x => (x.type !== 'receiveFollowRequest') || reqs.some(r => r.followerId === x.notifierId));
}

return await Promise.all(validNotifications.map(x => this.packGrouped(x, meId, {}, {
packedNotes,
packedUsers,
})));
const packPromises = validNotifications.map(x => {
return this.pack(
x,
meId,
{ checkValidNotifier: false },
{ packedNotes, packedUsers },
);
});

return (await Promise.all(packPromises)).filter(isNotNull);
}

@bindThis
public async pack(
src: MiNotification | MiGroupedNotification,
meId: MiUser['id'],
// eslint-disable-next-line @typescript-eslint/ban-types
options: {
checkValidNotifier?: boolean;
},
hint?: {
packedNotes: Map<MiNote['id'], Packed<'Note'>>;
packedUsers: Map<MiUser['id'], Packed<'UserLite'>>;
},
): Promise<Packed<'Notification'> | null> {
return await this.#packInternal(src, meId, options, hint);
}

@bindThis
public async packMany(
notifications: MiNotification[],
meId: MiUser['id'],
): Promise<MiNotification[]> {
return await this.#packManyInternal(notifications, meId);
}

@bindThis
public async packGroupedMany(
notifications: MiGroupedNotification[],
meId: MiUser['id'],
): Promise<MiGroupedNotification[]> {
return await this.#packManyInternal(notifications, meId);
}

/**
* notifierが存在するか、ミュートされていないか、サスペンドされていないかを確認するvalidator
*/
#validateNotifier <T extends MiNotification | MiGroupedNotification> (
notification: T,
userIdsWhoMeMuting: Set<MiUser['id']>,
userMutedInstances: Set<string>,
notifiers: MiUser[],
): boolean {
if (!('notifierId' in notification)) return true;
if (userIdsWhoMeMuting.has(notification.notifierId)) return false;

const notifier = notifiers.find(x => x.id === notification.notifierId) ?? null;

if (notifier == null) return false;
if (notifier.host && userMutedInstances.has(notifier.host)) return false;

if (notifier.isSuspended) return false;

return true;
}

/**
* notifierが存在するか、ミュートされていないか、サスペンドされていないかを実際に確認する
*/
async #isValidNotifier(
notification: MiNotification | MiGroupedNotification,
meId: MiUser['id'],
): Promise<boolean> {
return (await this.#filterValidNotifier([notification], meId)).length === 1;
}

/**
* notifierが存在するか、ミュートされていないか、サスペンドされていないかを実際に複数確認する
*/
async #filterValidNotifier <T extends MiNotification | MiGroupedNotification> (
notifications: T[],
meId: MiUser['id'],
): Promise<T[]> {
const [
userIdsWhoMeMuting,
userMutedInstances,
] = await Promise.all([
this.cacheService.userMutingsCache.fetch(meId),
this.cacheService.userProfileCache.fetch(meId).then(p => new Set(p.mutedInstances)),
]);

const notifierIds = notifications.map(notification => 'notifierId' in notification ? notification.notifierId : null).filter(isNotNull);
const notifiers = notifierIds.length > 0 ? await this.usersRepository.find({
where: { id: In(notifierIds) },
}) : [];

const filteredNotifications = ((await Promise.all(notifications.map(async (notification) => {
const isValid = this.#validateNotifier(notification, userIdsWhoMeMuting, userMutedInstances, notifiers);
return isValid ? notification : null;
}))) as [T | null] ).filter(isNotNull);

return filteredNotifications;
}
}
Loading

0 comments on commit 5f43c2f

Please sign in to comment.