Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

公開投稿以外の配送を制限する機能 #574

Merged
merged 54 commits into from
Jan 11, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
e089a21
wip
kozakura913 Dec 18, 2024
56e4ff0
wip
kozakura913 Dec 18, 2024
bc4b511
wip
kozakura913 Dec 18, 2024
dae169e
wip
kozakura913 Dec 18, 2024
de948cd
clear cache
kozakura913 Dec 18, 2024
9a6c144
制限対象は投稿のみ
kozakura913 Dec 18, 2024
7c4a98e
wip
kozakura913 Dec 18, 2024
4e4251a
log
kozakura913 Dec 18, 2024
7742d45
wip
kozakura913 Dec 18, 2024
329a9c3
log
kozakura913 Dec 18, 2024
d3fcbae
wip
kozakura913 Dec 18, 2024
57889ec
wip
kozakura913 Dec 18, 2024
5ae83ca
wip
kozakura913 Dec 18, 2024
06639ab
log
kozakura913 Dec 18, 2024
9f98e33
log
kozakura913 Dec 18, 2024
6041784
wip
kozakura913 Dec 18, 2024
5199ee3
log
kozakura913 Dec 18, 2024
fd7e33e
log
kozakura913 Dec 18, 2024
a2afe12
log
kozakura913 Dec 18, 2024
6999e1c
ui
kozakura913 Dec 18, 2024
620a9fb
fix
kozakura913 Dec 18, 2024
66adb77
変更が無い時はスキップ
kozakura913 Dec 18, 2024
e764dbd
fix
kozakura913 Dec 18, 2024
a72aac1
fix
kozakura913 Dec 18, 2024
29998a2
モデログ
kozakura913 Dec 19, 2024
bdbc2eb
fix
kozakura913 Dec 19, 2024
f6cee88
Merge branch 'develop' of git@github.com:kozakura913/yojo-art.git int…
kozakura913 Jan 8, 2025
d80f3f1
Merge branch 'develop' into quarantined_instances
kozakura913 Jan 11, 2025
1eb8438
add:test
kozakura913 Jan 11, 2025
0cad783
wip
kozakura913 Jan 11, 2025
0509239
fix
kozakura913 Jan 11, 2025
453e5cb
wip
kozakura913 Jan 11, 2025
eb49d45
fix
kozakura913 Jan 11, 2025
f2e6a71
wip
kozakura913 Jan 11, 2025
e9465d3
wip
kozakura913 Jan 11, 2025
bcde09b
wip
kozakura913 Jan 11, 2025
741c4e6
wip
kozakura913 Jan 11, 2025
a395cbf
wip
kozakura913 Jan 11, 2025
7c8b91b
isQuarantineLimited
kozakura913 Jan 11, 2025
586f3ad
fix
kozakura913 Jan 11, 2025
c006a09
rm
kozakura913 Jan 11, 2025
95730a6
block
kozakura913 Jan 11, 2025
1a83fea
undo block
kozakura913 Jan 11, 2025
289c4de
wip
kozakura913 Jan 11, 2025
fc0a65c
Revert "wip"
kozakura913 Jan 11, 2025
7929255
wip
kozakura913 Jan 11, 2025
f408bf8
assert.rejects
kozakura913 Jan 11, 2025
2325f6e
fixme
kozakura913 Jan 11, 2025
528f7a4
changelog
kozakura913 Jan 11, 2025
322fb30
fix?
kozakura913 Jan 11, 2025
48810ca
Revert "fix?"
kozakura913 Jan 11, 2025
8d50aa4
wip
kozakura913 Jan 11, 2025
65d1871
add
kozakura913 Jan 11, 2025
b1db96e
pnpm run build-cherrypick-js-with-types
kozakura913 Jan 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG_YOJO.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

### General
- Feat: システムユーザーのファイル一覧を追加 [#595](https://github.com/yojo-art/cherrypick/pull/595)
- Feat: 公開投稿以外の配送を制限する機能 [#574](https://github.com/yojo-art/cherrypick/pull/574)
- Enhance: Fedibird形式の絵文字情報連合に対応 [#604](https://github.com/yojo-art/cherrypick/pull/604)
- 以下の情報が対応ソフト間で連合されます
- キーワード
Expand Down
3 changes: 3 additions & 0 deletions locales/ja-JP.yml
Original file line number Diff line number Diff line change
Expand Up @@ -331,6 +331,7 @@ stopActivityDelivery: "アクティビティの配送を停止"
blockThisInstance: "このサーバーをブロック"
silenceThisInstance: "サーバーをサイレンス"
mediaSilenceThisInstance: "サーバーをメディアサイレンス"
quarantineThisInstance: "サーバーに公開投稿のみ配送"
operations: "操作"
software: "ソフトウェア"
version: "バージョン"
Expand Down Expand Up @@ -3102,6 +3103,8 @@ _moderationLogTypes:
deleteGalleryPost: "ギャラリーの投稿を削除"
updateOfficialTags: "公式タグ一覧を更新"
promoteQueue: "ジョブキューを再試行"
quarantineRemoteInstance: "公開投稿のみ配送に制限"
unquarantineRemoteInstance: "公開投稿のみ配送を解除"

_fileViewer:
title: "ファイルの詳細"
Expand Down
16 changes: 16 additions & 0 deletions packages/backend/migration/1734500881453-AddQuarantineLimited.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project, yojo-art team
* SPDX-License-Identifier: AGPL-3.0-only
*/

export class AddQuarantineLimited1734500881453 {
name = 'AddQuarantineLimited1734500881453'

async up(queryRunner) {
await queryRunner.query(`ALTER TABLE "instance" ADD "quarantineLimited" boolean NOT NULL DEFAULT false`);
}

async down(queryRunner) {
await queryRunner.query(`ALTER TABLE "instance" DROP COLUMN "quarantineLimited"`);
}
}
1 change: 1 addition & 0 deletions packages/backend/src/core/GlobalEventService.ts
Original file line number Diff line number Diff line change
Expand Up @@ -288,6 +288,7 @@ export interface InternalEventTypes {
unmute: { muterId: MiUser['id']; muteeId: MiUser['id']; };
userListMemberAdded: { userListId: MiUserList['id']; memberId: MiUser['id']; };
userListMemberRemoved: { userListId: MiUserList['id']; memberId: MiUser['id']; };
clearQuarantinedHostsCache: string;
}

type EventTypesToEventPayload<T> = EventUnionFromDictionary<UndefinedAsNullAll<SerializedAll<T>>>;
Expand Down
29 changes: 28 additions & 1 deletion packages/backend/src/core/QueueService.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

import { randomUUID } from 'node:crypto';
import { Inject, Injectable } from '@nestjs/common';
import type { IActivity } from '@/core/activitypub/type.js';
import { isAnnounce, isBlock, isPost, isUndo, type IActivity } from '@/core/activitypub/type.js';
import type { MiDriveFile } from '@/models/DriveFile.js';
import type { MiAbuseUserReport } from '@/models/AbuseUserReport.js';
import type { MiWebhook, webhookEventTypes } from '@/models/Webhook.js';
Expand Down Expand Up @@ -107,6 +107,31 @@ export class QueueService {
});
}

private isPublicContent(content: IActivity) {
let toPublicOnly = false;
if (isAnnounce(content)) {
penginn-net marked this conversation as resolved.
Show resolved Hide resolved
toPublicOnly = true;
}
if (isBlock(content)) {
toPublicOnly = true;
}
if (typeof content.object !== 'string') {
if (isUndo(content)) {
if (isBlock(content.object)) {
toPublicOnly = true;
}
}
if (isPost(content.object)) {
toPublicOnly = true;
}
}
if (toPublicOnly) {
return String(content.to) === 'https://www.w3.org/ns/activitystreams#Public' || String(content.cc) === 'https://www.w3.org/ns/activitystreams#Public';
} else {
return true;
}
}

@bindThis
public deliver(user: ThinUser, content: IActivity | null, to: string | null, isSharedInbox: boolean) {
if (content == null) return null;
Expand All @@ -123,6 +148,7 @@ export class QueueService {
digest,
to,
isSharedInbox,
isPublicContent: this.isPublicContent(content),
};

return this.deliverQueue.add(to, data, {
Expand Down Expand Up @@ -165,6 +191,7 @@ export class QueueService {
digest,
to: d[0],
isSharedInbox: d[1],
isPublicContent: this.isPublicContent(content),
},
opts,
})));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ export class InstanceEntityService {
latestRequestReceivedAt: instance.latestRequestReceivedAt ? instance.latestRequestReceivedAt.toISOString() : null,
moderationNote: iAmModerator ? instance.moderationNote : null,
reversiVersion: instance.reversiVersion,
isQuarantineLimited: instance.quarantineLimited,
};
}

Expand Down
8 changes: 8 additions & 0 deletions packages/backend/src/models/Instance.ts
Original file line number Diff line number Diff line change
Expand Up @@ -163,4 +163,12 @@ export class MiInstance {
length: 64, nullable: true,
})
public reversiVersion: string | null;
/**
* このインスタンスへの配送制限
*/
@Index()
@Column('boolean', {
default: false,
})
public quarantineLimited: boolean;
}
Original file line number Diff line number Diff line change
Expand Up @@ -124,5 +124,9 @@ export const packedFederationInstanceSchema = {
type: 'string',
optional: true, nullable: true,
},
isQuarantineLimited: {
type: 'boolean',
optional: false, nullable: false,
},
},
} as const;
39 changes: 39 additions & 0 deletions packages/backend/src/queue/processors/DeliverProcessorService.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import { Inject, Injectable } from '@nestjs/common';
import * as Bull from 'bullmq';
import { Not } from 'typeorm';
import * as Redis from 'ioredis';
import { DI } from '@/di-symbols.js';
import type { InstancesRepository, MiMeta } from '@/models/_.js';
import type Logger from '@/logger.js';
Expand All @@ -20,13 +21,15 @@ import FederationChart from '@/core/chart/charts/federation.js';
import { StatusError } from '@/misc/status-error.js';
import { UtilityService } from '@/core/UtilityService.js';
import { bindThis } from '@/decorators.js';
import { GlobalEvents } from '@/core/GlobalEventService.js';
import { QueueLoggerService } from '../QueueLoggerService.js';
import type { DeliverJobData } from '../types.js';

@Injectable()
export class DeliverProcessorService {
private logger: Logger;
private suspendedHostsCache: MemorySingleCache<MiInstance[]>;
private quarantinedHostsCache: MemorySingleCache<MiInstance[]>;
private latest: string | null;

constructor(
Expand All @@ -35,6 +38,8 @@ export class DeliverProcessorService {

@Inject(DI.instancesRepository)
private instancesRepository: InstancesRepository,
@Inject(DI.redisForSub)
private redisForSub: Redis.Redis,

private utilityService: UtilityService,
private federatedInstanceService: FederatedInstanceService,
Expand All @@ -47,6 +52,25 @@ export class DeliverProcessorService {
) {
this.logger = this.queueLoggerService.logger.createSubLogger('deliver');
this.suspendedHostsCache = new MemorySingleCache<MiInstance[]>(1000 * 60 * 60); // 1h
this.quarantinedHostsCache = new MemorySingleCache<MiInstance[]>(1000 * 60 * 60); // 1h
this.redisForSub.on('message', this.onMessage);
}

@bindThis
private async onMessage(_: string, data: string): Promise<void> {
const obj = JSON.parse(data);

if (obj.channel === 'internal') {
const { type, body } = obj.message as GlobalEvents['internal']['payload'];
switch (type) {
case 'clearQuarantinedHostsCache': {
this.quarantinedHostsCache.delete();
break;
}
default:
break;
}
}
}

@bindThis
Expand All @@ -70,6 +94,21 @@ export class DeliverProcessorService {
if (suspendedHosts.map(x => x.host).includes(this.utilityService.toPuny(host))) {
return 'skip (suspended)';
}
// isQuarantinedなら中断
let quarantinedHosts = this.quarantinedHostsCache.get();
if (quarantinedHosts == null) {
quarantinedHosts = await this.instancesRepository.find({
where: {
quarantineLimited: true,
},
});
this.quarantinedHostsCache.set(quarantinedHosts);
}
if (!job.data.isPublicContent) {
if (quarantinedHosts.map(x => x.host).includes(this.utilityService.toPuny(host))) {
return 'skip (quarantined)';
}
}

try {
await this.apRequestService.signedPost(job.data.user, job.data.to, job.data.content, job.data.digest);
Expand Down
2 changes: 2 additions & 0 deletions packages/backend/src/queue/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ export type DeliverJobData = {
to: string;
/** whether it is sharedInbox */
isSharedInbox: boolean;
/** Activity is Public */
isPublicContent: boolean;
};

export type InboxJobData = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { UtilityService } from '@/core/UtilityService.js';
import { DI } from '@/di-symbols.js';
import { FederatedInstanceService } from '@/core/FederatedInstanceService.js';
import { ModerationLogService } from '@/core/ModerationLogService.js';
import { GlobalEventService } from '@/core/GlobalEventService.js';

export const meta = {
tags: ['admin'],
Expand All @@ -25,6 +26,7 @@ export const paramDef = {
host: { type: 'string' },
isSuspended: { type: 'boolean' },
moderationNote: { type: 'string' },
isQuarantineLimit: { type: 'boolean' },
},
required: ['host'],
} as const;
Expand All @@ -34,6 +36,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
constructor(
@Inject(DI.instancesRepository)
private instancesRepository: InstancesRepository,
private globalEventService: GlobalEventService,

private utilityService: UtilityService,
private federatedInstanceService: FederatedInstanceService,
Expand All @@ -52,9 +55,22 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
if (ps.isSuspended != null && isSuspendedBefore !== ps.isSuspended) {
suspensionState = ps.isSuspended ? 'manuallySuspended' : 'none';
}
const isQuarantineLimitBefore = instance.quarantineLimited;
let quarantineLimited: undefined | boolean;

if (ps.isQuarantineLimit != null && isQuarantineLimitBefore !== ps.isQuarantineLimit) {
quarantineLimited = ps.isQuarantineLimit;
}
const moderationNoteBefore = instance.moderationNote;
if ((ps.moderationNote === undefined || moderationNoteBefore === ps.moderationNote) && (quarantineLimited === undefined || isQuarantineLimitBefore === quarantineLimited) && (suspensionState === undefined || isSuspendedBefore === ps.isSuspended)) {
//何も変更が無い時はupdateを呼ばない
//呼ぶとエラーが発生する
return;
}

await this.federatedInstanceService.update(instance.id, {
suspensionState,
quarantineLimited,
moderationNote: ps.moderationNote,
});

Expand All @@ -71,6 +87,20 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
});
}
}
if (ps.isQuarantineLimit != null && isQuarantineLimitBefore !== ps.isQuarantineLimit) {
if (ps.isQuarantineLimit) {
this.moderationLogService.log(me, 'quarantineRemoteInstance', {
id: instance.id,
host: instance.host,
});
} else {
this.moderationLogService.log(me, 'unquarantineRemoteInstance', {
id: instance.id,
host: instance.host,
});
}
this.globalEventService.publishInternalEvent('clearQuarantinedHostsCache', '');
}

if (ps.moderationNote != null && instance.moderationNote !== ps.moderationNote) {
this.moderationLogService.log(me, 'updateRemoteInstanceNote', {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ export const paramDef = {
federating: { type: 'boolean', nullable: true },
subscribing: { type: 'boolean', nullable: true },
publishing: { type: 'boolean', nullable: true },
quarantined: { type: 'boolean', nullable: true },
limit: { type: 'integer', minimum: 1, maximum: 100, default: 30 },
offset: { type: 'integer', default: 0 },
sort: {
Expand Down Expand Up @@ -126,6 +127,13 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
query.andWhere('instance.suspensionState = \'none\'');
}
}
if (typeof ps.quarantined === 'boolean') {
if (ps.quarantined) {
query.andWhere('instance.quarantineLimited = TRUE');
} else {
query.andWhere('instance.quarantineLimited = FALSE');
}
}

if (typeof ps.silenced === 'boolean') {
const meta = await this.metaService.fetch(true);
Expand Down
12 changes: 11 additions & 1 deletion packages/backend/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,8 @@ export const moderationLogTypes = [
'deleteGalleryPost',
'updateOfficialTags',
'unsetUserMutualLink',
'quarantineRemoteInstance',
'unquarantineRemoteInstance',
] as const;

export type ModerationLogPayloads = {
Expand Down Expand Up @@ -392,7 +394,15 @@ export type ModerationLogPayloads = {
userId: string;
userUsername: string;
userMutualLinkSections: { name: string | null; mutualLinks: { fileId: string; description: string | null; imgSrc: string; }[]; }[] | []
}
};
quarantineRemoteInstance: {
id: string;
host: string;
};
unquarantineRemoteInstance: {
id: string;
host: string;
};
};

export type Serialized<T> = {
Expand Down
Loading
Loading