Bot downloads YT videos with extreme compression

[R32fanboy]

The bot downloads videos (from youtube specifically) with a resolution of 640x480, no matter the length of the video and compresses the video to fit within the confines of a 74.3Mb cap (observed via passing multiple YT URLs to the bot). The BotAPI server is confirmed to be running and am using the most up-to-date image for the downloader.

[vaaski]

Hey, I'm about 90% sure that that is a limitation on YouTube's end. I made the bot so that it downloads the best format that contains both video and audio, and historically YouTube served that up to 720p. From what I've seen recently they've lowered that.

To support higher resolutions, the bot would have to download the separate video and audio streams and either re-encode oder remux the two. Re-encoding takes a bunch of resources and time and remuxing behaves weirdly with some players and platforms in my experience so I'm not sure if I want to do either.

If you can provide an example link I can show you what formats YouTube makes available. Or run yt-dlp -F <video url> yourself to list them.

[R32fanboy]

In my testing, the settings

-f "bv*[filesize<2000M]+ba" --remux-video mp4 -S "height:1080"

serve up a good resolution with a filesize under the self-hosted botAPI limit, and remuxing doesn't seem to take that much CPU time or storage.

[vaaski]

Yeah remuxing is quick but sometimes behaves weird in some players. Unfortunately introducing runtime options per-user to the bot would make it a bunch more complicated and I'm not sure if remuxing should be the default. I am not completely against it either though.

Maybe as a global configuration option on the container?

[R32fanboy]

That last option would definitely be a welcome change, if possible!

[vaaski]

I was beginning to implement this, but I realized that for regular downloads, the bot doesn't even download anything to the server, but hands the direct stream URL directly to telegram, effectively skipping a step. This was the hacky trick I used to make this bot a lot faster. So I think I won't implement this because it would defeat the point of having a fast downloading solution as it would add multiple points of slowdowns.

There's also some throttling YouTube seems to apply to certain formats, so a download that takes just a couple of seconds on the current release, took a solid 4 minutes to download on "max" settings. The resulting file is 566MB instead of 30MB and their resolutions are 2160p and 360p respectively.

Furthermore, the resulting video looks fine but plays without audio on my Windows 10 desktop and looks stretched but with working audio on my iPhone. This is what I meant by weird playback behavior due to remuxing.

Here is the modified src/index.ts based on 22699a1 for reference

import { downloadFromInfo, getInfo, streamFromInfo } from "@resync-tv/yt-dlp"
import { InputFile } from "grammy"
import { deleteMessage, errorMessage } from "./bot-util"
import { t, tiktokArgs } from "./constants"
import { ADMIN_ID, WHITELISTED_IDS } from "./environment"
import { getThumbnail, urlMatcher } from "./media-util"
import { Queue } from "./queue"
import { bot } from "./setup"
import { removeHashtagsMentions } from "./textutil"
import { translateText } from "./translate"
import { Updater } from "./updater"

const queue = new Queue()
const updater = new Updater()

bot.use(async (ctx, next) => {
	if (ctx.chat?.type !== "private") return

	await next()
})

//? filter out messages from non-whitelisted users
bot.on("message:text", async (ctx, next) => {
	if (WHITELISTED_IDS.length === 0) return await next()
	if (WHITELISTED_IDS.includes(ctx.from?.id)) return await next()

	const deniedResponse = await ctx.replyWithHTML(t.deniedMessage, {
		link_preview_options: { is_disabled: true },
	})

	await Promise.all([
		(async () => {
			if (ctx.from.language_code && ctx.from.language_code !== "en") {
				const translated = await translateText(
					t.deniedMessage,
					ctx.from.language_code,
				)
				if (translated === t.deniedMessage) return
				await bot.api.editMessageText(
					ctx.chat.id,
					deniedResponse.message_id,
					translated,
					{ parse_mode: "HTML", link_preview_options: { is_disabled: true } },
				)
			}
		})(),
		(async () => {
			const forwarded = await ctx.forwardMessage(ADMIN_ID, {
				disable_notification: true,
			})
			await bot.api.setMessageReaction(
				forwarded.chat.id,
				forwarded.message_id,
				[{ type: "emoji", emoji: "🖕" }],
			)
		})(),
	])
})

bot.on("message:text", async (ctx, next) => {
	if (updater.updating === false) return await next()

	const maintenanceNotice = await ctx.replyWithHTML(t.maintenanceNotice)
	await updater.updating

	await deleteMessage(maintenanceNotice)
	await next()
})

bot.on("message:text").on("::url", async (ctx, next) => {
	const [url] = ctx.entities("url")
	if (!url) return await next()

	const processingMessage = await ctx.replyWithHTML(t.processing, {
		disable_notification: true,
	})

	if (ctx.chat.id !== ADMIN_ID) {
		ctx
			.forwardMessage(ADMIN_ID, { disable_notification: true })
			.then(async (forwarded) => {
				await bot.api.setMessageReaction(
					forwarded.chat.id,
					forwarded.message_id,
					[{ type: "emoji", emoji: "🤝" }],
				)
			})
	}

	queue.add(async () => {
		try {
			const isTiktok = urlMatcher(url.text, "tiktok.com")
			const isYouTubeMusic = urlMatcher(url.text, "music.youtube.com")
			const additionalArgs = isTiktok ? tiktokArgs : []

			const info = await getInfo(url.text, [
				"-f",
				"(bestvideo*+bestaudio/best)[filesize<?2000M][filesize_approx<2000M]",
				"--remux-video",
				"mp4",
				"--no-playlist",
				...additionalArgs,
			])

			const [download] = info.requested_downloads ?? []
			if (!download?.url && !download?.requested_formats) {
				throw new Error("No download available")
			}

			const title = removeHashtagsMentions(info.title)

			if (download.vcodec !== "none" && !isYouTubeMusic) {
				let video: InputFile | string

				if (isTiktok) {
					const stream = downloadFromInfo(info, "-")
					video = new InputFile(stream.stdout, title)
				} else if (download.url) {
					video = new InputFile({ url: download.url }, title)
				} else {
					const stream = downloadFromInfo(info, "-", [
						"-f",
						"(bestvideo*+bestaudio/best)[filesize<?2000M][filesize_approx<2000M]",
						"--remux-video",
						"mp4",
						"--no-playlist",
					])
					video = new InputFile(stream.stdout, title)
				}

				await ctx.replyWithVideo(video, {
					caption: title,
					supports_streaming: true,
					duration: info.duration,
					reply_parameters: {
						message_id: ctx.message?.message_id,
						allow_sending_without_reply: true,
					},
				})
			} else if (download.acodec !== "none") {
				const stream = downloadFromInfo(info, "-", [
					"-x",
					"--audio-format",
					"mp3",
				])
				const audio = new InputFile(stream.stdout)

				await ctx.replyWithAudio(audio, {
					caption: title,
					performer: info.uploader,
					title: info.title,
					thumbnail: getThumbnail(info.thumbnails),
					duration: info.duration,
					reply_parameters: {
						message_id: ctx.message?.message_id,
						allow_sending_without_reply: true,
					},
				})
			} else {
				throw new Error("No download available")
			}
		} catch (error) {
			return error instanceof Error
				? errorMessage(ctx.chat, error.message)
				: errorMessage(ctx.chat, `Couldn't download ${url}`)
		} finally {
			await deleteMessage(processingMessage)
		}
	})
})

bot.on("message:text", async (ctx) => {
	const response = await ctx.replyWithHTML(t.urlReminder)

	if (ctx.from.language_code && ctx.from.language_code !== "en") {
		const translated = await translateText(
			t.urlReminder,
			ctx.from.language_code,
		)
		if (translated === t.urlReminder) return
		await bot.api.editMessageText(
			ctx.chat.id,
			response.message_id,
			translated,
			{ parse_mode: "HTML", link_preview_options: { is_disabled: true } },
		)
	}
})

If you really want a full-quality download I'd recommend just using yt-dlp, any yt-dlp GUI wrapper or cobalt.tools.

The video in question: https://youtu.be/OqNUDqjKT_A