ai-robots-txt in Astro

[oobleck]

Thought I'd share how I'm using this project on my Astro site, in case it's helpful to anybody else. I created 2 files in my project's src/ folder: src/robots.txt.ts and src/ai.txt.ts. Each has the same contents, except I change which file in this project it pulls:

src/robots.txt.ts

export async function GET() {
  const request = await fetch("https://raw.githubusercontent.com/ai-robots-txt/ai.robots.txt/main/robots.txt");
  const robotsTxt = await request.text();
  return new Response(`# Pulled from https://github.com/ai-robots-txt/ai.robots.txt
# ${new Date().toUTCString()}

${robotsTxt}`);
};

src/ai.txt.ts

export async function GET() {
  const request = await fetch("https://raw.githubusercontent.com/ai-robots-txt/ai.robots.txt/main/ai.txt");
  const result = await request.text();
  return new Response(`# Pulled from https://github.com/ai-robots-txt/ai.txt
# ${new Date().toUTCString()}

${result}`);
}