Skip to content

Updated sharp docs #1403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion docs/guides/examples/ffmpeg-video-processing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,12 @@ description: "These examples show you how to process videos in various ways usin

import LocalDevelopment from "/snippets/local-development-extensions.mdx";

## Adding the FFmpeg build extension
## Prerequisites

- A project with [Trigger.dev initialized](/quick-start)
- [FFmpeg](https://www.ffmpeg.org/download.html) installed on your machine

### Adding the FFmpeg build extension

To use these example tasks, you'll first need to add our FFmpeg extension to your project configuration like this:

Expand Down
17 changes: 11 additions & 6 deletions docs/guides/examples/puppeteer.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ description: "These examples demonstrate how to use Puppeteer with Trigger.dev."
import LocalDevelopment from "/snippets/local-development-extensions.mdx";
import ScrapingWarning from "/snippets/web-scraping-warning.mdx";

## Prerequisites

- A project with [Trigger.dev initialized](/quick-start)
- [Puppeteer](https://pptr.dev/guides/installation) installed on your machine

## Overview

There are 3 example tasks to follow on this page:
Expand All @@ -15,7 +20,7 @@ There are 3 example tasks to follow on this page:
2. [Generate a PDF from a web page](/guides/examples/puppeteer#generate-a-pdf-from-a-web-page)
3. [Scrape content from a web page](/guides/examples/puppeteer#scrape-content-from-a-web-page)

<ScrapingWarning/>
<ScrapingWarning />

## Build configurations

Expand Down Expand Up @@ -133,7 +138,6 @@ export const puppeteerWebpageToPDF = task({
return { pdfUrl: s3Url };
},
});

```

### Testing your task
Expand All @@ -146,9 +150,10 @@ There's no payload required for this task so you can just click "Run test" from

In this example we use [Puppeteer](https://pptr.dev/) with a [BrowserBase](https://www.browserbase.com/) proxy to scrape the GitHub stars count from the [Trigger.dev](https://trigger.dev) landing page and log it out. See [this list](/guides/examples/puppeteer#proxying) for more proxying services we recommend.

<Note>
When web scraping, you MUST use the technique below which uses a proxy with Puppeteer. Direct scraping without using `browserWSEndpoint` is prohibited and will result in account suspension.
</Note>
<Warning>
When web scraping, you MUST use the technique below which uses a proxy with Puppeteer. Direct
scraping without using `browserWSEndpoint` is prohibited and will result in account suspension.
</Warning>

### Task code

Expand Down Expand Up @@ -209,4 +214,4 @@ Here are a list of proxy services we recommend:
- [Browserless](https://browserless.io/)
- [Oxylabs](https://oxylabs.io/)
- [ScrapingBee](https://scrapingbee.com/)
- [Smartproxy](https://smartproxy.com/)
- [Smartproxy](https://smartproxy.com/)
92 changes: 43 additions & 49 deletions docs/guides/examples/sharp-image-processing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,15 @@ import LocalDevelopment from "/snippets/local-development-extensions.mdx";

## Overview

This task optimizes and watermarks an image using the Sharp library, and then uploads the processed image to R2 storage.
This task processes and watermarks an image using the Sharp library, and then uploads it to R2 storage.

## Adding build configurations
## Prerequisites

- A project with [Trigger.dev initialized](/quick-start)
- The [Sharp](https://sharp.pixelplumbing.com/install) library installed on your machine
- An R2-compatible object storage service, such as [Cloudflare R2](https://developers.cloudflare.com/r2)

## Adding the build configuration

To use this example, you'll first need to add these build settings to your `trigger.config.ts` file:

Expand All @@ -34,22 +40,22 @@ export default defineConfig({

## Key features

- Resizes and rotates an image
- Adds a watermark to the image
- Uploads the processed image to R2 storage
- Resizes a JPEG image to 800x800 pixels
- Adds a watermark to the image, positioned in the bottom-right corner, using a PNG image
- Uploads the processed image to R2 storage

## Task code

```ts trigger/sharp-image-processing.ts
import { PutObjectCommand, S3Client } from "@aws-sdk/client-s3";
import { S3Client } from "@aws-sdk/client-s3";
import { Upload } from "@aws-sdk/lib-storage";
import { logger, task } from "@trigger.dev/sdk/v3";
import fs from "fs/promises";
import fetch from "node-fetch";
import os from "os";
import path from "path";
import sharp from "sharp";

// Initialize R2 client
// Initialize R2 client using your R2 account details
const r2Client = new S3Client({
region: "auto",
endpoint: process.env.R2_ENDPOINT,
Expand All @@ -61,63 +67,51 @@ const r2Client = new S3Client({

export const sharpProcessImage = task({
id: "sharp-process-image",
retry: { maxAttempts: 1 },
run: async (payload: { imageUrl: string; watermarkUrl: string }) => {
const { imageUrl, watermarkUrl } = payload;
const outputPath = path.join(os.tmpdir(), `output_${Date.now()}.jpg`);

// Generate temporary and output file names
const tempDirectory = os.tmpdir();
const outputPath = path.join(tempDirectory, `output_${Date.now()}.jpg`);

// Fetch the image and watermark
const [imageResponse, watermarkResponse] = await Promise.all([
fetch(imageUrl),
fetch(watermarkUrl),
]);
const imageBuffer = await imageResponse.arrayBuffer();
const watermarkBuffer = await watermarkResponse.arrayBuffer();

// Optimize the image using Sharp
await sharp(Buffer.from(imageBuffer))
.rotate(90) // Rotate the image by 90 degrees
.resize(800, 600) // Resize the image to 800x600
.resize(800, 800) // Resize the image to 800x800px
.composite([
{
input: Buffer.from(watermarkBuffer),
gravity: "southeast", // Position the watermark in the bottom-right corner
},
])
.toFormat("jpeg")
.toFile(outputPath);

// Log the output file path
logger.log(`Optimized image saved at: ${outputPath}`);

// Read the optimized image file
const optimizedImageBuffer = await fs.readFile(outputPath);

// Upload the optimized image to R2, replacing slashes with underscores
const r2Key = `processed-images/${path.basename(outputPath)}`;

const uploadParams = {
Bucket: process.env.R2_BUCKET,
Key: r2Key,
Body: optimizedImageBuffer,
};

// Upload the image to R2 and get the URL
await r2Client.send(new PutObjectCommand(uploadParams));
const r2Url = `https://${process.env.R2_ACCOUNT_ID}.r2.cloudflarestorage.com/${process.env.R2_BUCKET}/${r2Key}`;
logger.log("Optimized image uploaded to R2", { url: r2Url });

// Delete the temporary file
await fs.unlink(outputPath);

// Return the optimized image buffer, file path, and R2 URL
return {
optimizedImageBuffer,
optimizedImagePath: outputPath,
r2Url,
};
.jpeg() // Convert to jpeg
.toBuffer() // Convert to buffer
.then(async (outputBuffer) => {
await fs.writeFile(outputPath, outputBuffer); // Write the buffer to file

const r2Key = `processed-images/${path.basename(outputPath)}`;
const uploadParams = {
Bucket: process.env.R2_BUCKET,
Key: r2Key,
Body: await fs.readFile(outputPath),
};

const upload = new Upload({
client: r2Client,
params: uploadParams,
});

await upload.done();
logger.log("Image uploaded to R2 storage.", {
path: `/${process.env.R2_BUCKET}/${r2Key}`,
});

await fs.unlink(outputPath); // Clean up the temporary file
return { r2Key };
});
},
});
```
Expand All @@ -133,4 +127,4 @@ To test this task in the dashboard, you can use the following payload:
}
```

<LocalDevelopment packages={"the Sharp image processing library"} />
<LocalDevelopment packages={"the Sharp image processing library"} />