Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] PDF snapshot tests #19253

Closed
nicojs opened this issue Dec 4, 2022 · 9 comments
Closed

[Feature] PDF snapshot tests #19253

nicojs opened this issue Dec 4, 2022 · 9 comments

Comments

@nicojs
Copy link
Contributor

nicojs commented Dec 4, 2022

Hi friends πŸ™‹β€β™‚οΈ. Thanks for creating this valuable project!

I'm looking for ways to test my generated PDF file with decktape using snapshot tests. I would expect such a thing to be possible with playwright since underlying browsers generally support displaying PDF files.

However, I ran into this issue: #6091, which is closed as out-of-scope. I have two questions:

  1. Are there any technical reasons for it to be out of scope?
  2. Would you want to reconsider this stance for 2023 πŸŽ†?
@aslushnikov
Copy link
Collaborator

@nicojs

Are there any technical reasons for it to be out of scope?

The PDF generally falls outside of the web platform, and Playwright focuses on Web (as of 2022).

Would you want to reconsider this stance for 2023 πŸŽ†?

As far as I can tell, that's unlikely since these's still a lot to be done for web testing.

In general, the suggested work-around would be to use something like PDF.js to render PDF and then test it as a regular website.

Hope this helps!

@nicojs
Copy link
Contributor Author

nicojs commented Dec 6, 2022

Thanks! I didn't know about pdf.js but it seems to work like a charm. Here is my code for anyone that wants to do the same thing:

// print.spec.ts
import test, { expect } from '@playwright/test';

test.describe('Print', () => {
	let numberOfPages: number;

	test.beforeEach(async ({ page, browserName }) => {
		test.skip(browserName !== 'chromium', 'Print pdf is only validated in chromium');
		await page.goto('http://localhost:15005/print.html');
		await page.waitForFunction(() => typeof print=== 'object');
		numberOfPages = await page.evaluate(async () => print.numPages);
	});

	test('no visual regressions of the example slides', async ({ page }) => {
		const theCanvas = page.locator('#the-canvas');
		for (let i = 0; i < numberOfPages; i++) {
			expect(await theCanvas.screenshot()).toMatchSnapshot({ name: `page-${i}.png` });
			await page.evaluate(() => print.nextPage());
		}
	});
});
<!-- print.html -->
<!DOCTYPE html>
<html lang="en">
	<head>
		<meta charset="UTF-8" />
		<meta http-equiv="X-UA-Compatible" content="IE=edge" />
		<meta name="viewport" content="width=device-width, initial-scale=1.0" />
		<title>Document</title>
	</head>
	<body>
		<canvas id="the-canvas"></canvas>
		<script src="https://cdn.jsdelivr.net/npm/pdfjs-dist@3.1.81/build/pdf.min.js"></script>
		<script type="module">
			const pdf = await pdfjsLib.getDocument("./example.pdf").promise;
			let currentPage = 1;
			await renderPage();

			window.print= {
				nextPage: async () => {
					if (currentPage < pdf.numPages) {
						currentPage++;
						await renderPage();
					}
				},
				numPages: pdf.numPages,
			};
			async function renderPage() {
				const page = await pdf.getPage(currentPage);
				const scale = 1.5;
				const viewport = page.getViewport({ scale: scale });
				// Support HiDPI-screens.
				const outputScale = window.devicePixelRatio || 1;

				const canvas = document.getElementById("the-canvas");
				const canvasContext = canvas.getContext("2d");
				canvas.width = Math.floor(viewport.width * outputScale);
				canvas.height = Math.floor(viewport.height * outputScale);
				canvas.style.width = Math.floor(viewport.width) + "px";
				canvas.style.height = Math.floor(viewport.height) + "px";
				const transform =
					outputScale !== 1
						? [outputScale, 0, 0, outputScale, 0, 0]
						: null;

				await page.render({ canvasContext, transform, viewport })
					.promise;
			}
		</script>
	</body>
</html>

@codeami
Copy link

codeami commented Jan 30, 2023

I found this easier to use: -
https://www.npmjs.com/package/compare-pdf

@karlhorky
Copy link
Contributor

karlhorky commented Jun 13, 2024

@codeami how did you use it in your Playwright tests?

It would be great to get a canonical example and official Playwright team recommendation of PDF snapshot testing with Playwright, using PDFs from URLs.

The version that we currently have uses page.setContent() to create a PDF viewer using PDF.js using the PDF.js docs Hello World example, but it is verbose, and requires changes to our content security policy.

Ideally, we could just use the built-in PDF viewer in Chrome, but it appears that this is not a path that the Playwright team wants to invest in.

@karlhorky
Copy link
Contributor

karlhorky commented Jun 13, 2024

Here's my version of screenshot / snapshot testing with PDF.js, based on the PDF.js docs Hello World example:

// HTML template string no-op for VS Code highlighting / formatting
function html(strings: TemplateStringsArray, ...values: unknown[]) {
  return strings.reduce((result, string, i) => {
    return result + string + (values[i] ?? '');
  }, '');
}

test('PDF has screenshot', async ({ page }) => {
  // Go to page without Content-Security-Policy header, to avoid CSP
  // prevention of script loading from https://mozilla.github.io
  await page.goto('about:blank');

  await page.setContent(html`
    <!doctype html>
    <html>
      <head>
        <meta charset="UTF-8" />
      </head>
      <body>
        <canvas></canvas>
        <script src="https://mozilla.github.io/pdf.js/build/pdf.mjs" type="module"></script>
        <script type="module">
          pdfjsLib.GlobalWorkerOptions.workerSrc =
            'https://mozilla.github.io/pdf.js/build/pdf.worker.mjs';

          try {
            const pdf = await pdfjsLib.getDocument(
               'https://raw.githubusercontent.com/mozilla/pdf.js/ba2edeae/examples/learning/helloworld.pdf',
            ).promise;

            const page = await pdf.getPage(1);
            const viewport = page.getViewport({ scale: 1.5 });

            const canvas = document.querySelector('canvas');
            canvas.height = viewport.height;
            canvas.width = viewport.width;

            await page.render({
              canvasContext: canvas.getContext('2d'),
              viewport,
            }).promise;
          } catch (error) {
            console.error('Error loading PDF:', error);
          }
        </script>
      </body>
    </html>
  `);

  await page.waitForTimeout(1000);

  await expect(page).toHaveScreenshot({ fullPage: true });
});

Also added to playwright-tricks:

@karlhorky
Copy link
Contributor

karlhorky commented Nov 29, 2024

Oh, it seems like Playwright 1.49.0 introduced rendering of PDFs in page (at least in Chromium), as @dgozman mentions in Changes in Chromium headless in Playwright v1.49 (#33566):

PDF documents are now rendered in the page, instead of being downloaded. We recommend to update your tests accordingly.

I can confirm that await expect(page).toHaveScreenshot() works when navigating to a PDF URL πŸŽ‰

Although I need to figure out how to make it so that it shows the entire PDF and is not cut off... πŸ€” (and no gray background)

This is an example snapshot, showing the unfortunate cropping:

Image

@gselsidi
Copy link

gselsidi commented Jan 8, 2025

@karlhorky I actually wasted 2 days with the above, the default chrome pdf viewer sets a shadow dom to closed, so you can't interact with anything on the page such as setting the bounding box to capture the whole page as a screenshot, as fullPage: true doesn't work because it's inside an iframe

I got as far as injecting JS to set the shadow-root to open theoretically playwright still can't interact with it, and returns a pretty limited DOM when i query it.

But when I pause the test and inspect the page I see shadow-dom set to open so unsure what's really going on, feel free to go down the rabbit whole with this...

Going to implement the solutions shared above instead of wasting more time.

  await context.addInitScript(() => {
    const originalAttachShadow = Element.prototype.attachShadow;
    Element.prototype.attachShadow = function (init) {
      if (init.mode === "closed") {
        init.mode = "open";
      }
      await console.log("ShadowRoot created and set to open:", this);
      return originalAttachShadow.call(this, init);
    };
  });

  // Capture the new page (tab) opened
  const [newPage] = await Promise.all([
    context.waitForEvent("page"),
    ***your actions to click to get the new page with the pdf***
  ]);```

@codeami
Copy link

codeami commented Feb 12, 2025

@codeami how did you use it in your Playwright tests?

It would be great to get a canonical example and official Playwright team recommendation of PDF snapshot testing with Playwright, using PDFs from URLs.

The version that we currently have uses page.setContent() to create a PDF viewer using PDF.js using the PDF.js docs Hello World example, but it is verbose, and requires changes to our content security policy.

Ideally, we could just use the built-in PDF viewer in Chrome, but it appears that this is not a path that the Playwright team wants to invest in.

Sorry for the delay, and although the issue is closed, here is my code anyway
using https://www.npmjs.com/package/compare-pdf

`
/*

  • Export Moodboard Pdf And Validate
    */
    // Todo : Handle Windows Filesystem
    // Todo : Move hardcoded settings to external config

async exportMoodboardPdfAndValidate (slideRange = "", range = '1-1', count, suffix) {
// Save file using direct playwright API
// Compare file vs baseline and return comparisonResults
var suggestedFilename = ""
var fileFullName = ""
let comparisonResults = ""

comparisonResults = I.usePlaywrightTo('download files', async ({ page, context }) => {
  // use browser, page, context objects inside this function
  // Start waiting for download before clicking. Note no await.

  const downloadPromise = page.waitForEvent('download', { timeout: 181000 });
  await page.getByTestId('export-btn-export').click({ timeout: 182000 });


  // Wait for the download process to complete
  await page.waitForSelector('.consolidated-export-progress .progress-line');
  actualMessage = await page.textContent('.consolidated-export-progress');


  console.log(actualMessage);
  expect(actualMessage).equal(expectedMessage);

  const download = await downloadPromise;
  suggestedFilename = await download.suggestedFilename();

  suggestedFilename = suggestedFilename.substring(0, suggestedFilename.length - 4) + '_' + suffix + '.pdf';

  // Timestamp format: 04Aug13_21_
  const lengthOfTimeStamp = 11; // todo Create global
  const lengthOfAUT_Prefix = 4; // todo Create global
  var suggestedBaselineFilename = suggestedFilename.substring(0, suggestedFilename.indexOf('AUT_') + 4) + suggestedFilename.substring(suggestedFilename.indexOf('AUT_') + 15, suggestedFilename.length)
  // Save downloaded file somewhere
  fileFullName = process.cwd() + '/data/actualPdfs/' + suggestedFilename;

  await download.saveAs(fileFullName);
  // Image based compare pdfs

  const fs = require("fs");
  const pdftBuffer = fs.readFileSync(fileFullName);

  // suggestedFilename = "Moodboard_1000000147_AUT_16Aug23_00_VisualRegression_VisualRegression_slidesAll16_Previews-Prelines_FW22_stg.pdf"
  // suggestedBaselineFilename = "Moodboard_1000000147_AUT_VisualRegression_VisualRegression_slidesAll16_Previews-Prelines_FW22_stg.pdf"

  // allure.createAttachment("screenshot", screenshotBuffer, "image/png");
  await codeceptjs.container.plugins('allure').addAttachment("Actual exported PDF file (open in new tab and save as pdf to open)", pdftBuffer, "image/png")
  console.log("suggestedFilename: " + suggestedFilename);
  console.log("suggestedBaselineFilename: " + suggestedBaselineFilename);

  let comparisonResults = new comparePdf(codeceptConfig.config.comparePdfConfig)
  .actualPdfFile(suggestedFilename)
  .baselinePdfFile(suggestedBaselineFilename)
  .compare()
  await output.print(comparisonResults); //works OK, 
  return comparisonResults
});
return comparisonResults

}

`

@karlhorky
Copy link
Contributor

@codeami thanks!

In case you want to improve the formatting of your code, you can edit your post and create fenced code blocks with syntax highlighting on GitHub like this:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants