Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add custom webpack loader to remove unused mimetypes from mime-db #5148

Merged
merged 2 commits into from
May 22, 2024

Conversation

absidue
Copy link
Member

@absidue absidue commented May 21, 2024

Add custom webpack loader to remove unused mimetypes from mime-db

Pull Request Type

  • Other - Build optimization

Description

To be able to offer it's Save xyz As... context menu entries electron-context-menu needs to know what file extension it needs to give to the downloaded files, which is a bit difficult as web uses mime types/media types instead of file extensions. That's where mime-db comes in, it's a gigantic list of mime types and their file extensions where they are known.

As mentioned in the previous paragragh that list is gigantic (see it for yourself: https://github.com/jshttp/mime-db/blob/master/db.json) and takes up more than half of the main.js file's file size. So I set about to rectify that (well I've known about it for a while now, but I've only just come up with a good solution for it). As we only use the Save Image As... context menu entry, we only need the image ones, which already gets rid of a large number of them, we can also get rid of some non-standard ones.

The new webpack loader only takes a few milliseconds to run on my machine, to take the mime-db module from 143 KiB to 1.98 KiB, so low impact on the build time but high impact on the file size.

All that means that the main.js file went from being 247 KiB to 106 KiB!!!

Screenshots

Extracts from the webpack build log for the main.js file.

Before:

built modules 415 KiB [built]
  modules by path ./ 415 KiB
    modules by path ./node_modules/@seald-io/ 146 KiB 16 modules
    modules by path ./node_modules/mime-db/ 143 KiB 2 modules
    modules by path ./node_modules/sort-keys/ 1.02 KiB 2 modules
    + 4 modules

After:

built modules 274 KiB [built]
  modules by path ./ 274 KiB
    modules by path ./node_modules/@seald-io/ 146 KiB 16 modules
    modules by path ./node_modules/mime-db/ 1.98 KiB 2 modules
    modules by path ./node_modules/sort-keys/ 1.02 KiB 2 modules
    + 4 modules

Testing

  1. yarn run pack
  2. Look at the size of dist/main.js

Desktop

  • OS: Windows
  • OS Version: 10
  • FreeTube version: 11a61d2

@FreeTubeBot FreeTubeBot enabled auto-merge (squash) May 21, 2024 20:55
@github-actions github-actions bot added the PR: waiting for review For PRs that are complete, tested, and ready for review label May 21, 2024
@absidue absidue changed the title Add custom webpack loader to remove used mimetypes from mime-db Add custom webpack loader to remove unused mimetypes from mime-db May 21, 2024
@absidue absidue force-pushed the smaller-mime-db branch from 70bc0fc to 813432c Compare May 21, 2024 20:56
Co-authored-by: Jason <84899178+jasonhenriquez@users.noreply.github.com>
Comment on lines +14 to +22
if (mimeType.startsWith('image/') && original[mimeType].extensions &&
(!mimeType.startsWith('image/x-') || mimeType === 'image/x-icon' || mimeType === 'image/x-ms-bmp') &&
(!mimeType.startsWith('image/vnd.') || mimeType === 'image/vnd.microsoft.icon')) {

// Only the extensions field is needed, see: https://github.com/kevva/ext-list/blob/v2.2.2/index.js
reduced[mimeType] = {
extensions: original[mimeType].extensions
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just make it easier to read
Coz the logic seems to be

  1. Add all image/ MIME types not matching specific prefixes with extensions
  2. Add specific MIME types
Suggested change
if (mimeType.startsWith('image/') && original[mimeType].extensions &&
(!mimeType.startsWith('image/x-') || mimeType === 'image/x-icon' || mimeType === 'image/x-ms-bmp') &&
(!mimeType.startsWith('image/vnd.') || mimeType === 'image/vnd.microsoft.icon')) {
// Only the extensions field is needed, see: https://github.com/kevva/ext-list/blob/v2.2.2/index.js
reduced[mimeType] = {
extensions: original[mimeType].extensions
}
}
if (!mimeType.startsWith('image/')) { continue }
if (original[mimeType].extensions == null) { continue }
let shouldBeAdded = false
if (!mimeType.startsWith('image/x-') && !mimeType.startsWith('image/vnd.')) {
shouldBeAdded = true
} else {
shouldBeAdded = mimeType === 'image/x-icon' ||
mimeType === 'image/x-ms-bmp' ||
mimeType === 'image/vnd.microsoft.icon'
}
if (!shouldBeAdded) { continue }
// Only the extensions field is needed, see: https://github.com/kevva/ext-list/blob/v2.2.2/index.js
reduced[mimeType] = {
extensions: original[mimeType].extensions
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current code is easier to read imo

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @efb4f5ff-1298-471a-8973-3d47447115dc I think the current code is more readable, also your code changes the logic, resulting in a different output.

I prefer the or/|| operator over writing lots of single line ifs that have the same outcome, as to me it's a lot more readable, especially when the variable names are short.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @efb4f5ff-1298-471a-8973-3d47447115dc I think the current code is more readable, also your code changes the logic, resulting in a different output.

I prefer the or/|| operator over writing lots of single line ifs that have the same outcome, as to me it's a lot more readable, especially when the variable names are short.

Also we really don't need to check for null here, as this code runs on a json file with a known structure so the only options are non-existent/undefined or an array, so checking for loose equality with null is rather confusing. Also the truthy check already covers both undefined and null.

Copy link
Collaborator

@PikachuEXE PikachuEXE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file won't be changed frequently so it's fine

@FreeTubeBot FreeTubeBot merged commit c4f58df into FreeTubeApp:development May 22, 2024
5 checks passed
@github-actions github-actions bot removed the PR: waiting for review For PRs that are complete, tested, and ready for review label May 22, 2024
@absidue
Copy link
Member Author

absidue commented May 22, 2024

This file won't be changed frequently so it's fine

Even if it were going to get frequent changes, using a more verbose, less readable code style, that is also less efficient, would still be a bad idea in my book.

@absidue absidue deleted the smaller-mime-db branch May 22, 2024 18:16
PikachuEXE added a commit to PikachuEXE/FreeTube that referenced this pull request May 23, 2024
* development:
  Update playlist name with title (FreeTubeApp#5150)
  User playlists as grid (FreeTubeApp#4949)
  Add custom webpack loader to remove unused mimetypes from mime-db (FreeTubeApp#5148)
  ^ Update GH action eps1lon/actions-label-merge-conflict (FreeTubeApp#5034)
  Translated using Weblate (Italian)
  Translated using Weblate (Serbian)
  Translated using Weblate (Estonian)
  Translated using Weblate (Bulgarian)
  Translated using Weblate (Spanish)
  Translated using Weblate (Italian)
  Translated using Weblate (Polish)
  Translated using Weblate (Portuguese (Brazil))
  Translated using Weblate (French)
  Translated using Weblate (Chinese (Traditional))
  Translated using Weblate (Chinese (Simplified))
  Fix gap next to banner when Hide Side Bar Labels is enabled (FreeTubeApp#5120)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants