Maintenance strategy: Considering splitting type-fest into smaller importable parts

[voxpelli]

The issue in #784, which got fixed in the highlights a larger problem with the 4.10.2-release, is one that highlights a bigger challenge with the current scope of the type-fest package:

It contains a lot of types, some which are more complex than others.

Eg. MergeDeep and CamelCase are far more complex than eg. JsonValue and Primitive are, yet if you import type-fest all types gets parsed and validated (unless you set skipLibCheck, something that disables even .d.ts checks within ones own project, see #785)

Context

I run my own canary type tests on my own modules and eg. my umzeption project is still failing its type canary, despite the release of 4.10.2.

Why? Well, umzug (a dependency of my module) relies on type-fest@^3.0.0 (see sequelize/umzug#645) in two places: SetRequired in one place and MergeExclusive in another place – neither which relies on MergeDeep, yet the canary tests of my modules fail due to MergeDeep in 3.x not being compatible with TS 5.4.

Solution

In a way type-fest is a "lodash for types" and just like lodash moved to publish more granular packages in addition to their main package I think we should do something similar.

Eg. following the README groups (should maybe be more granular as eg. utilities includes a lot):

• @type-fest/basic
• @type-fest/utilities
• @type-fest/type-guard
• @type-fest/json
• @type-fest/async
• @type-fest/string
• @type-fest/array
• @type-fest/numeric
• @type-fest/casing
• @type-fest/misc

And probably also:

• @type-fest/core – core internal utilities that's relied on between multiple of above packages
• type-fest – as today, but importing the individual modules rather than containing all the code itself

I'm not a huge fan of mono-repos itself but I think it would serve a good purpose here and allow for a package like umzug to rely on type-fest without exposing themselves to all the possible future incompatibilities that the complex types may cause in the future.

Thoughts @sindresorhus, @skarab42, @Emiyaaaaa, @BendingBender, @CvX and others?

Upvote & Fund

• We're using Polar.sh so you can upvote and help fund this issue.
• The funding will be given to active contributors.
• Thank you in advance for helping prioritize & fund our backlog.

[mmkal]

Could exports be used to allow import MergeExclusive from 'type-fest/merge-exclusive'? That way, it'd be non-breaking - most people, most of the time can still just import {MergeExclusive} from 'type-fest', if they're willing to take the risk/"pay" for all the types.

Note, lodash does this too. The options are:

import {merge} from 'lodash'
import merge from 'lodash/merge'
import merge from 'lodash.merge'
import {merge} from 'lodash-es'
import merge from 'lodash-es/merge'

Edit: I imaging the reason for lodash micro packages are to reduce install size - not so much of a concern for type-fest I think.

So adding exports wouldn't prevent publishing micro packages in future. But micro packages sound like hell to manage, both for maintainers and users.

I've asked renovate to update type-fest for umzug in the meantime!

[voxpelli]

@mmkal Yes, that was my thinking, that type-fest would eg. move from:

export * from './source/primitive';
export * from './source/typed-array';
export * from './source/basic';
export * from './source/observable-like';

To:

export * from '@type-fest/basic';

I imaging the reason for lodash micro packages are to reduce install size - not so much of a concern for type-fest I think.

If TypeScript is parsing and validating lots more types than it needs to, then I guess it could have an impact on its performance, especially when type-fest appears in plenty of versions in the same dependency tree (which in my experience is rather common)

Edit: I misread you, yeah pushing for using import MergeExclusive from "type-fest/merge-exclusive' could be an alternative

[Emiyaaaaa]

So adding exports wouldn't prevent publishing micro packages in future. But micro packages sound like hell to manage, both for maintainers and users.

I think so too.

type-fest is strongly correlated with TS version. It's not uncommon for mismatched versions to have problems. Either use the correct (recommended in the README) version, or use skipLibCheck.

but "excessive import" (or why this type SomeTypeInTypefest throw an error but I've never/needn't used it? ) is a real problem.

I think we should balance cost and how likely is such a problem to occur in the future, especially after adding type-fest to TS canary tests and very thanks @voxpelli for doing it.

That's all I can think of currently.

[skarab42]

I like the suggestion of using exports.

[voxpelli]

Either use the correct (recommended in the README) version, or use skipLibCheck.

skipLibCheck can only be used of all of the .d.ts files on ones project are automatically generated, as else those will not be type checked

Also – unless we encourage packages to set type-fest as a peer dependency there will be outdated version – either because one remains in a lock file and people hasn't done a lock file maintenance or because a dependency has an older major version, like type-fest@^3.0.0, in their dependencies (eg. even @sindresorhus himself has a package with a very outdated type-fest@^0.20.2)

exports could solve this as well, but is less discoverable (how do one get eg. VSCode to prefer that import path rather than the main export?) – having dedicated packages would make it more visible (especially if people opt to install them rather than type-fest directly)

[mmkal]

exports could solve this as well, but is less discoverable (how do one get eg. VSCode to prefer that import path rather than the main export?)

That can be solved by using named exports rather than default. Lodash use default, but type-fest could do:

import {MergeDeep} from 'type-fest/merge-deep'

That way vscode would offer both options when typing "MergeD..." and pressing cmd-space, without forcing users to know about and install hundreds of packages.

[voxpelli]

That way vscode would offer both options when typing "MergeD..." and pressing cmd-space

Would it be much different from what it is today? Would there be a way to know that its an officially supported way to import? Today you also get suggestion to import it from a subpath, but hopefully people don't right now

[mmkal]

Yes, I think it'd be different, because being part of exports is a strong signal that it's an official part of the package API surface. Being a random file that happens to be in your node_modules directory like it is now, is a strong signal you shouldn't use it.

Adding to exports, combined with some docs, would be enough to hint that people can/should use it, I think.

But you're right it's not much different from the status quo, especially in terms of how little implementation + maintenance work it is. But that's a good thing!

[voxpelli]

One could potentially use something like https://github.com/timocov/dts-bundle-generator to bundle and publish those types from a totally separate repository, avoiding any added complexity here

[sindresorhus]

Publishing micro-packages for type-fest is not going to happen. Lodash tried it and abandoned it.

[sindresorhus]

yet if you import type-fest all types gets parsed and validated

Why is TypeScript even parsing and validating unused types? I guess it's because there could be global declarations or interface extensions? Sound like this needs to be fixed in TypeScript. It needs some kind of config that says that the types are pure and only depend on import/export.

I also wonder if moving to ESM will improve this. At least for JS, ES modules are lazily parsed, so you only pay for what's imported.

[voxpelli]

Sound like this needs to be fixed in TypeScript. It needs some kind of config that says that the types are pure and only depend on import/export.

Anyone up to open such an issue / investigate?

@sindresorhus What's your view on import { MergeDeep } from 'type-fest/merge-deep' as a stop-gap measure? Waste of time? Better to rely on canary tests helping us catch and fix any regressions before they become a problem?

I'm also pondering of this from the Small Focused Module point of view. The one focus of this module is collection of essential TypeScript types but that's about as focused as collection of essential JavaScript helpers (which is essentially lodash), but you have typically not released such all-in-one modules but rather focused on many small individually versioned modules instead? Is this module different because its about types or has your general philosophy shifted @sindresorhus?

[sindresorhus]

Waste of time? Better to rely on canary tests helping us catch and fix any regressions before they become a problem?

I prefer to focus on the real fix. This is not a wide-spread problem anyway.

I'm also pondering of this from the Small Focused Module point of view. The one focus of this module is collection of essential TypeScript types but that's about as focused as collection of essential JavaScript helpers (which is essentially lodash), but you have typically not released such all-in-one modules but rather focused on many small individually versioned modules instead? Is this module different because its about types or has your general philosophy shifted @sindresorhus?

One of the main benefits of small focused modules is reducing runtime size, but that's not a concern for types as they are a compile-time construct. In this case, having everything in one package is the pragmatic choice, especially because many types depend on others, and having them in separate packages would add a lot of overhead in maintenance, install time, and usage. I still believe in small focused modules, but I think I have become more pragmatic about it throughout the years. For example, with is or uint8array-extras.

[voxpelli]

I'll close this one then and we'll pick up the fix for TypeScript when/if anyone wants to pursue that :)