No simple way to author and share flow packages.

[Gozala]

Documentation does not seem to provide much insight on what is an idiomatic way to write packages like what file extension to use how to build and publish to allow others consume your packages without needing to customize their configurations.

I will just assume that setup used by graphql-js is an idiomatic way to go about it:

• Author source in src/ directory as module.js files.
• Use babel to transpile to plain JS.
• Publish to npm.

Problem with such setup is that users of my library loose all of the type information that original source contained. There are ways to workaround it but they either require flow configuration on the consumer side or non trivial build step from the publisher side.

Proposal

I would like to propose following things to improve current situation & grow number of flow typed packages distributed over npm.

1. Documentation should certainly include a guide on how to author and publish flow typed package.
2. I would suggest to make module.flow an idiomatic file extension for flow typed files. For following reasons:
   1. Flow extends JS syntax so sharing same file extensions does not really makes sense.
   2. If you author files flow code in module.js you can't really generate compiled JS files alongside of them.
3. In 0.19 introduced useful feature, that allows putting module.js.flow along the side of module.js in which case flow would give precedence to module.js.flow over module.js when importing ./module. Unfortunately no attempt to load module.flow is made. It would actually make sense to alter algorithm slightly so that flow would try module.flow then module.js.flow and then module.js which in fact would match node's algorithm more closely which attempts to load module and then module.js.

This would enable users to author their packages in module.flow files and then compile all module.flow files to module.js files along the side them. Publishing such package into npm also would preserve all the type information for consumers, since flow would first look for dep/module.flow (which will be an original source) before dep/module.js.

Note: Technically you could achieve more or less same even if your flow code was authored in module.js files but that would require copying and renaming them to module.js.flow which as far as I can tell has no trivial cross platform solution & requiring extra scripts isn't great. Another alternative is to just author code as module.js.flow and generate module.js files, it's just authoring in .js.flow file extension seems awkward.

[vkurchatkin]

It seems completely unnecessary. You can just have index.js and index.js.flow in root folder, no need to rename all source files

[Gozala]

That assumes that everything used by user is exposed through index module which is not necessarily the case, definitely not the case for packages I work with

Typed on tiny keyboard

On Jun 23, 2016, at 1:41 PM, Vladimir Kurchatkin notifications@github.com wrote:

It seems completely unnecessary. You can just have index.js and index.js.flow in root folder, no need to rename all source files

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[zerkalica]

+1 Yes, can't separate js.flow from transpiled sources and put them to separate root. flow-files kept class realizations, not only declarations.

Sources in my packages compiled from src to dist directory. package.json entry point is "main": "dist/index.js"

Documentation is weak, there is no common way to distribute types with npm package. graphql-js example is not a common way, flow files together with code.

Some flow-typed projects projects declare all interface in module namespace.

Flow developers use smurf-code style type names in global, as React$Component
And some ugly tricks to export them:

declare module React {
  declare var exports: $Exports<'react'>;
}

I think, best way is to autoextract flow-types from sources and put them to separate directory, generate flow-files like c++ h-files, But i don't know how.

Workaround: manually put interfaces to separate directory and include it into package.

my-package/.flowconfig

[options]
module.name_mapper='^my-package/i/\(.*\)' -> '<PROJECT_ROOT>/i/\1'

my-package/i/public.js:

// @flow
export type MyType = number;

my-app/app.js:

// @flow
import type {MyType} from 'my-package/i/public'

In this way we steel need to manually write entry point declarations and put it into flow-typed/my-package.js:

my-package/flow-typed/my-package.js:

declare module 'my-package' {
   declare var exports = ...
}

Another workaround: all interfaces as declarations in separate file. But there is a problem on reusing types between declaration files: #1778

my-package/flow-typed/my-package.js:

  declare module 'my-package' {
    declare type myType = number;
  }

my-app/app.js:

// @flow
import type {MyType} from 'my-package'

[vkurchatkin]

That assumes that everything used by user is exposed through index module which is not necessarily the case, definitely not the case for packages I work with

In this case you can compile flow files and then copy original with .flow suffix to the same folder. It seems like copying files around is something that could be solved outside of flow.

[Gozala]

That assumes that everything used by user is exposed through index module which is not necessarily the case, definitely not the case for packages I work with
In this case you can compile flow files and then copy original with .flow suffix to the same folder. It seems like copying files around is something that could be solved outside of flow.

Sure one could write such tool, but I'd argue it would be best for the ecosystem if such tool was bundled with flow itself or at least there was a guide on package authoring & distribution with a clear guide & tool recommendation.

[Gozala]

Alternative option (and arguably better one) maybe to do what typescript does, and allow type annotation generation from the source http://www.typescriptlang.org/docs/handbook/compiler-options.html

Either way I still think that defaulting to module.flow for flow syntax is better than piggy backing on .js extension. Tooling can be configured differently based of file extension, github can actually tell you the language code is authored in etc..

[jeffmo]

Hey! So I've been thinking a lot about this over the past month or two as I've been working on flow-typed. I'm in the middle of several multi-day/heads-down chunks of work at the moment so I'll have to come back in a couple of days for a more complete mind dump of what I had in mind -- but I'm all about getting more thoughts and feedback here.

tldr: Shipping .js.flow "shadow files" alongside compiled implementation files is the current best practice (as you suggest in your proposal, @Gozala) -- we should definitely document this better. I also intend to work on integrating "shadow files" as well as tools for managing them deeper into both Flow itself as well as flow-typed (more on this in my follow-up).

[bouk]

I found a very easy way to do this, assuming you ship your package with two folders, src (containing the original ES2015+ source) and dist (containing the transpiled files). If you have dist/index.js as the entrypoint in your package, you can just add a single file dist/index.js.flow next to it containing the following:

/* @flow */

export * from '../src';

Flow will then correctly read the type definitions from the source.

Hope this helps somebody

[jeffmo]

So I promised to come back to this thread and expand on my thoughts above, so here goes (and I promise to re-format this into actual documentation on the website by next week sometime -- that is long overdue):

I'll back up and give lots of context as a (potentially rambly) draft for the docs I just promised to write:

Today we have 2 kinds of files in Flow: Implementation files and libdefs.

Implementation Files

These are the normal .js files you write code in. You might write some annotations in them to be compiled away later by Babel, but ultimately they contain the meat of your source code and the stuff that actually runs in the browser/node/etc.

In addition to normal .js files, Flow also supports a second kind of implementation file which we'll call "shadow implementations". These are just files that end in .js.flow. Shadow implementation files can have everything in them that a normal .js file can have, except that Flow will give them precedence over an adjacent .js file with the same name. So if foo.js and foo.js.flow sit right next to each other in the same directory, Flow will completely ignore foo.js and only read foo.js.flow instead.

These shadow files are the recommended best-practice for shipping types with your projects! They let you include the original, typed source code right next to the compiled version of the code which can actually run in the browser/node/etc.

As such, when you ship your projects to npm, you should simply include the original source code in a .js.flow file that sits next to (and is named the same as) the compiled source code.

Libdef Files

Libdefs are the files that you either put in <PROJECT_ROOT>/flow-typed or in the place(s) that you specify in the [libs] section of your config. They can _only_ describe interfaces by way of declare statements -- but you can never write normal JS statements in them. flow-typed is a growing collection of versioned and tested libdefs for public 3rd-party libraries, but you can also write custom ones if you want.

When we talk about libdefs we are _not_ talking about .js.flow files -- they are different things. Libdefs can only contain declare statements -- including declare module statements. Shadow files act just like a .js file, except they shadow the .js file they would be if the ".flow" were removed from their name. Putting a declare module inside a shadow file is not useful and almost certainly doesn't do what you want since shadow files already represent a module themselves.

(Note to Jeff -- not for the docs: Let's revisit removing support for declare module anywhere except for libdefs)

The great thing about libdefs is that, because they only define interfaces, they have much less surface area to be typechecked. This is a good thing because it is both faster for Flow to typecheck and it limits the number of breaking changes in Flow that might shine through (each new version of Flow is smarter than the last one, so its easy for version N+1 to find new kinds of issues that version N wasn't looking for or missed).

Libdefs aren't completely immune to breaking changes in Flow, though: Sometimes new annotation syntax comes along that you might want to use (like declare module.exports) or old annotation syntax could be deprecated (like using declare var exports to declare the type of module.exports). We deal with this by versioning libdefs in flow-typed by the version of Flow they've been tested against (and by actually requiring tests for that libdef so that we can run it against future versions of Flow to see if it's still compatible). Luckily, though, most libdefs are forward-compatible most of the time since we don't make that many breaking changes to libdef syntax/semantics.

flow-typed

flow-typed is a shared repository of tested and versioned libdefs for public libraries. The various libdefs are maintained and contributed by the Flow community -- and you should contribute too! Check out the Readme there for more details.

flow-typed is relatively new, but in the near future you can expect to see flow-typed play an increasingly important role in the day-to-day use of Flow with 3rd party libraries.

For example: Let's say your library uses Flow v0.28 and everything typechecks perfectly before you publish to npm with .js.flow files.

2 weeks go by and Flow v0.29 ships. Luckily everything in your library is clean and 0.29 doesn't find any new errors for you. At this point, anyone using your library with either Flow v0.28 or Flow v0.29 will benefit directly from the .js.flow files you shipped (because they are compatible and typecheck with both versions of Flow).

2 more weeks go by and Flow v0.30 ships with lots of new kinds of error detection. In fact, it finds some issues with your library that 0.29 didn't notice before! (or maybe it's a little more strict and your library's code just needs some tweaking to preserve safety guarantees). Either way, you're busy and you just haven't had a chance to upgrade your library to v0.30 yet. Meanwhile users of your library have! Now we have a problem: The .js.flow files you shipped to npm actually cause Flow errors in your consumers' projects!

These newfound errors may or may not be relevant to your library's consumers. If they are relevant (or if the people who notice are good netizens), they'll come to your GitHub repo and send a pull request to fix the issue. But if the errors are not relevant to them, then those consumers are now burdened by noisy type errors that they don't really care about (and maybe don't have time to fix).

Luckily there is a solution for the consumers in the latter camp: They can write a libdef to stub out your 3rd party library's interfaces! Better yet: They can then contribute those libdefs up to flow-typed so that others may benefit from their hard work as well. Because flow-typed allows you to specify the version of both Flow and the library the libdef is intended for, it's possible for users to quickly contribute interfaces for your library written using Flow v0.29 that also works for Flow v0.30.

(Ok, I have to run to dinner now -- and I haven't had a chance to proof-read my ramblings yet -- but yolo I'm clicking "Comment" and will come back to clean this up and add to the website docs later...)

[Macil]

I made the simple flow-copy-source module which will take a directory of .js files, and copy them into a destination directory with ".flow" appended to the filename. For example, it works well for projects that have a (.npmignore'd) "src" directory with ES6+/flow-typed code, and a (.gitignore'd) "js" directory containing the transpiled code and the .flow shadow files.

ud and react-draggable-list are two projects published on npm using this which have the .js.flow files next to all of the transpiled .js files so that Flow types are automatically detected. This is all handled in the prepublish script of each of those modules' package.json files: "prepublish": "babel -s inline -d js/ src/ && flow-copy-source -v src js".

[Gozala]

Thanks @jeffmo for writing this all up. Here is my feedback / thoughts on the plan:

1. libdefs for libraries seems like a great solution due to less maintenance & coordination required between library author and users, as it's more immune to changes in flow the flow to affect it.
2. In order for a library author to take advantage of flow, inline type annotations are required. But then there is no straight forward way for authors to publish flow typed libraries such that users can take advantage of it without hassles. And even if we author uses flow-copy-source or alternative there is still an issue you mentioned that new version of flow may no longer type check.
3. Question at hand is why not allow generation of typedefs from the flow typed source instead ? It seems that library authors will take advantage of flow itself as well and have no pressure to keep up with latest greatest version & users will benefit from more stable type info dep libraries.
4. Furthermore I keep questioning why flow needs two different ways to add type information. For example I have attempted to pursue diff people from core team to allow external type annotations kind of similar to how it's in haskell. That way files with no implementations could be just treated as libdefs and if file happens to contain implementations, then you could just enforce that type definitions and implementations would match up. This would also make it trivial to generate libdefs or in this case type definitions from the original source, by just stripping implementation parts.

[jeffmo]

Question at hand is why not allow generation of typedefs from the flow typed source instead ? It seems that library authors will take advantage of flow itself as well and have no pressure to keep up with latest greatest version & users will benefit from more stable type info dep libraries.

I'm glad you brought this up! A tool like this is on my medium-term roadmap. It turns out such a tool could easily generate both libdefs and .js.flow files (since declare statements are also valid within implementation files):

foo.js

// @flow

export function addNums(a: number, b: number): number {
  return a + b;
};

**foo.js.flow** (naive version -- suffers from flow-versioning issues)
```js
// @flow

export function addNums(a: number, b: number): number {
  return a + b;
};

foo.js.flow (compressed version -- less likely to suffer from flow-versioning issues)

// @flow
declare export function addNums(a: number, b: number): number;

Both versions of foo.js.flow above work today, but the first one is "auto-generated" most easily via the cp command. A tool that can generate the second version is what I would like to build.

So why support the first version at all once the tool is ready?

The main benefit of .js.flow being a full implementation file rather than a libdef is mostly just an acceptance of the simplicity of the cp command. We can (and will) build said code-gen tools, but for small projects that want minimal overhead in their build/deploy setup: It's hard to beat something as simple and familiar as cp. When you're ready to spend time understanding the generation tool, you can "upgrade" to using flow generate-interfaces with all its bells and whistles.

Additionally, there are some other opportunities in the future for .js.flow files as implementation files that the team is sort of kicking around:

What if .js.flow were mode-flipped such that they do not require @flow at the top? Some users seem to dislike the current requirement for greenfield projects to have to write @flow at the top of every file... Instead, what if you just wrote your code in a foo.js.flow file and compiled that across to a .js file as your publish step? Some probably won't like the new file extension, some probably will -- but the great thing is you can decide for yourself which you prefer and neither seems to offer more objective caveats than the other.

[jeffmo]

Oh and I forgot to mention: Another great use-case for this kind of tool would be for generating best-effort libdefs for, say, old versions of Flow that aren't able to type-check a given implementation file without errors.

By starting with a libdef that's 95% working you can minimize the amount of work to be done to tweak that version in order to work with Flow vN-1 (and then just upload this to flow-typed)

[STRML]

Why not just use babel-plugin-transform-flow-comments in your build?

When minified for browser use, the comments will be removed - when used in development and in Node, they work fine.

It adds a tiny amount of bloat compared to shipping .js.flow files but it is much easier to do - simply add the transform to your build.

React-Grid-Layout does this, and we see output like:

// Types
/*:: import type {ResizeEvent, DragEvent, Layout, LayoutItem} from './utils';*/

// End Types

/**
 * A reactive, fluid grid layout with draggable, resizable components.
 */

/*:: type State = {
  activeDrag: ?LayoutItem,
  layout: Layout,
  mounted: boolean,
  oldDragItem: ?LayoutItem,
  oldResizeItem: ?LayoutItem
};*/


var ReactGridLayout = function (_React$Component) {
  _inherits(ReactGridLayout, _React$Component);
...

[rpominov]

@STRML But will this work if, for example, a class then transformed to es5 "class"? What will happen with type comments that correspond to class methods etc.?

[STRML]

You're right - for those transpiled features it will not work correctly.

[sslotsky]

I appreciate the proposals here, but lack of current, working examples seems to be an issue. I believe I've tried everything suggested here.

1. Flow comments - I tried to use a babel plugin that transforms the type annotations to comments. My transpiled files contain the comments as expected. But if I install my library from another project that's using flow, the flow checker insists that I need a libdef.
2. Shadowing - I've also tried using the flow-copy-source library. This copies my files from src into lib with the .flow extension as expected, but the flow checker from my consuming project still wants a libdef.
3. flow-typed - Just to see if it would work, I installed flow-typed and then installed a stub for my library into my example project. It still wants a libdef.
4. libdef - I've run into a couple problems so far with libdef.
   a. For one thing, it's not really clear where it should go. I keep seeing <PROJECT_ROOT>/flow-typed as the suggested location. However, in most cases it's <PROJECT_ROOT>/src that gets transpiled into the lib (or dist) folder. Therefore, the libdefs in flow-typed would not be part of the package if we placed the flow-typed folder at the root.
   b. Additionally, while I seem to be able to reference things like React.Element<*> in my source files, flow complains if I use these in my libdef. How do I get access to these types?

I suspect that some concise examples from existing libraries would go along way, but barring that, any suggestions are greatly welcome.

Clarification

When I say that the flow checker wants a libdef, what I should say instead is that it complains that the module is not found.

[Macil]

2. Shadowing - I've also tried using the flow-copy-source library. This copies my files from src into lib with the .flow extension as expected, but the flow checker from my consuming project still wants a libdef.

That shouldn't be happening. Do you have node_modules under the [ignore] section of your .flowconfig? You shouldn't do that. (If there are specific dependencies causing unnecessary Flow errors, then block them specifically.)

[sslotsky]

Thanks @agentme, this must have been what happened. I tried again and flow no longer complains that the module is not found.

However, it also passes all checks even when I change my props to something invalid. My .flow files are definitely present in node_modules:

▾ node_modules/
  ▾ @orange-marmalade/reformd/
    ▾ lib/
        Check.js
        Check.jsx.flow
        Checkbox.js
        Checkbox.jsx.flow
        CheckboxGroup.js
        CheckboxGroup.jsx.flow
        index.js
        index.js.flow
        types.js
        types.js.flow
      package.json

[Hypnosphi]

I promise to re-format this into actual documentation on the website by next week sometime

=(

[lll000111]

@Yuripetusko You have to see the Flow types as separate from the Javascript code. The Flow type imports have nothing to do with what's going on in Javascript. You just import the types from where they are, that has nothing to do with what's going on in Javascript.

[Yuripetusko]

@lll000111

Thank you, my problem was that i was trying to mix normal js exports and flowtypes somehow together, as soon as i started treating them separately it all worked fine

[mrkev]

tl;dr, is this issue solved? should I close it?

[alexeyraspopov]

Is there any place in docs that describe the way to bundle flow typings (.js.flow files) with a lib?

[saadq]

I don't think there is any mention of .js.flow files anywhere in general on the Flow site, I would say this issue should remain open until there is some docs about it.

[rgbkrk]

tl;dr, is this issue solved? should I close it?

Likely not. We probably won't have the "simple" asked for above until flow can generate libdefs for the user (rather than the bloated results from copying original source to .flow.js files).

[TrySound]

You don't need to generate or copy sources just reuse them.
https://github.com/renatorib/react-powerplug/blob/master/package.json#L14

[Gozala]

@TrySound that does make sense although you'd still want to generate file per module if you want to allow import per module rather than whole bundle. @agentme Is there any reason for flow-copy-source not to generate pointers instead of copying the whole file ?

[TrySound]

@Gozala Do you really need to install tool to solve this? echo works everywhere afaik.

[Gozala]

I also have being considering to incorporate transform-flow-comments into my build tool chain which if works may just be good enough solution.

[Gozala]

@Gozala Do you really need to install tool to solve this? echo works everywhere afaik.

@TrySound Doing that cross-platform in nested directory structures isn't trivial and more than what I'm comfortable putting in the npm scripts field. That not to say you or anyone else shouldn't do it.

[Macil]

Generating a single pointer pointing at the source index.js would work in the case that the module isn't built for any other files to be directly imported.

Generating a .js.flow pointer per source file would work, but I never gave it much thought because when flow-copy-source is usually used, the work babel does on every file probably dwarfs the difference between flow-copy-source copying files vs creating pointers, and doing pointers comes with some minor drawbacks: 50% more files in the npm package, the .flow files can't be moved relative to the source files, and the src/ directory can't be .npmignored which you could do when flow generate-interfaces is ready.

[LoganBarnett]

If @bouk's suggestion works for you, I've created a script that does just that - it takes your entry point (ie index.js) and creates a index.js.flow that points to your src. The runtime still consumes the dist. Here's a simple script that creates the index.js.flow. It assumes your index.js is where you export all of the public things for your lib.

https://github.com/LoganBarnett/cubed.js/blob/master/flow-dist.sh

In package.json, the main entry needs to point to dist/index.js.
https://github.com/LoganBarnett/cubed.js/blob/master/package.json

If you need a more cross platform solution, I would imagine doing the text substitution would be fairly simple from a js file. This needs no recursion and the only artifact here is a relatively small index.js.flow that the runtime doesn't even consume.

[steelbrain]

I've opened #6504 to fix this. Please have a look

[amrit07]

I have a library which has .flow files alongside .js files ; yet when I import that in one of my application and wrongly use its methods ; the flow does not complain at all.
Details are:
Application:
.flowconfig
[ignore]
./dist/
./.storybook/
./.history/
./.config.js/.*
flow-typed
[untyped]

[declarations]
node_modules/testLib/.*

[options]
module.system.node.resolve_dirname=node_modules
[strict]

flow-bin version: 0.96.0.

Library:
package.json
{
"name": "testLib",
"version": "1.0.0",
"description": "",
"main": "lib/index.js",
"files": [
"lib"
],
"scripts": {
"flow": "flow",
"build": "rimraf lib && babel src/ -d lib/ && npm run prepack:flow",
"prepack:flow": "flow-copy-source src lib"
},
"keywords": [],
"author": "",
"license": "ISC",
"devDependencies": {
"@babel/cli": "^7.4.3",
"@babel/core": "^7.4.3",
"@babel/preset-flow": "^7.0.0",
"flow": "^0.2.3",
"flow-bin": "^0.96.0",
"flow-copy-source": "^2.0.3",
"rimraf": "^2.6.3"
}
}

Note: I'm using npm link method for linking the node_modules.

Any help is much appreciated. I've tried and searched all the solutions present in the net especially https://javascriptplayground.com/npm-flowjs-javascript/ but I've not had success.

[Macil]

It shouldn't be necessary to list the library's files in the declarations section. That might be causing the problem.

[amrit07]

removing it also doesn't help

[yurtaev]

@amrit07 if you use index.js to export modules then try to add // @flow to index.js

[maxammann]

@amrit07 have the exact same problem. The imported types are interpreted as 'any'