Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Monorepo structure #2300

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Monorepo structure #2300

wants to merge 2 commits into from

Conversation

gchoqueux
Copy link
Contributor

@gchoqueux gchoqueux commented Mar 21, 2024

Description

Refactoring iTowns to monorepo structure.

The goal is to split itowns down into feature packages.
As an example, I've started with the geographic functionalities. (@itowns/geographic)

  • move unit test
  • improve dev server
  • babel resolver plugin is broken
  • documentation
  • coverage
  • bump version
  • move all src in packages
  • watch src packages
  • publishing, auto publish with github actions
  • create organization NPM to support @iTowns
  • move dependencies to packages
  • webpack : use alias to resolve dependencies instead of exportsFields
  • use babel monorepo configuration
  • move export from private root package.json
  • write all readme.md files

For the moment Debug and Widget modules are private.

Motivation and Context

Split code in packages for clearly structured code.
Simplifies development and facilitates contributions.
Increases the scope of users who only want to use a few functions.
This structure makes it necessary to make functions independent

Code movement

The classes Coordinates, Crs, Ellipsoid, Extent, OrientationUtils files are moved to ./packages/geographic/src.
The proj4 dependency moves to geographic package.

The unit tests moves to packages/xxx/test.
the code utils/debug moves packages/Debug
the code src/Utils/Gui moves packages/Widget

dependencies

This proposal from monorepo doesn't use monorepo package (by example Lerna).
New dev dependency is Concurrently, it's used to watch script in parallel
Itowns dependencies (three, proj4...) are moved to the packages that use them

node scripts

The scripts uses the options --workspaces or -ws , to call the corresponding script in each sub packages.

@gchoqueux
Copy link
Contributor Author

gchoqueux commented Mar 27, 2024

@Desplandis I resolve some issues

  • resolve module with exportsFields option
  • remove lerna with --workspace cli option

@Desplandis
Copy link
Contributor

Desplandis commented Mar 27, 2024

@gchoqueux Nice! I will take a quick look of your changes and write a todo list of things that should absolutely be tested before reviewing more thoroughly this PR. Expect it for tomorrow or next tuesday!

@jailln @mgermerie @ftoromanoff @AnthonyGlt I think we'll need your inputs on this change! ;)

This should be a good start to fix #2197, #1930, #2201 (list not exhaustive, feel free to complete). Shall we create a meta-issue to aggregate all those issues?

@gchoqueux gchoqueux force-pushed the subpackages branch 5 times, most recently from e5ec2ea to b2308ff Compare April 3, 2024 12:02
@gchoqueux gchoqueux marked this pull request as ready for review April 3, 2024 13:54
@gchoqueux gchoqueux force-pushed the subpackages branch 6 times, most recently from ea0116d to 962c99d Compare April 9, 2024 14:56
Copy link
Contributor

@jailln jailln left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not much to add to @Desplandis review.

I just think we should explain this new structure in the README and add sub package installation possibilities.

Also, how does it work with releasing and versioning? does subpackage all have the same version or can we (should we?) release them independently ?

packages/Geographic/README.md Show resolved Hide resolved
packages/Geographic/README.md Outdated Show resolved Hide resolved
package.json Outdated Show resolved Hide resolved
@gchoqueux gchoqueux force-pushed the subpackages branch 2 times, most recently from ac0a1ad to 451da03 Compare November 28, 2024 12:16
@gchoqueux
Copy link
Contributor Author

I just think we should explain this new structure in the README and add sub package installation possibilities.

Yes, I'll document

Also, how does it work with releasing and versioning? does subpackage all have the same version or can we (should we?) release them independently ?

In a monorepo, it's more complicated to manage a different version for each module, but if we can find a simple and robust solution we can implement it later.

@gchoqueux gchoqueux force-pushed the subpackages branch 3 times, most recently from d4616ba to b1d0ba1 Compare November 28, 2024 14:34
@Desplandis
Copy link
Contributor

Desplandis commented Dec 2, 2024

@gchoqueux Did you try to publish it in your own npm packages? I tried to do it with a custom alias, but didn't manage to make it working...

This is the a log of my last attempt:
https://github.com/Desplandis/itowns/actions/runs/12121853265/job/33793853058

The CI works correctly but I would like to ensure that both npm and website publishing are working.

Copy link
Contributor

@jailln jailln left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last batch of comments for me, only documentation remarks, I let you see with @Desplandis for package publishing. Thanks for the changes you already made

packages/Geographic/README.md Outdated Show resolved Hide resolved
packages/Geographic/README.md Show resolved Hide resolved
packages/Geographic/README.md Show resolved Hide resolved
packages/Geographic/README.md Outdated Show resolved Hide resolved
packages/Geographic/README.md Outdated Show resolved Hide resolved
packages/Geographic/README.md Outdated Show resolved Hide resolved
packages/Geographic/README.md Outdated Show resolved Hide resolved
packages/Geographic/README.md Show resolved Hide resolved
```bash
npm install --save itowns
```

```js
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to replace by something more generic and that will be easier to update when new packages are added and to remove examples since we already have examples in each sub-package documentation:

iTowns is currently moving to a monorepo organization and to a segmentation in sub-modules, allowing to import only some of itowns functionalities. Current itowns sub-modules are:
- [@itowns/geographic](packages/Geographic/README.md): `npm install --save @itowns/geographic`
- [@itowns/widgets](packages/Widgets/README.md): `npm install --save @itowns/widgets`
- [@itowns/core](packages/Main/README.md); `npm install --save @itowns/core`

Note the s at the end of @itowns/widgets since there is more: can you update this package name and also the folder name: packages/Widget -> packages/Widgets

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rename packages/Widget to packages/Widgets

@itowns/widgets is private and @itowns/core doesn't exist for the moment. I could remove the sub-modules examples ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! Ah yes ok sorry I thought we chose to make it public in this PR and didn't look if it was the case. Making it public now means also improving the doc, so I guess we can do that in a subsequent PR so we can merge this PR quicker

Ok for @itowns/core, my bad. So yes, only that is sufficient in my opinion:

iTowns is currently moving to a monorepo organization and to a segmentation in sub-modules, allowing to import only some of itowns functionalities. Current itowns sub-modules are:
- [@itowns/geographic](packages/Geographic/README.md): `npm install --save @itowns/geographic`

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

packages/Main/README.md Outdated Show resolved Hide resolved
@gchoqueux gchoqueux force-pushed the subpackages branch 13 times, most recently from a6179f1 to 3569118 Compare December 6, 2024 13:51
@gchoqueux
Copy link
Contributor Author

gchoqueux commented Dec 11, 2024

This is the a log of my last attempt: https://github.com/Desplandis/itowns/actions/runs/12121853265/job/33793853058

You need to register the packages, it seems to manage dependencies as a single package. I think I used a publish-latest in local to register all subpackages.

The deploying an releasing work see jobs
but there's a error for the publishing npm error Invalid tag name "@latest" of package "@latest": Tags may not have any characters that encodeURIComponent encodes.

I'm trying to solve this problem, and I think I've got a lead.

@gchoqueux gchoqueux force-pushed the subpackages branch 4 times, most recently from f902474 to 582a15a Compare December 11, 2024 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants