RFC: Retire Reaction

[damassi]

Update:

See this migration PR.

---

(Tagged a few folks who have expressed opinions about this in the past, but looking for anyone who has thoughts.)

Proposal:

Artsy's front-end development experience has come a long way over the past few years, and in large part this is due to Reaction. It introduced the team to TypeScript and Relay, and expanded our use of React by organizing UI creation around Storybooks and other helpful patterns. However, I'd like to argue that its time is now past, and that we could eliminate a few layers of friction by retiring Reaction and moving that code back into force, under a v2 folder directory (or some other name). This follows similar movements on our iOS side, where Emission was recently retired and the code merged back into Eigen in order to simplify.

Reasoning

History

At the time Reaction was born our tooling situation was much different. This was before Babel 7, and in particular before babel-typescript. This was also before we implemented Webpack. What this meant in practice was that in order to support TypeScript -- the main motivation for Reaction -- we would have to erect an entirely separate build system on top of the one that we had in Force and then find a way to smoothly consume the output of one inside of the other. Rather than work all of this out inside of an already massive app, the decision was made that we would move all of these tooling requirements into a separate repo, start from a clean slate, and publish changes in the form of a library that Force would then consume.

In the beginning, this was very rough. In order to see changes in force we had to manually publish library versions from the command line. Later, we introduced semantic-release to add a level of automation to this process of publishing, but still had to update Force manually once a new version of Reaction was released. Then we introduced Renovate, which further reduced friction by automatically updating Force.

At the moment this is where we're at. And things work pretty well! However, this separation is no longer needed due to a few different factors:

1. Force now supports TypeScript -- or more specifically: Frontend tooling between Force and Reaction is now identical. Both use babel-typescript, both use Webpack. Both follow the same patterns in terms of CI setup via Orbs running Jobs that reference the same npm script commands. Things are so identical that code can literally be copied and pasted from one location to the other and compile.

2. Reaction is no longer just a "Component Library". Reaction was Palette before Palette was Palette, and as such was shared between a few different apps, including Volt and Positron. Now that Palette is our primary UI library, Volt has been able to remove Reaction entirely, which leaves only Positron and its publishing related code. In order to keep things simple™, what I'm proposing is that Reaction lives on exclusively in support of this publishing code since in practice the /Publishing folder in Reaction is it's own UI library, complete with its own design idioms, and siloed from wider patterns. Positron will be able to continue on as if nothing has changed; same with the editorial apps in Force. As changes to this area of the codebase are rare, there is less urgency to reduce friction here. If changes are needed we can continue relying on existing Renovate / Auto CI patterns.

3. Related to 2, Reaction is increasingly becoming its own application environment. The Apps folder grows larger every month, as old apps are rebuilt and new apps come online. We have a sophisticated SSR rendering library that we maintain for SEO purposes, and tooling that dynamically splits apart our JS assets into individual bundles -- this is low-level tooling that is by circumstance spread across two repos, when really it should exist in one, as close to the compiler as possible. Additionally, we'll be deploying a new application shell soon (currently still in an A/B test), that leverages our SSR router in order to deliver a best in class user experience, but which again integrates heavily into force and spreads mission-critical code across multiple repos for reasons of circumstance rather than need. All of these things can be consolidated.

4. We have a pretty serious JS "common bundle" size problem that extends directly from our old force app dependencies mingling with new Reaction app dependencies. For example, there's no reason that artsy.net/collect -- an entirely new-tech Reaction-based app -- should be required to load a common.js file that after gzip deflation is ~5mb in size. Or artwork/id -- or any new app for the matter. By having things divided between two repos the problem simply grows more complicated than it should be. (E.g., the first step in separating our old apps from our new apps is to no longer use .jade files for our root page templates, and to use React -- but where should the primary shell files live? in Reaction close to the new App shell? or in Force, close to our express app server?)

5. Development experience. Where to begin here... Basically, if you have a 13" Macbook Pro with less than 32mb of ram and you'd like to develop a feature for Force, your development experience will be so maddeningly slow that you'll think there's a problem with your computer setup. A quick search through Slack will show numerous devs literally running out of memory while trying to boot Force. We've got complex intermediary tools in place to work around problems with symlinks, but which also adds complexity to our stack. The list goes on. But it doesn't need to be so. When developing in Force and linking with Reaction, we boot two identical environments that consume valuable resources. This only needs to happen once, in force. Once all of this code lives together, we can optimize Webpack to compile only those parts of the app we're working in (v2) and exclude the thousands and thousands of old-force files that never get touched anymore. This should dramatically improve the development experience for our team.

Exceptions:

The RFC title is somewhat misleading to get the point across. Reaction will live on in support for Publishing related code shared between force and Positron.

How is this RFC resolved?

If we decide to go forward, we move Reaction code verbatim over to force and into a v2 (or some other name folder). We then update non-publishing-related code references to @artsy/reaction to point at v2. The refactor is fairly straight forward. We'll also need to stand up a new storybooks instance.

As follow-up we can start thinking about how to optimize CI so that we only run tests related to files that have changed -- e.g., how things currently work in Reaction via @zephraph's recent opt-in orb changes, which has significantly cut down wait time, but that's unrelated to this RFC. Being able to simply skip the library publishing cyle over in Reaction and avoid hassles related to linking will save us many minutes on it own and improve dev time significantly.

Lastly, all of this work is largely in support of bullet point #4 up above -- how to get our JS asset bundle size down. We desperately need to fix this problem.

[ds300]

Big thumbs up from me :)

Merging emission into eigen removed a lot of process friction, infrastructure, and tech debt, while opening up opportunities for wide-reaching improvements which would otherwise have been very difficult. It seems like the potential wins for reaction/force would be even greater.

Thanks for this thorough write-up @damassi !

[zephraph]

Yep, 👍.

I agree 100%.

The final development experience is something that I think is super important. Figuring out how to decrease force’s resource consumption and CI times will be a challenge we need to tackle.

That said, I don’t think that should block us from starting this work.

The biggest question in my mind is this:

What’s the incremental path forward?

[damassi]

@zephraph - I don't think there needs to be an incremental path forward and we can just to do it. There will be four steps:

1. Copy Reaction's src contents (minus publishing) over to src/v2 in force
2. Update paths in force
3. Delete code unrelated to Publishing in Reaction, update README
4. Publish a major version and update Force / Reaction

I don't envision this migration taking more than a couple hours, where most of that time is just verification.

[joeyAghion]

This ceiling-cat is convinced.

I'd encourage us not to start skipping tests anytime soon. Combining repos eliminates a number of steps even without that (publishing, automatic PRs...), and a slower overall Force build may be a small price to pay for those. Once successfully consolidated, there'll of course be opportunities to speed up the combined build.

How "straightforward" is the initial reorganization, actually? Does it require a cross-team effort, or can it happen through a single PR? If it's too large a refactor for a single PR, maybe one group of components can migrate into Force as a proof-of-concept (including whatever duplication is necessary from Reaction), so that ongoing development can follow that pattern as components are touched?

[damassi]

@joeyAghion

How "straightforward" is the initial reorganization, actually?

It is straightforward, would require one PR, and wouldn't require cross-team effort, aside from either a) pausing work in Reaction during the migration, or b) asking devs to move their changes to Force once its done. The migration should take no more than a day.

maybe one group of components can migrate into Force as a proof-of-concept (including whatever duplication is necessary from Reaction), so that ongoing development can follow that pattern as components are touched?

The FE architecture between the two repos really is close to identical, so it will be best to migrate everything at once. However, a possible two PR approach here would be to copy files over, keeping old files in Reaction (so that all codepaths are covered); and then delete in reaction and publish a major update. I'm less in favor of this than deleting in reaction, publishing a major, and then letting typescript / our compiler drive things and fixing via errors. TypeScript will catch most broken code paths, and Babel will catch whatever is remaining (should reaction code be called from old CoffeeScript areas of the codebase, for example.)

I'd encourage us not to start skipping tests anytime soon

No prob for this.

My future thinking is: Force's CI time is so slow that for codepaths outside of the src/v2 folder we can skip running old tests and only run jest tests inside of v2, similar to running the full reaction test suite. Then on merges to master we can run the full suite. A further optimization is only running tests related to changed files, how reaction works right now. But there's really no reason for us to run thousands of tests in distant areas of the codebase for PRs.

[erikdstock]

@damassi

Copy Reaction's src contents (minus publishing) over to src/v2 in force

the '2' is doing a lot of work here... 😏

[eessex]

I'm curious what this would mean for analytics instrumentation between the two places -- especially concerning the publishing case that you raised, where events with tracking would still live within reaction.

[damassi]

There would be no change to analytics; we use an event bus that connects back to here.

[alloy]

Nice one 👌 Glad to see this years-long incremental path come to an end 😁

Proud of y’all 👏

[damassi]

(This is complete, closing.)