[Shallow Renderer] Plan forward

[gaearon]

Let's discuss what to do with the shallow renderer here. As I mentioned in #16168 (comment), we aren't using it much and don't consider it a best practice. So we aren't going to be very good stewards of its API going forward.

My proposal was for Enzyme or folks interested in it to copy the code into another repo, and continue maintaining it there under a difference package name. It would make sense to Enzyme to start depending on that fork. We would then deprecate react-test-renderer/shallow in favor of the community-maintained fork.

If you'd like to volunteer to set up a repo, please let us know in this issue!

cc @davidmarkclements regarding the proposed react-shallow-renderer package name.

[ljharb]

The main concern I have isn’t maintenance of the code - it’s how it might break with new react releases.

Given that most of what the dev tools does seems to overlap with what enzyme needs, i wonder if a better path might not be in sharing those components between enzyme and the dev tools?

[gaearon]

At least for shallow renderer, I wouldn't expect breakage with new React releases because the code itself is standalone. It can lag behind in features but it doesn't actually depend on React internals the way the more invasive introspection does. This is because it's completely implemented from scratch and doesn't use the real codepaths.

[gaearon]

I'm not sure I see how DevTools would be helpful given that DevTools introspect a real tree — but shallow renderer is a 100% custom implementation that doesn't use any of the real reconciliation mechanisms.

[kumar303]

This comment about shallow renderer's dependencies implies that forking it into a standalone module would require copying/pasting a lot of core code.

[gaearon]

This is not a lot of code. Each of these is a tiny utility and they probably don't add up to more than ~200 lines.

[gaearon]

Also you can replace invariant with throw Error, warning with console.error, and objectIs with Object.is.

[ljharb]

That's a fair point; it would still be preferable to avoid it lagging behind in new features if possible.

[tomstuart]

For clarification: what is the main reason that the regular test renderer can’t support a shallow mode where composite child components are left inline instead of recursively rendered into host components? Is it for reasons of API backward compatibility, a fundamental limitation of the test renderer, a design principle, or something else?

[robertknight]

If it wasn't for the need to support the very large number of tests that I presume already exist, I think I would advocate for Enzyme to deprecate (and maybe even eventually drop) its shallow rendering mode entirely and instead document methods for users to mock at the import level as Dan described here. Anecdotally I observed that the differences in normal vs shallow rendering in Enzyme were the source of quite a lot of confusion for other devs on my team and that led us to replace it with normal rendering + import-level mocking instead.

If Enzyme were to do that, perhaps the shallow renderer could be "frozen" so that it is kept working, but no new features are added to it as a matter of policy.

[ljharb]

@robertknight That presumes that it's actually a better pattern to do explicit import mocking of each sub-component (it also presumes that a pattern that forces jest is a reasonable thing to recommend). This isn't the place for a longer writeup of why shallow rendering is critical, a good pattern, and required for robust testing, however.

[robertknight]

That presumes that it's actually a better pattern to do explicit import mocking of each sub-component (it also presumes that a pattern that forces jest is a reasonable thing to recommend).

I agree that both of those things would be undesirable. We automatically mock imported components using some simple heuristics rather than requiring developers to remember to configure each one. We also use a Babel plugin, which is not tied to any particular test runner, rather than Jest, to do this.

This isn't the place for a longer writeup of why shallow rendering is critical, a good pattern, and required for robust testing, however.

That's fair enough. Perhaps I can continue the discussion in an issue on the Enzyme repo?

[Sonic12040]

Would you be willing to speak more about the automatic mocking? I'd love to learn about the flow that you've found successful and the use-cases.

Edit: Realize I didn't tag @robertknight

[thedanchez]

@gaearon @ljharb I'd like to volunteer setting up a repo if no else will -- appreciate any direction on the details of it. But I think it's important to keep the ball moving on this endeavor and not letting it go stale.

[thedanchez]

@gaearon @ljharb Just wanted to follow up on my volunteering around this plan forward.

[thedanchez]

@gaearon @ljharb Any updates on this? I'm still waiting, willing and trying to help here.

[ljharb]

I remain happy to take over the shallow renderer, but the work to maintain it would be prohibitive unless the react team was willing to commit to helping to keep its tests passing as a precondition for any React releases (whether submitting failing test cases or notifying about upcoming failed tests with sufficient advance notice that i can make fixes, or whether making the fixes themselves)

[perrin4869]

Does this mean the shallow renderer is practically abandoned?

[JimLynchCodes]

@gaearon Myself and some of my colleague engineers are using react-test-renderer/shallow and do consider it to be a best practice. Unit testing is about "testing at the edges", and philosophically a parent component should only care that its children are rendered with the correct props without caring about the children components' implementation details, and this is exactly what shallow rendering is. Do you disagree with this?

[thedanchez]

@gaearon I have to say I'm very discouraged by the lack of activity on what I think to be a very important topic of discussion here. When will the React Core Team revisit this?

[gaearon]

@thedanchez

As far as I'm aware, there is nothing that prevents you from creating a repository, moving the code there, and releasing a package. Shallow renderer has no dependencies on the actual React implementation, so you shouldn't need our permission or assistance to start working on this.

I'm sorry I haven't replied to you earlier — as you can see, there are 476 other open issues, and notifications occasionally get lost. I wasn't actively checking this issue because, again, there is nothing from our side that prevents you from starting on this.

Let me know if there's something else you want us to revisit!

[gaearon]

cc @NMinhNguyen who also expressed interest in setting up the repository and a package. He has a good understanding of how these are set up in general, so he can help you out if you're blocked on figuring this out.

[NMinhNguyen]

@gaearon I’ll start a new repo tomorrow (ish)

[ljharb]

@gaearon it's not just about setting up a package tho; is there any interest from the react team in #17321 (comment) ?

[NMinhNguyen]

@ljharb #16168 (comment)

We'd be happy to run some tests against a separate package fwiw. Similar to how we have integration tests with create-react-class.

[ljharb]

Specifically, this part:

commit to helping to keep its tests passing as a precondition for any React releases

[ljharb]

With that commitment, I'll make a repo for it this week, and start moving towards using it in enzyme.

[NMinhNguyen]

FYI I've begun work on extracting the shallow renderer and got all tests passing locally: https://github.com/NMinhNguyen/react-shallow-renderer. Will work on setting up builds and CI etc. later this week

[ljharb]

@gaearon by changing what "rendering" means such that the shallow renderer's output no longer mirrors the React's output (for the first level, ofc).

[NMinhNguyen]

Hey @davidmarkclements, I've now set up a new repo for react-shallow-renderer, would you be able to transfer the npm package to my username, minh.nguyen?

[ljharb]

@NMinhNguyen if you're interested in making it work for enzyme, please add me on github and npm as well.

[robertknight]

I mentioned in an earlier comment that we replaced shallow rendering with a different approach that provides the same benefits, using semi-automatic mocking with a Babel plugin. I found some time to write up a blog post with details and links to real-world examples in a reasonably complex application. We are using Preact, but the approach works exactly the same way with React. I hope this is of use to anyone who is evaluating whether to continue using the shallow renderer or do something else.

[gaearon]

This is great. Want to add a link to this plugin to our testing page where we mention module mocking?

[NMinhNguyen]

@davidmarkclements has transferred the package name to me and I've just published the initial version of it, v16.12.0 - https://www.npmjs.com/package/react-shallow-renderer. @ljharb I've added you on both npm and GitHub.

[ljharb]

@NMinhNguyen i've invited you to the new enzymejs org on github (announcement); it'd be great to transfer the shallow renderer repo there.

[mikeborozdin]

I've created a package that solves the issue - https://www.npmjs.com/package/jest-react-hooks-shallow

[dschinkel]

As I mentioned in #16168 (comment), we aren't using it much and don't consider it a best practice. So we aren't going to be very good stewards of its API going forward.

I'm very disappointed in @gaearon's statement above considering there are many React users who do not agree with this stance. The fact that you folks like integration tests vs shallow and making the assumption that this is best practice is absurd. Many of us use shallow rendering (with lower-level unit tests) especially as we TDD and keep our tests still decoupled, lightweight, and fast.

Just because you, Dodds and others haven't found benefits in shallow, many of us do and have. For you to just diss it like that, because you haven't acknowledge others do prefer shallow rendering to me is a huge disappointment to the JS community as a whole for those who do not want the ice cream cone / trophy way of test driven development or just testing in general. I think this is a huge misstep on your part to keep thinking your way is best so you won't support other approaches to testing React. It's been proven in the past that having mostly integration tests ends up being very painful..on the contrary to what you, Dodds, and others preach.

But, whether you disagree or not, that's besides the point. The point here is that you should be trying to make React as testable as possible in many ways, not just in an integrated test approach. A good codebase should be able to be easily tested whether it's shallow or deep rendering. And you should be supportive of both approaches, not just one.

[nielskrijger]

@dschinkel This seems a bit harsh.

Gaearon has discussed how to use jest.mock as if you'd use a shallow test; #16168 (comment)

By changing tests to explicitly mock any child components using jest.mock('Button', () => 'button') you still get the same snapshots and much the same behaviour as when using shallow. I've been doing TDD and unit testing just fine with this approach. Granted, having to migrate 1500 tests that relied on shallow took a while (and I wasn't happy about it!), but it took a few days, not weeks due to the ease of jest.mock.

How much to mock: With components, the distinction between a “unit” and “integration” test can be blurry. If you’re testing a form, should its test also test the buttons inside of it? Or should a button component have its own test suite? Should refactoring a button ever break the form test?https://reactjs.org/docs/testing.html

I agree with that statement; sometimes it's blurry, even more so when using something like styled-components. With the tools that remain you can still do both approaches, shallow is just very convenient and guarantees you're never testing more than the unit under test. In that sense it's definitely useful; but not critical to do unit testing.

[dschinkel]

response to @nielskrijger

I'll leave at this and this (I know, Thank God right? get dschinkel out of here!):

• Many of us do not want to use Jest Mocks or just a lot mocking frameworks at all. They impose a learning curve, magic you have to learn, and you are reliant on that framework's way, updates, etc. The simpler you can keep your tests, the better off you are. Enzyme Shallow, while yes it mocked under the hood, kept things very very simple and minimal

• Many of us do not prefer to test through the UI like you are when we test for several good reasons. You can get enough safety without a ton of heavy integration tests, no really...you can

• Many of us do not prefer to test through buttons, through forms, through web driver, through trees of components. We prefer not to test mostly through "imperative shell" so much and through webdriver. We don't want to have to litter our specs by plugging holes with mocks all over the place, making our test suite brittle and complex, setup more complex, etc. Simplicity is important on the test side, not just prod code.

If you want to know who the "many of us" I'd be happy to start listing those I know who prerfer not to test through the browser or through trees of components via a heavy UI integration test way. There are other ways to TDD the FE, not just what React tells you to do.

Those who TDD (I don't care if you don't, I am not here to stir you up, these are good test principles & practices regardless below), at least where I come from look for more than just "does it work".

When we write tests we look for:

• fast feedback on "does it work"

• lower level design feedback/pressure that gives us smells when prod design is decoupled and not simple. Integration tests do not give you that feedback

• lower-level feedback on "where did the problem happen and why quickly so we do not have sit there and constantly debug all the time". Those small fast unit tests tell you the problem quickly and clearly; lots of integration tests. tell you something happened way up high, and that's it

• narrowly scoped outside tests that are at the UI component level that don't really test that much wiring TBH, they just test that a child is being rendered with expected output state; keep components dumb even in containers or hooks, and we don't need to test the wiring "use it as a user" because in the past it's been shown that the ice cream cone really just leads to more pain writing and maintaining those kinds of tests

• "narrowly scoped" unit tests that sit below components (no this is not over testing) at some (not all) layers that sit below UI components

• Most integration tests that people write that test through mount, trees, or web driver can really be done through shallow and jsdom with the same amount of feedback. If you really really want to test through the UI more and flows, then maybe you need something else for that small set of tests but the majority of your tests to not need to be heavy integration tests.

So many are assuming everyone just wants to test the way Dodds and others paint as "best". That is not the case. A framework, any codebase should be flexible enough to invite and allow other approaches in testing to take place without a huge amount of friction.

This is all to say that there are other approaches that the community is shunning and just carelessly discounting, and that's not good for the profession as a whole. When I see it continually bashed, I will pipe up...someone needs to.

[Sonic12040]

I apologize, but I don't quite understand where you're coming from, and would like to obtain clarity.

The way I understand things, Dan was saying that because they are moving away from the shallow testing methodology, they would not be continuing work on the shallow renderer.

As a result, the shallow renderer has been spun into its own repository, and React will continue with their recommended approach. Also above is a blog on the semi-automated mocking approach employed.

I think this is a good compromise, with the shallow renderer moved into a package where teams using it can continue to receive support.

Can you help me understand the part I'm missing?

[dschinkel]

It's fine @Sonic12040, I know many will misunderstand what I'm trying to say and why I choose to say it here. I'm happy that there is still a push for shallow by the rest of us and realize I am late to the party. I apologize, I've been in Angular land (TestBed...awful) in the past few months and a bit behind on the the latest shallow news because that's all I ever needed...shallow and I'm invested in it.

Dan stated

Let's discuss what to do with the shallow renderer here

Well this is my 2cents, and I'm throwing it in here. Simple as that.

I just get tired of the discounting of shallow by some of the React maintainers and React team TBH. Just wanted to speak out on it here in this specific thread. They should be helping to support it IMO even if they don't write tests that way, their API needs to make things easier for those who do want to test other ways. Instead we're having to continually work against the React API to keep that in place and the fact that the React team in general opposes shallow rendering doesn't help the cause.

it encourages brittle tests that verify implementation detail.

this is a real problem because that statement I've seen over and over. It all depends how you are writing your tests. I have not had a lot of brittle issues the way I use it. Your tests should also be breaking, for the right reasons. There is a sense that they should never break, or rarely break and if they do, it's automatically "fragile" or "bad". There are ways to write tests differently even lower down with shallow and unit that do not cause those issues to happen.

So when I see blanket statements like this, it drives me batty. In fact I have had more brittleness using even more mocks, troubleshooting integration tests by debugging, having my tests be coupled to the UI more and having slow tests be a big pain in the ass quite frankly so that's why some like myself are pushing back on those kinds of blanket statements.

Moreover what's even more important is that we all should be pushing logic out of our UI components leaving humble views...and unit testing that functional core behavior underneath UI components.

[gaearon]

Over the years, the shallow renderer has stagnated. This was partially because we don’t use it ourselves anymore. So it’s difficult for us to make any design decisions around it (how to represent context in it? how should it work with refs and effects?).

Some people, including you, clearly find it useful. That’s great. The point of moving it out of the repository was to let folks who want to work on its design and implementation do exactly that without waiting for us. Now it’s fully decoupled so you’re welcome to file issues on that repository and help maintain and design it. I’m not sure what you’re unhappy with — that seems like the outcome most aligned with your preferences.

You said that we design APIs with not enough regard for testing. I don’t believe this is true. But if you have any specific feedback you are always welcome to express it as a comment on the relevant RFCs: https://reactjs.org/blog/2017/12/07/introducing-the-react-rfc-process.html.

What I ultimately hear from your message is you’re unhappy with us not specifically recommending shallow rendering. That’s okay. We all have opinions informed by our experience, and we need to be accepting of these differences. If shallow renderer serves you well, you are always welcome to become more involved in its design and development. That’s what extracting the repo was for.

[dschinkel]

Thanks for understanding Dan.

[mikeborozdin]

@dschinkel ,

I'm a big fan of TDD and I use shallow rendering a lot. So I created a library that brings hooks, such as useEffect() to shallow rendering - jest-react-hooks-shallow.

That said, once the shallow renderer is a proper standlone library, it'll be possible to extend it. So that we don't even have to rely on my library.

[gaearon]

So I created a library that brings hooks, such as useEffect() to shallow rendering - jest-react-hooks-shallow.

Btw I found this wording a bit confusing in announcements, since shallow renderer supports Hooks perfectly fine. It doesn't support effects specifically — not because it's hard to do, but as a design decision — just like it never supported running componentDidMount/Update. Those methods tend to use refs so we don't think it makes a lot of sense to run them with shallow rendering.

I know Enzyme has a different stance on this (partially stemming from a misunderstanding where folks working on it thought refs might be null in *Did* methods so good code would guard against null refs regardless). But it's a bit misleading to say shallow renderer "didn't support Hooks".

[dschinkel]

@gaearon

shallow renderer supports Hooks perfectly fine

What a lot of us want to do is test part of public our component's public API directly on instance like we were able to with classes. Test our own custom functions that mutate state, again only a subset of functions we decide to expose that makes sense (depends on how you are designing x component to choose what that will be) during shallow rendering.

I push as much logic down and out of my components and unit test, but we still want to be able to test a bit of the remaining little logic we have in our React Hooks and not have to write up heavy tests (mount, heavy mocking, or through button wiring or events) in order to invoke that part of our App.

Hooks appears to literally prevent you from doing so, from being able to expose some methods as public on the actual hook to provide as the public API from your tests. I don't want to get into this "testing implementation details" argument because that's a big conversation and that truly depends on how you are writing your tests and WHAT you deem as the public API.

So when you say "shallow supports hooks" can you briefly explain what you mean by that in sum?

[dschinkel]

@mikeborozdin @NMinhNguyen
Appreciate your efforts so far, I really do, because you care about flexible testing and simplicity of tests.

The way some of us test or really it's a lean workflow where we TDD in fact, and I know others are doing as well, is that we don't like testing through so much imperative shell, or in other words so much of the DOM wiring. Contrary to popular belief that this is "best/good practice", a lot of have another "best/good practice" that works nicely and allows us to bypass that wiring, and get enough confidence and more direct prod design feedback by testing a small part of the public API of our React components (e.g. Domain methods in the component) as the "contract" and we don't view the full DOM and its wiring as a required or critical part of the "contract".

So when we render we actually had a very good approach to shallow render then by calling and exposing carefully/smartly, a subset of instance methods (the public API - custom methods of our own, that we do decide to expose without exposing implementation details) from our shallowed tests. We still didn't test implementation details (like exposing internal data structures or react API "stuff" and testing those via props or spying on them...that's just bad period)...we test part of our public API via domain/behavioral methods, which does not expose and hides low level details; so we are still invoking and testing implementation indirectly.

So right now I can't really use your contributions because it doesn't resolve the inability to test "under the skin" a little. It seems like with both your wrappers, you still have to test through DOM wiring, events, and such to invoke behavior in a React Hook (and of course we still want to indirectly assert via output of the render which we can). I still see all examples in your repo doing all invoking through button, events, and other "wiring".

The Second problem is with mounted components (unrelated to your lib); we shouldn't have to rely on Jest mocks (whether manually prescribing them or auto) as a commonly suggested hack mimic to try and mimic shallow, in order to get around shallow shortcomings. I come from a land where mocking everything is a big smell (which I can certainly explain smell in detail but not here). Mocking Angular mounted components via TestBed; it's absolute hell. The fact that we're now being suggested to just do that in the React Community by many, really is bad. We should be able to test a lib like React with whatever testing framework we want using shallow and there should be some support for this somehow, somewhere on this planet.

make sense?