Testing: React Native layout based snapshots

[orta]

So, we want to be able to generate reasonable visual layout snapshots. Because when we get bugs, they tend to be layout driven regressions, on iOS this was easy to handle and we made a few libraries to make it feel good.

High level, pixel-accurate representations of the layouts can only be done on a simulator, meaning they can't be fast.

Ideally we could look at taking the render tree of a snapshot, and create our own image representation from that.

A snapshot test looks like:

it('looks like expected', () => {
  const  gene = {...}
  const tree = renderer.create(<About gene={gene}/>).toJSON()
  ).toJSON()

  expect(tree).toMatchSnapshot()
})

I think it would make sense to add our own matcher:

  expect(tree).toMatchLayoutSnapshot({width: 540, height: 480})
  expect(tree).toMatchSnapshot()

Which would take the same rendered React tree JSON used in the snapshot but instead render through YogaJS npm: yoga-layout to an png of flat colors.

---

Rendering

The relative aim:

Conceptual things to consider

• Deciding whether to show a View subclass or not

• We mock out top-level Native Components in tests, so they might not see any styles applied, e.g. it might look more like this instead:

  
  Which I think is still a good trade-off.

• Color: We could use the Component name (and potentially state, might get messy though?) to derive a color, so that an Artwork component would always be the same color. This is a nice color generator I made for our iOS.

• File System representation: We could do the same thing as Jest, in that we make a __layouts__ folder? or even re-use __snapshots__ and instead of about-test.js.snap do about-test.js.layout?

Technical things to consider

• Speed: FBSnapshot is fast because it's doing a data to data comparison, we would probably have to do the same thing. There is an argument that maybe we don't need to actually do a comparison inside the matcher at all, if the layout was broken, chances are the toMatchSnapshot() would fail on the next line. We could in theory just always save to disk, git and jest-snapshots would record the changes.

• Size: In eigen snapshots we manipulate the size of the iOS device to fake iPhone vs iPad etc. We're already finding it tricky to do device-specific layout/tests, in part because of the shallow rendering [Dev] isPad is an anti-pattern and should be isolated #240 Figure out a way to test layouts that rely on onLayout callback #410 Use layout events instead of Dimensions #401 at the bare minimum we'd need to provide device dimensions to work with ( or else Yoga won't know what to do )

• Text rendering Do we apply sizing to text blocks based on the content? We have it in the JSON, and it can offer more insight into the layout of a component.

Proof of concept

In theory you would be able to to take a JSON object like this:

{
  "type": "View",
  "props": {},
  "children": [
    {
      "type": "View",
      "props": {},
      "children": [
        {
          "type": "View",
          "props": {
            "style": {
              "marginLeft": 0,
              "marginRight": 0
            }
          },
          "children": [
            {
              "type": "Text",
              "props": {
                "style": [
                  {
                    "fontSize": 17
                  },
                  {
                    "fontSize": 16,
                    "lineHeight": 20,
                    "marginBottom": 20
                  },
                  {
                    "fontFamily": "AGaramondPro-Regular"
                  }
                ],
                "numberOfLines": 0,
                "accessible": true,
                "allowFontScaling": true,
                "ellipsizeMode": "tail"
              },
              "children": [
                "Deep time refers to the concept of an expansive time that stretches far beyond human history to\n                  include the approximately 4.5 billion-year geological history of Earth and the estimated 13.8\n                  billion-year history of the universe. In art, deep time could be explored in works that deal with the\n                  long-term processes of geological formation and the cosmos. Such concerns are important, for example,\n                  to Hiroshi Sugimoto’s waterscapes and Darren Almond’s landscapes, both of\n                  which use long-exposure photography to capture a sense of expanded time."
              ]
            }
          ]
        },
        {
          "type": "View",
          "props": {
            "style": [
              {
                "height": 1,
                "marginLeft": -20,
                "marginRight": -20,
                "backgroundColor": "#e5e5e5"
              },
              {
                "marginBottom": 20
              }
            ]
          },
          "children": null
        }
      ]
    },
    {
      "type": "View",
      "props": {
        "style": {
          "flexDirection": "column"
        }
      },
      "children": [
        {
          "type": "Text",
          "props": {
            "style": [
              {
                "fontSize": 17
              },
              {
                "fontSize": 20
              },
              {
                "fontFamily": "AGaramondPro-Regular"
              }
            ],
            "numberOfLines": 1,
            "accessible": true,
            "allowFontScaling": true,
            "ellipsizeMode": "tail"
          },
          "children": [
            "Related Artists"
          ]
        },
        {
          "type": "View",
          "props": {
            "style": {
              "flexWrap": "wrap",
              "flexDirection": "row",
              "alignItems": "flex-start",
              "justifyContent": "space-around",
              "marginTop": 12,
              "marginLeft": -10,
              "marginRight": -10
            }
          },
          "children": [
            {
              "type": "TouchableWithoutFeedback",
              "props": {},
              "children": [
                {
                  "type": "View",
                  "props": {
                    "style": {
                      "margin": 5,
                      "paddingBottom": 20,
                      "width": 1
                    }
                  },
                  "children": [
                    {
                      "type": "ImageView",
                      "props": {
                        "style": {
                          "width": 1,
                          "height": 1
                        },
                        "imageURL": "cloudfront.net/some-url"
                      },
                      "children": null
                    },
                    {
                      "type": "Text",
                      "props": {
                        "style": {
                          "fontSize": 12,
                          "textAlign": "left",
                          "marginTop": 10,
                          "fontFamily": "Avant Garde Gothic ITCW01Dm"
                        },
                        "accessible": true,
                        "allowFontScaling": true,
                        "ellipsizeMode": "tail"
                      },
                      "children": [
                        "LITA ALBUQUERQUE"
                      ]
                    },
                    {
                      "type": "Text",
                      "props": {
                        "style": {
                          "fontFamily": "AGaramondPro-Regular",
                          "fontSize": 16,
                          "marginTop": 5,
                          "color": "#666666"
                        },
                        "accessible": true,
                        "allowFontScaling": true,
                        "ellipsizeMode": "tail"
                      },
                      "children": [
                        "36 works, 30 for sale"
                      ]
                    }
                  ]
                }
              ]
            },
            {
              "type": "TouchableWithoutFeedback",
              "props": {},
              "children": [
                {
                  "type": "View",
                  "props": {
                    "style": {
                      "margin": 5,
                      "paddingBottom": 20,
                      "width": 1
                    }
                  },
                  "children": [
                    {
                      "type": "ImageView",
                      "props": {
                        "style": {
                          "width": 1,
                          "height": 1
                        },
                        "imageURL": "cloudfront.net/some-url"
                      },
                      "children": null
                    },
                    {
                      "type": "Text",
                      "props": {
                        "style": {
                          "fontSize": 12,
                          "textAlign": "left",
                          "marginTop": 10,
                          "fontFamily": "Avant Garde Gothic ITCW01Dm"
                        },
                        "accessible": true,
                        "allowFontScaling": true,
                        "ellipsizeMode": "tail"
                      },
                      "children": [
                        "DOUG ARGUE"
                      ]
                    },
                    {
                      "type": "Text",
                      "props": {
                        "style": {
                          "fontFamily": "AGaramondPro-Regular",
                          "fontSize": 16,
                          "marginTop": 5,
                          "color": "#666666"
                        },
                        "accessible": true,
                        "allowFontScaling": true,
                        "ellipsizeMode": "tail"
                      },
                      "children": [
                        "31 works, 24 for sale"
                      ]
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

and generate an image like:

[orta]

@cpojer does this feel about right to you?

[alloy]

I was actually thinking more of expanding on the existing JSON tests with Yoga in such a way that the various dynamic layouting we do in the app based on screen size actually is reflected in the JSON.

[orta]

Isn't that mostly handled by faking our own LayoutEvent? #400

I think we'd need to do more than just shallow rendering, e.g. like enzyme may be of use. Which I think would run the RN layouting algorithm? ( though they imply it is shallow, so maybe not)

We may also have more luck with a better abstraction on how we handle device sizes #240 - media queries are something we can probably DI in per-test easily.

Otherwise it could be that we'd maybe need to build a version of React Shallow Renderer that mixes Yoga in with the renderer. Considering ^ is only 22 LOC, the real work is happening inside React - so maybe there's somewhere we can hook in.

[cpojer]

this sounds incredibly exciting. I would love to see this experiment and if it is successful I'd be supportive to incorporate it into Jest somehow or into the Jest repo as a separate package.

[orta]

We ended up more issues that could have been handled by something like this. So, I've created a repo and set up all the required infra in terms of testing and building this out, https://github.com/orta/jest-snapshots-svg

Wil try spend an hour or two a day on it, as I'm building something highly visual myself now in RN.

[alloy]

@orta started on this, I don’t think we need to keep this open any longer.