fix: fix local builds with plugins and build settings from the ui

[danez]

Summary

This is a long story. I was debugging yesterday for hours.

Problem

The initial problem was that I tried deploying a nextjs site from the cli and it did not work. The setup is as following:

• project cloned from nextra a nextjs documentation application
• No local config (netlify.toml)
• in the netlify UI I configured github connection, build command and publish directory.
• netlify link done

When I tried deploying this site, the nextjs-plugin would be installed automatically, but during the build at first the onPreBuild would run for the nextjs-plugin. Next the onBuild for the nextjs-plugin woudl run and this fails because the build command was not triggered at all.

Investigation

After hours of trying to understand @netlify/build and it's connection to the cli and to @netlify/config I found out what is happening. Let me try to summarize:

1. The cli build command is triggered
2. In the cli the base-command calls @netlify/config for the config and gets the correct config back (including ui settings)
3. @netlify/build is called with the config supplied from the cli
4. resolveInitialConfig is called in @netlify/build which respects the passed in config from the cli. all fine so far
5. The onPreBuild event is triggered for the build plugin and the plugin makes changes to the config.
6. After the plugin is finished, @netlify/build computes a config changeset called configMutation. This details exactly what the plugin changed in the config, in order to be able to print the changes to the output but also to be always able to reapply these changes again to a config object.
7. We now call @netlify/config to get a fresh copy of the config and with the configMutations applied. Because we didn't supplied siteId in the first place this now failed to grab anything from the API which is site related. So at this point we basically revert the config to some default values (no build command, publish directory = root directory, etc.) and this is the reason that my build command is never run.

By adding the siteId to the input Options it all starts working again, and this is now inline with buildbot, which also supplies the siteId.

[danez]

This, and this only is the fix, the rest is for the tests.

[github-actions]

📊 Benchmark results

Comparing with f1d1d91

Package size: 222 MB

(no change)

^  222 MB  222 MB  228 MB                                                                                 
│   ┌──┐    ┌──┐    ┌──┐   222 MB  222 MB  222 MB  222 MB  222 MB  222 MB  222 MB  222 MB  222 MB  222 MB 
│   |  |    |  |    |  |    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
└───┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴──>
    T-12    T-11    T-10    T-9     T-8     T-7     T-6     T-5     T-4     T-3     T-2     T-1      T    

Legend

• T-30 (dc0615e): 228 MB
• T-29 (27b4843): 222 MB
• T-28 (1e83238): 222 MB
• T-27 (ea87d45): 222 MB
• T-26 (d00841d): 222 MB
• T-25 (060c205): 222 MB
• T-24 (d2d7665): 222 MB
• T-23 (55df500): 222 MB
• T-22 (e1b911f): 222 MB
• T-21 (58732a2): 222 MB
• T-20 (ddeec5e): 222 MB
• T-19 (ec1a05e): 222 MB
• T-18 (9e1e55b): 222 MB
• T-17 (0c90be7): 222 MB
• T-16 (1c6107f): 222 MB
• T-15 (75d581f): 222 MB
• T-14 (e9fa5e3): 222 MB
• T-13 (f6c3d78): 222 MB
• T-12 (85e652e): 222 MB
• T-11 (1914c3f): 222 MB
• T-10 (11e657c): 228 MB
• T-9 (50035ba): 222 MB
• T-8 (016b718): 222 MB
• T-7 (c90adde): 222 MB
• T-6 (c6eb30e): 222 MB
• T-5 (9fa41a5): 222 MB
• T-4 (94415f9): 222 MB
• T-3 (8e50080): 222 MB
• T-2 (886eebc): 222 MB
• T-1 (f1d1d91): 222 MB
• T (current commit): 222 MB

[danez]

This is not nice,.
The host could be supplied by setting apiHost but the scheme cannot be changed besides the testOpts.

I'm also not sure if NETLIFY_API_URL is only for tests or if that is meant to be used by users?

Should I add apiScheme to @netlify/build? Or is it okay this way?

[danez]

I opted to not set it for every test here, because it does not really reflect real world scenarios.
If NETLIFY_SITE_ID is set this bug I'm fixing here never surfaces, because @netlify/build and @netlify/config both also read it.
But most users never set it probably.

I instead opted to add a new method to the site-builder withStateFile, which mimics netlify link

[ericapisani]

Quite the journey to get this bug sorted. Thanks for the detailed summary and the fix, this LGTM 🚀