feat: add local dev experience for scheduled functions #3689

netlify-team-account-1 · 2021-11-23T07:10:31Z

🎉 Thanks for submitting a pull request! 🎉

Summary

This PR adds special behaviour for scheduled functions. To mimic planned production behaviour, the local dev environment will not accept HTTP calls to scheduled functions. It will accept calls made through ntl functions:invoke.

A picture of a cute animal (not mandatory, but encouraged)

github-actions · 2021-11-23T07:11:09Z

📊 Benchmark results

Comparing with 40b1cf9

Package size: 360 MB

⬆️ 0.01% increase vs. 40b1cf9

^  359 MB  359 MB  359 MB  359 MB  357 MB  357 MB  357 MB  357 MB  357 MB  357 MB  357 MB  357 MB  360 MB 
│   ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐    ┌──┐  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
└───┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴──>
    T-12    T-11    T-10    T-9     T-8     T-7     T-6     T-5     T-4     T-3     T-2     T-1      T

Legend

T-30 (40b1cf9): 360 MB
T-29 (39eb031): 360 MB
T-28 (8720854): 360 MB
T-27 (7bb0322): 360 MB
T-26 (4e5eb1a): 360 MB
T-25 (3d7a4e1): 360 MB
T-24 (b247e69): 360 MB
T-23 (4d58ae5): 360 MB
T-22 (826ef86): 360 MB
T-21 (4c13e5d): 360 MB
T-20 (3c807bc): 360 MB
T-19 (68ef285): 360 MB
T-18 (b41c4c9): 360 MB
T-17 (918d1ad): 360 MB
T-16 (ab8276f): 360 MB
T-15 (2722b53): 360 MB
T-14 (a50c4d9): 359 MB
T-13 (d588dd2): 359 MB
T-12 (f66cb40): 359 MB
T-11 (647647b): 359 MB
T-10 (372a438): 359 MB
T-9 (8c8cb84): 359 MB
T-8 (dd39ea2): 357 MB
T-7 (b15f228): 357 MB
T-6 (d0e93d6): 357 MB
T-5 (79ac875): 357 MB
T-4 (6ddf42a): 357 MB
T-3 (eff74d0): 357 MB
T-2 (142cd30): 357 MB
T-1 (2c28f21): 357 MB
T (current commit): 360 MB

eduardoboucas

Looks great! Left a few minor comments.

src/lib/functions/netlify-function.js

src/lib/functions/schedule.js

src/lib/functions/server.js

tests/command.functions.test.js

eduardoboucas

LGTM! Once we bring this branch up-to-date and get the tests passing, I think we're good to go.

src/lib/functions/runtimes/js/builders/zisi.js

netlify-team-account-1 · 2021-12-20T10:05:18Z

77ef972 updates to the prerelease of netlify/zip-it-and-ship-it#899 (comment) to move this PR forward

eduardoboucas

Left a couple of comments, none of them blocking for the beta launch.

src/lib/functions/server.js

src/commands/functions/functions-invoke.js

eduardoboucas · 2022-01-05T15:00:12Z

src/commands/functions/functions-invoke.js

-  let headers = {}
+  const headers = {
+    'user-agent': 'netlify-cli',
+  }
  let body = {}

  if (eventTriggeredFunctions.has(functionToTrigger)) {


Do you think scheduled functions should fall on this branch?

I think this is addressed by c61a214#diff-9664db1fa0286aca02effcde71a026913114080a1c960e2257a357718ebb41eeR165-R171

netlify-team-account-1 · 2022-01-24T17:37:32Z

we decided to move forward with the change. let's get this shipped! 🚢 @eduardoboucas @lukasholzer would love another review :)

lukasholzer · 2022-01-26T16:57:44Z

Something is killing the tests: Command was killed with SIGKILL (Forced termination):

lukasholzer

Looks really good! Nice PR 🐳, some small nit pick

lukasholzer · 2022-01-24T08:30:38Z

src/commands/functions/functions-invoke.js

@@ -129,6 +130,13 @@ const getFunctionToTrigger = function (options, argumentName) {
  return argumentName
 }

+const getNextRun = function (schedule) {
+  const cron = CronParser.parseExpression(schedule, {
+    tz: 'Etc/UTC',


do we document that somewhere that we rely here un UTC and not on the users timezone?

yes, will be document in the Labs docs.

src/lib/functions/runtimes/js/builders/zisi.js

lukasholzer · 2022-01-26T17:01:09Z

src/lib/functions/scheduled.js

+
+const handleScheduledFunction = ({ error, request, response, result }) => {
+  const acceptsHtml = request.headers.accept && request.headers.accept.includes('text/html')
+  const paragraph = (text) => {


NIT: I think you can move this out of this function as you don't need the outer context

it needs acceptsHtml: https://github.com/netlify/cli/pull/3689/files/fa6f2d514dd00267573708e859b751a698f88a64#diff-4227587c310bc932a01198e05d935b060aa74a99364c8848af38478654d493beR12

lukasholzer · 2022-01-26T17:03:10Z

src/lib/functions/scheduled.js

+
+const { formatLambdaError } = require('./utils')
+
+const handleScheduledFunction = ({ error, request, response, result }) => {


could you please write a small unit test for this?

done!
6015fd6

Co-authored-by: Lukas Holzer <lukas.holzer@netlify.com>

netlify-team-account-1 · 2022-01-27T08:14:59Z

Something is killing the tests: Command was killed with SIGKILL (Forced termination)

@lukasholzer do you have an idea what's causing this? do we know if it's a problem with the PR, or an external thing?

lukasholzer · 2022-01-27T09:52:55Z

Something is killing the tests: Command was killed with SIGKILL (Forced termination)

@lukasholzer do you have an idea what's causing this? do we know if it's a problem with the PR, or an external thing?

My first wild guess would be that it is somehow connected with the PR as the error is somehow related to functions and I've never seen this one before.

So if tests are flaky then they are failing differently (by http endpoint timed out or something like that).

This one seems to be more on the mem crashing side of things on macos :D

while the ubuntu-latest test seems to be something different but related to scheduled functions:

lukasholzer · 2022-01-27T09:53:49Z

@netlify-team-account-1 do they pass locally?

co-authored-by: Lukas Holzer <lukas.holzer@netlify.com>

lukasholzer · 2022-01-28T15:29:35Z

@Skn0tt still not passing? 😬

Skn0tt · 2022-01-28T15:31:56Z

nope

netlify-team-account-1 added the type: feature code contributing to the implementation of a feature and/or user facing functionality label Nov 23, 2021

netlify-team-account-1 requested a review from eduardoboucas November 23, 2021 07:10

eduardoboucas reviewed Nov 23, 2021

View reviewed changes

src/lib/functions/netlify-function.js Outdated Show resolved Hide resolved

src/lib/functions/schedule.js Outdated Show resolved Hide resolved

src/lib/functions/server.js Outdated Show resolved Hide resolved

tests/command.functions.test.js Outdated Show resolved Hide resolved

Skn0tt force-pushed the scheduled-jobs-endpoint branch from 93878c1 to 3fa4c14 Compare November 23, 2021 13:20

eduardoboucas previously approved these changes Dec 13, 2021

View reviewed changes

Skn0tt dismissed eduardoboucas’s stale review via c6fa092 December 14, 2021 16:55

netlify-team-account-1 commented Dec 14, 2021

View reviewed changes

src/lib/functions/runtimes/js/builders/zisi.js Outdated Show resolved Hide resolved

netlify-team-account-1 added 10 commits December 20, 2021 10:24

feat: add local dev experience for scheduled functions

e567731

feat: use synchronous function's timeout

0f3dd2d

chore: rename to scheduled.js

304239c

feat: return 400 when trying to invoke scheduled function via http

362159b

chore: use testMatrix pattern

bbc9675

fix: config-based schedule without esbuild

cc8800c

feat: add support for ISC-declared flags

9ba29de

fix: node v12 doesn't understand optional chaining

db5c942

fix: allow esbuild to read mock files by writing them to disk

2f4e100

fix: wrong import

c14699b

Skn0tt force-pushed the scheduled-jobs-endpoint branch from 36b6c20 to c14699b Compare December 20, 2021 09:27

netlify-team-account-1 added 2 commits December 20, 2021 10:59

feat: use listFunction to detect ISC schedule

77ef972

feat: remove unused feature flag

304fba9

netlify-team-account-1 added 4 commits December 21, 2021 12:35

Merge branch 'main' into scheduled-jobs-endpoint

0c1eac8

chore: update zisi

11cbe9f

Merge branch 'main' into scheduled-jobs-endpoint

819ebc1

fix: enable parseISC hook

7481c2a

eduardoboucas previously approved these changes Jan 5, 2022

View reviewed changes

src/lib/functions/server.js Outdated Show resolved Hide resolved

src/commands/functions/functions-invoke.js Show resolved Hide resolved

eduardoboucas reviewed Jan 5, 2022

View reviewed changes

netlify-team-account-1 added 2 commits January 6, 2022 09:03

feat: give full command

227fe7a

refactor: move clockwork simulation to calling side

c61a214

netlify-team-account-1 added 3 commits January 24, 2022 17:55

fix: optional chaining doesnt exist

30e7ba4

chore: add test for helpful tips and tricks

9f8de55

fix: correct tests

07add5b

netlify-team-account-1 requested review from eduardoboucas and lukasholzer January 24, 2022 17:37

fix: add missing property to test

fa6f2d5

lukasholzer previously approved these changes Jan 26, 2022

View reviewed changes

Update src/lib/functions/runtimes/js/builders/zisi.js

b8d06e1

Co-authored-by: Lukas Holzer <lukas.holzer@netlify.com>

netlify-team-account-1 dismissed lukasholzer’s stale review via b8d06e1 January 27, 2022 07:43

netlify-team-account-1 and others added 3 commits January 27, 2022 09:09

refactor: extract help response into its own testable function

c266016

chore: add some unit tests

6015fd6

Merge branch 'main' into scheduled-jobs-endpoint

a866d96

netlify-team-account-1 and others added 4 commits January 28, 2022 10:07

fix: replaceAll not available on node v12

603d146

co-authored-by: Lukas Holzer <lukas.holzer@netlify.com>

fix: remove unneeded level field

acb4504

refactor: remove unused test matrix

9e5fd20

fix: increase file change delay for macOS and windows

fea14a8

netlify-team-account-1 requested a review from lukasholzer January 28, 2022 10:31

lukasholzer approved these changes Jan 28, 2022

View reviewed changes

Merge branch 'main' into scheduled-jobs-endpoint

17285f4

lukasholzer merged commit 1c8968d into main Jan 28, 2022

lukasholzer deleted the scheduled-jobs-endpoint branch January 28, 2022 16:14

token-generator-app bot mentioned this pull request Jan 28, 2022

chore(main): release 8.15.0 #4144

Merged

token-generator-app bot mentioned this pull request May 26, 2022

chore(main): release 10.3.3 #4629

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add local dev experience for scheduled functions #3689

feat: add local dev experience for scheduled functions #3689

netlify-team-account-1 commented Nov 23, 2021

github-actions bot commented Nov 23, 2021 •

edited

Loading

eduardoboucas left a comment

eduardoboucas left a comment

netlify-team-account-1 commented Dec 20, 2021

eduardoboucas left a comment

eduardoboucas Jan 5, 2022

netlify-team-account-1 Jan 7, 2022

netlify-team-account-1 commented Jan 24, 2022 •

edited

Loading

lukasholzer commented Jan 26, 2022

lukasholzer left a comment

lukasholzer Jan 24, 2022

netlify-team-account-1 Jan 27, 2022

lukasholzer Jan 26, 2022

netlify-team-account-1 Jan 27, 2022

lukasholzer Jan 26, 2022

netlify-team-account-1 Jan 27, 2022

netlify-team-account-1 commented Jan 27, 2022 •

edited

Loading

lukasholzer commented Jan 27, 2022 •

edited

Loading

lukasholzer commented Jan 27, 2022

lukasholzer commented Jan 28, 2022

Skn0tt commented Jan 28, 2022


		const { formatLambdaError } = require('./utils')

		const handleScheduledFunction = ({ error, request, response, result }) => {

feat: add local dev experience for scheduled functions #3689

feat: add local dev experience for scheduled functions #3689

Conversation

netlify-team-account-1 commented Nov 23, 2021

Summary

github-actions bot commented Nov 23, 2021 • edited Loading

📊 Benchmark results

Package size: 360 MB

eduardoboucas left a comment

Choose a reason for hiding this comment

eduardoboucas left a comment

Choose a reason for hiding this comment

netlify-team-account-1 commented Dec 20, 2021

eduardoboucas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

netlify-team-account-1 commented Jan 24, 2022 • edited Loading

lukasholzer commented Jan 26, 2022

lukasholzer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

netlify-team-account-1 commented Jan 27, 2022 • edited Loading

lukasholzer commented Jan 27, 2022 • edited Loading

lukasholzer commented Jan 27, 2022

lukasholzer commented Jan 28, 2022

Skn0tt commented Jan 28, 2022

github-actions bot commented Nov 23, 2021 •

edited

Loading

netlify-team-account-1 commented Jan 24, 2022 •

edited

Loading

netlify-team-account-1 commented Jan 27, 2022 •

edited

Loading

lukasholzer commented Jan 27, 2022 •

edited

Loading