Possible to gcloud functions deploy when using private GitHub packages?

[techieshark]

Summary

I've been getting an inexplicable error during deployment and I'm unable to debug it.

I have a private monorepo and am publishing individual private packages to it using GitHub Packages. It seems Cloud Build is able to fetch the package ("yarn" works fine) but it seems that package is mysteriously missing when the function actually executes. What am I missing?

Details

I have a .npmrc which looks like:

@myGithubOrgScopeName:registry=https://npm.pkg.github.com/
//npm.pkg.github.com/:_authToken=[my secret GitHub auth token]

In package.json, I have:

"dependencies": {
    "@myGithubOrgScopeName/my-package": "*",
     ...
}

I then have some npm scripts I run to deploy:

tsc # puts things in ./dist
cp .npmrc package.json ./dist/
gcloud functions deploy my-cloud-function --trigger-http --entry-point functionName --source ./dist --runtime nodejs10

When deploying, it seems the build phase works. The private package from the GitHub package repo seems to install - I can verify that by adding a postinstall script which just does ls node_modules/@myGithubOrgScopeName/:

Furthermore, I see no issues when viewing the Cloud Build logs via the url shown during the execution of gcloud functions deploy:

For Cloud Build Stackdriver Logs, visit: https://console.cloud.google.com/logs/viewer?project=our-project&advancedFilter=resource.type%3Dbuild%0Aresource.labels.build_id%3Defa76f49-503f-4ba3-90fa-50529bb5b7d4%0AlogName%3Dprojects%2Four-project%2Flogs%2Fcloudbuild

If it helps, here's the build id shown in the legacy logs viewer filter settings:

resource.type=build
resource.labels.build_id=efa76f49-503f-4ba3-90fa-50529bb5b7d4
logName=projects/our-project/logs/cloudbuild

However, the overall deploy fails ("on loading user code"):

ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging. Please visit https://cloud.google.com/functions/docs/troubleshooting for in-depth troubleshooting documentation.

When I go to Stackdriver Logs > Cloud Function > my function, I see that the module which should be from the GitHub package repo is not present (what?? it was there during deploy!).

Finally, I'm not sure how to further debug this, since:

a) The "Download Zip" feature on cloud functions console does not include the node_modules/, and also
b) it does include package.json and I can confirm that my desired package is in fact listed as a dependency (contrary to what the logs' suggestion indicates as a potential problem).

Is there a way, via the functions framework, to deploy to a local container I could then inspect? Or any other suggestions for how to fix this or what I might be doing wrong?

Thanks!

[grant]

Hi. I'm sorry @techieshark, this GitHub repo is not meant to file bugs against Cloud Functions. It's used to track issues directly in this npm module.

I don't know of this issue off hand.

For looking at the container, I believe you can build the container with Buildpacks and then use dive to inspect it:

https://cloud.google.com/functions/docs/building/pack
https://github.com/wagoodman/dive

For general support, please use this:

https://cloud.google.com/functions/docs/getting-support

[techieshark]

Hi @grant thanks for the quick response.

I'll take a look at the "pack" and "dive" links you shared and see if that helps for debugging. Really appreciate the pointers!

For the benefit of others who might land here, I found this open issue in the Google Issue Tracker which would suggest cloud functions doesn't actually support using GitHub package repo:

https://issuetracker.google.com/issues/147259775

[techieshark]

Thanks again @grant - the tip to use pack helped me sort out the issue, which wasn't that GitHub packages couldn't be installed, despite the Stackdriver log which seemed to indicate that.

For the sake of anyone landing here, I'll describe what I did.

First, build the cloud functions docker image (replace "myEntryPointFunction" with your entry point)

pack build --builder gcr.io/buildpacks/builder:v1 --env GOOGLE_RUNTIME=nodejs --env GOOGLE_FUNCTION_SIGNATURE_TYPE=http --env GOOGLE_FUNCTION_TARGET=myEntryPointFunction  your_cloud_function_image

Then open a shell in it:

docker run -it --entrypoint /bin/bash your_cloud_function_image

Inside the docker image, cd to /workspace and look around. Tip: Node is installed in a funny location (find / -name node turned it up in /layers/google.nodejs.runtime/node/bin/node).

Strangely, I got a very different error when trying to run the code (vs what I saw in Stackdriver log):

(So it turned out the issue was not that the entire module was missing - or that it wasn't installed from GitHub packages - but that I had the wrong file pointed to by the "main" field in the module's package.json.)

I hpoe this might help others. Thanks again for the tip!