refactor: move module source to root

[JustinBeckwith]

The additional subdir was unnecessary, and I suspect it's masking a few issues.

[stanley-cheung]

Looks like after the move, this line https://github.com/GoogleCloudPlatform/grpc-gcp-node/blob/master/grpc-gcp/test/integration/spanner_test.js#L25 needs to be fixed.

[stanley-cheung]

Looks like a leading / is missing

[JustinBeckwith]

@stanley-cheung the errors in the logs appear to be related to permissions....

Uncaught AssertionError [ERR_ASSERTION]: ifError got unwanted exception: 7 PERMISSION_DENIED: Caller is missing IAM permission spanner.sessions.create on resource projects/grpc-gcp/instances/sample/databases/benchmark.

Is this somehow assuming I personally have access to the gcp project? Or did I mess something else up 😆

[stanley-cheung]

I got to here: https://github.com/GoogleCloudPlatform/grpc-gcp-node/blob/master/.kokoro/system-test.sh#L22

That's the service account credentials used for that spanner_test, but I couldn't figure out who set that KOKORO_GFILE_DIR environment variable.

Ok, the KOKORO_GFILE_DIR should correspond to this resource: https://github.com/GoogleCloudPlatform/grpc-gcp-node/blob/master/.kokoro/presubmit/node10/system-test.cfg#L2: gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/google-cloud-nodejs", but I couldn't quite figure out where this leads to.

[stanley-cheung]

@bcoe Hi, I noticed it was you who added this system-test.sh and added the service account credentials here: https://github.com/GoogleCloudPlatform/grpc-gcp-node/blob/master/.kokoro/system-test.sh#L22-L23. Do you have some idea about this error: IAM permission spanner.sessions.create on resource projects/grpc-gcp/instances/sample/databases/benchmark?

I got to the long-door-651 GCP project, but when I click on Spanner from the navigation on the left, I got this error:

You do not have sufficient permissions to view this page
There was an error while loading /spanner/instances?folder=&organizationId=&project=long-door-651.

You are missing at least one of the following required permissions:
Project

spanner.instanceConfigs.list
spanner.instances.list
Check that the project ID is valid and you have permissions to access it. 

[stanley-cheung]

On the other hand, #82 seems to be the last PR who has this "ci/kokoro: System test" passed, but the PR #82 was merged on about 10 months ago 2020/07/07. I couldn't find any other successful run after that point.

So at least it seems like the Spanner permission error is not related to the change introduced in this PR.

[stanley-cheung]

@bcoe Gentle ping. We are having some permission issues here with the kokoro system test you introduced a while ago. Thanks!

[bcoe]

@stanley-cheung are you able to grant permissions on projects/grpc-gcp/instances/sample/databases/benchmark for the service account,

kokoro-system-test@long-door-651.iam.gserviceaccount.com

I believe at some point these must have been revoked, which appears to be leading to the failures.

[stanley-cheung]

@bcoe I don't think I have permission in the long-door-651 project. After I selected that project and try to click on Spanner, I got a permission error.

[bcoe]

@stanley-cheung I believe the permission needs to be granted on the project grpc-gcp, which is what unit tests appear to be talking to.

I'm not sure who created this testing project?

[stanley-cheung]

I checked with @WeiranFang and we both don't have access to the project grpc-gcp now.

I am still a little bit confused about this long-door-651 project here though: https://github.com/GoogleCloudPlatform/grpc-gcp-node/blob/master/.kokoro/system-test.sh#L23. long-door-651 seems to be the project ID corresponding to the project with the name google-cloud-node. @JustinBeckwith seems to be an Owner for that project google-cloud-node.

kokoro-system-test@long-door-651.iam.gserviceaccount.com does indeed has IAM permission for Cloud Spanner Database Admin under this google-cloud-node project. https://pantheon.corp.google.com/iam-admin/iam?folder=&organizationId=&project=long-door-651

So where do we go from here?

[stanley-cheung]

@JustinBeckwith Perhaps you can try if you can give me Owner access to this google-cloud-node project?

[bcoe]

@stanley-cheung the reason I'm fixating on the project grpc-gcp is that the errors I'm seeing in the log are as follows:

   Uncaught AssertionError [ERR_ASSERTION]: ifError got unwanted exception: 7 PERMISSION_DENIED: Caller is missing IAM permission spanner.sessions.create on resource projects/grpc-gcp/instances/sample/databases/benchmark.

Which suggests that the long-door-651 service account is attempting to interact with the resource:

projects/grpc-gcp/instances/sample/databases/benchmark

I have access to the long-door-651 project, but I don't know that we can get these existing tests to pass if we don't sort out its inability to access this external resource.

Could we potentially skip the tests that are the culprit?

[JustinBeckwith]

We need to get this resolved so we can safely land dependency updates and security fixes. Instead of trying to piece together how to use multiple GCP projects, @stanley-cheung could your team create a new GCP project responsible for running CI? Or if you like, we can figure out how to completely use our CI project - but being split between two is obviously causing issues :) @stanley-cheung what do you think? How do we get this unstuck?

[stanley-cheung]

Yea we will probably need to consolidate this into one single GCP project. I am open to which one. @bcoe which project should we use going forward? As long as we all have permission I am ok with either one. That should make maintenance going forward easier.

[bcoe]

Apologies for the slow reply.

@stanley-cheung let's use long-door-651, if that's okay with you. You have the access you need?

[nimf]

@JustinBeckwith did we replace kokoro with cloudbuild or we are going to bring back kokoro? The system-test is fixed now.