The proto zip release asset does not contain the google/rpc/status.proto file for v1.1.0

[dankeboy36]

Describe the problem

The google/rpc/status.proto has been removed from the 1.1.0 release, although it is used (for example, import "google/rpc/status.proto"; in cc/arduino/cli/commands/v1/commands.proto), and downstream it breaks the proto to TS/JS generation. I believe the Arduino IDE will most likely also encounter this issue. I am about to release a new TS API for the CLI, and I am getting an error. It worked fine with the 1.0.4 release: https://www.npmjs.com/package/ardunno-cli/v/0.1.9?activeTab=code.

To reproduce

Use curl/wget or another way to download the 1.0.4 proto zip and the 1.1.0 version. Unpack them and see the status.proto is not packed with the zip.

% unzip arduino-cli_1.1.0_proto.zip -d 1.1.0-out
Archive:  arduino-cli_1.1.0_proto.zip
  inflating: 1.1.0-out/cc/arduino/cli/commands/v1/commands.proto  
  inflating: 1.1.0-out/cc/arduino/cli/commands/v1/compile.proto  
  inflating: 1.1.0-out/cc/arduino/cli/commands/v1/upload.proto  
  inflating: 1.1.0-out/cc/arduino/cli/commands/v1/common.proto  
  inflating: 1.1.0-out/cc/arduino/cli/commands/v1/lib.proto  
  inflating: 1.1.0-out/cc/arduino/cli/commands/v1/port.proto  
  inflating: 1.1.0-out/cc/arduino/cli/commands/v1/monitor.proto  
  inflating: 1.1.0-out/cc/arduino/cli/commands/v1/core.proto  
  inflating: 1.1.0-out/cc/arduino/cli/commands/v1/board.proto  
  inflating: 1.1.0-out/cc/arduino/cli/commands/v1/settings.proto  
  inflating: 1.1.0-out/cc/arduino/cli/commands/v1/debug.proto  
% unzip arduino-cli_1.0.4_proto.zip -d 1.0.4-out
Archive:  arduino-cli_1.0.4_proto.zip
  inflating: 1.0.4-out/cc/arduino/cli/commands/v1/settings.proto  
  inflating: 1.0.4-out/cc/arduino/cli/commands/v1/monitor.proto  
  inflating: 1.0.4-out/cc/arduino/cli/commands/v1/core.proto  
  inflating: 1.0.4-out/cc/arduino/cli/commands/v1/compile.proto  
  inflating: 1.0.4-out/cc/arduino/cli/commands/v1/port.proto  
  inflating: 1.0.4-out/cc/arduino/cli/commands/v1/board.proto  
  inflating: 1.0.4-out/cc/arduino/cli/commands/v1/commands.proto  
  inflating: 1.0.4-out/cc/arduino/cli/commands/v1/common.proto  
  inflating: 1.0.4-out/cc/arduino/cli/commands/v1/upload.proto  
  inflating: 1.0.4-out/cc/arduino/cli/commands/v1/debug.proto  
  inflating: 1.0.4-out/cc/arduino/cli/commands/v1/lib.proto  
  inflating: 1.0.4-out/google/rpc/status.proto

Escaped command to reproduce:

"/Users/akos.kitta/Desktop/dev/ardunno-cli/node_modules/protoc/protoc/bin/protoc" "--plugin=/Users/akos.kitta/Desktop/dev/ardunno-cli/node_modules/ts-proto/protoc-gen-ts_proto" "--proto_path=/var/folders/g0/n_2cw4ds44l2byzv39xrc4340000gp/T/arduino-cli-17215-XcWyqFmh6gjw/rpc" "--ts_proto_opt=outputServices=nice-grpc,outputServices=generic-definitions,oneof=unions,useExactTypes=false,paths=source_relative,esModuleInterop=true,exportCommonSymbols=false,useOptionals=none" "--ts_proto_out=src/api" "cc/arduino/cli/commands/v1/board.proto" "cc/arduino/cli/commands/v1/commands.proto" "cc/arduino/cli/commands/v1/common.proto" "cc/arduino/cli/commands/v1/compile.proto" "cc/arduino/cli/commands/v1/core.proto" "cc/arduino/cli/commands/v1/debug.proto" "cc/arduino/cli/commands/v1/lib.proto" "cc/arduino/cli/commands/v1/monitor.proto" "cc/arduino/cli/commands/v1/port.proto" "cc/arduino/cli/commands/v1/settings.proto" "cc/arduino/cli/commands/v1/upload.proto"

Error

google/rpc/status.proto: File not found.\n' +
    'cc/arduino/cli/commands/v1/commands.proto:30:1: Import "google/rpc/status.proto" was not found or had errors.\n' +
    'cc/arduino/cli/commands/v1/commands.proto:219:5: "google.rpc.Status" is not defined.

Expected behavior

google/rpc/status.proto is part of the release proto artifact

Arduino CLI version

1.1.0

Operating system

N/A

Operating system version

n/a

Additional context

No response

Issue checklist

• I searched for previous reports in the issue tracker
• I verified the problem still occurs when using the nightly build
• My report contains all necessary details

[cmaglie]

We recently migrated all protobuf tooling to buf: #2736

One of the nice features of buf is that it can handle the "dependencies" automatically https://github.com/arduino/arduino-cli/blob/master/buf.yaml#L3, so we don't have to mirror the status.proto. The downside is that we do not package it anymore.

We may download it in some way before zipping the artifact @alessio-perugini

[dankeboy36]

We recently migrated all protobuf tooling to buf: #2736

I have an unrelated dev issue due to the mission status file. I want to use grpcurl to verify the CLI behavior.

grpcurl \
  -plaintext \
  -import-path ./rpc \
  -proto cc/arduino/cli/commands/v1/commands.proto \
  127.0.0.1:50051 \
  cc.arduino.cli.commands.v1.ArduinoCoreService.Create
Failed to process proto source files.: could not parse given files: cc/arduino/cli/commands/v1/commands.proto:30:8: open rpc/google/rpc/status.proto: no such file or directory

Previously, it was convenient to check out the repo and try reproducing grpc problems from the source code, but now I need to copy files to do the same.

What steps are needed to get the missing status.proto file downloaded? I checked the updated docs from the PR, it mentions buf. I installed it.

I figured out from the task file that I needed to run task protoc. I did, but I am still getting errors when I am trying to test the CLI with grpcurl.

task protoc
task: [protoc:check] buf lint
task: [protoc:format] buf format --write --exit-code
task: [protoc:breaking-change-detection] buf breaking --against '.git#branch=origin/master,subdir=rpc'
task: [protoc:compile] buf dep update
task: [protoc:compile] buf generate

task protoc:compile
task: [protoc:compile] buf dep update
task: [protoc:compile] buf generate

Thank you!

[per1234]

Hi @dankeboy36. I don't have an answer to your specific question, but I can offer an alternative solution, which is to switch from the excellent grpcurl to using Buf CLI, which is now the preferred tool for the Arduino CLI project. Buf CLI will take care of providing this missing dependency for you.

Here is the equivalent command using Buf CLI:

npx @bufbuild/buf \
  curl \
    --http2-prior-knowledge \
    --protocol grpc \
    --schema ./rpc \
    http://localhost:50051/cc.arduino.cli.commands.v1.ArduinoCoreService/Create

[alessio-perugini]

@dankeboy36

# Assume you're in the `arduino-cli` folder

$ buf export . -o /tmp/test
$ tree /tmp/test

/tmp/test
├── cc
│   └── arduino
│       └── cli
│           └── commands
│               └── v1
│                   ├── board.proto
│                   ├── commands.proto
│                   ├── common.proto
│                   ├── compile.proto
│                   ├── core.proto
│                   ├── debug.proto
│                   ├── lib.proto
│                   ├── monitor.proto
│                   ├── port.proto
│                   ├── settings.proto
│                   └── upload.proto
└── google
    └── rpc
        └── status.proto

8 directories, 12 files

This will generate all the dependencies, and you can continue to use grpcurl if you wish 😄