gRPC Protocol Modules

Generating the gRPC protocol modules

This step is no longer required, as the grpc scripts can now get the protocol module classes at run time via reflection, but generating the protocol modules will improve script startup time, and it would be a good idea to at least stash away the protoset file emitted by grpcurl (or extract_protoset.py) in case SpaceX ever turns off server reflection in the dish software. That being said, it's probably less error prone to use the run time reflection support rather than using the files generated in this step, so use your own judgement as to whether or not to proceed with this.

The grpc scripts require some generated code to support the specific gRPC protocol messages used. These would normally be generated from .proto files that specify those messages, but to date (2020-Dec), SpaceX has not publicly released such files. The gRPC service running on the dish appears to have server reflection enabled, though. grpcurl can use that to extract a protoset file, and the protoc compiler can use that to make the necessary generated code:

grpcurl -plaintext -protoset-out dish.protoset 192.168.100.1:9200 describe SpaceX.API.Device.Device
mkdir out
cd out
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/device/device.proto
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/common/status/status.proto
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/device/command.proto
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/device/common.proto
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/device/dish.proto
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/device/dish_config.proto
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/device/wifi.proto
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/device/wifi_config.proto
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/device/wifi_util.proto
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/device/transceiver.proto
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/satellites/network/ut_disablement_codes.proto
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/telemetron/public/common/time.proto
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/device/services/unlock/service.proto
python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/device/rssi_scan.proto

Then move the resulting files (ie: the spacex directory) to where the Python scripts can find them in the import path, such as in the same directory as the scripts themselves.

Note that the list of .proto files that need to be processed may shrink or grow over time as SpaceX makes changes in the gRPC service protocol. The full list can be pulled from the protoset file, or you can just run the starlink-grpc-tools scripts against them and keep adding any missing ones that result in missing _pb2 modules. It will be easier to find missing _pb2 modules if you uninstall yagrc (or just don't install it in the first place, as it is not needed when generating the protocol modules per these instructions).

Using extract_protoset.py

The extract_protoset.py script can be used in place of grpcurl to get the protoset file used in the prior section. Note that this script requires yagrc to be installed, contrary to the recommendation in the prior section. Usage for this case is simple:

python3 extract_protoset.py -v .

This will write the current protoset file to the current directory and print the filename it used. It will need to be renamed to dish.protoset in order to match the rest of the commands in the prior section.

This script is really targeted at the stash-away-the-protoset-file-in-case-server-reflection-ever-gets-disabled task, though. To this end, it can be run periodically to record the protoset file as the firmware is updated, or just run with looping enabled. Use of a dedicated directory for output is recommended in this case. Use -h command line option to get full list of options. Note that the script's loop interval is specified in seconds for consistency with the other scripts in this project, but checking the protocol data once per day is probably sufficient.