Operating System 24 | gRPC and protobuf Starter Tutorial

Series: Operating System

Operating System 24 | gRPC and protobuf Starter Tutorial

1. gRPC Setup

In C++, we need to build and install gRPC before we test our Hello World example. Suppose we have MacOS, and first we need to choose a directory to hold locally installed packages. This page assumes that the environment variable MY_INSTALL_DIR holds this directory path.

$ export MY_INSTALL_DIR=$HOME/.local

Ensure that the directory exists,

$ mkdir -p $MY_INSTALL_DIR

Add the local bin folder to your path variable,

$ export PATH="$PATH:$MY_INSTALL_DIR/bin"

You need version 3.19.6 or later of cmake. Install it by,

$ wget -q -O cmake-linux.sh https://github.com/Kitware/CMake/releases/download/v3.19.6/cmake-3.19.6-Linux-x86_64.sh
$ sh cmake-linux.sh -- --skip-license --prefix=$MY_INSTALL_DIR
$ rm cmake-linux.sh

Then check the version of cmake,

$ cmake --version
cmake version 3.19.6

Install the basic tools required to build gRPC,

$ brew install autoconf automake libtool pkg-config

Then, clone the gRPC repo,

$ git clone --recurse-submodules -b v1.35.0 https://github.com/grpc/grpc

While not mandatory, gRPC applications usually IDL proto3 for service definitions and data serialization,

$ cd grpc
$ mkdir -p cmake/build
$ pushd cmake/build
$ cmake -DgRPC_INSTALL=ON \
      -DgRPC_BUILD_TESTS=OFF \
      -DCMAKE_INSTALL_PREFIX=$MY_INSTALL_DIR \
      ../..
$ make install -j4  // 4 means that you should have a 4-core system
$ popd

2. Helloworld Example

First, let’s check the helloworld example. The example code is part of the grpc repo source, which you cloned as part of the steps of the previous section. First, let’s change to the example’s directory,

$ cd examples/cpp/helloworld

Then build the example using cmake,

$ mkdir -p cmake/build
$ pushd cmake/build
$ cmake -DCMAKE_PREFIX_PATH=$MY_INSTALL_DIR ../..
$ make -j4

After we meet,

[100%] Built target greeter_client

Then it means that we can test our code. From the first terminal, we can use

$ ./greeter_server
Server listening on 0.0.0.0:50051

In the second terminal, run

$ ./greeter_client

And we will receive the following message,

Greeter received: Hello world

Congratulations! You’ve just run a client-server application with gRPC.

3. Protocol Buffer

In the last section, we have discussed that we can use the XDR files to construct the client stub and the server stub. For the gRPC, we will use protobuf by default. This is a serializing open-source compiler developed by Google. We can install the protobuf by,

$ brew install protobuf

4. Protocol Buffer Version

The first step when working with protocol buffers is to define the structure for the data you want to serialize in a proto file, which is an ordinary text file with a .proto extension. In the first line of a .proto file, we must explicitly point out that we will use proto3 because by default, we will be using proto2.

syntax = "proto3";

5. Proto Message

Protocol buffer data is structured as messages, where each message is a small logical record of information containing a series of name-value pairs called fields. Here’s a simple example in the helloworld example,

// The request message containing the user's name.
message HelloRequest {
    string name = 1;
}

We can try to compile the following simple.proto code by the compiler protoc,

By,

$ mkdir -p build
$ protoc simple.proto --cpp_out=./build
$ ls ./build
simple.pb.cc    simple.pb.h

6. Field and Field Number

In the previous example, we have found out that in a specific message, we must have name-value pairs called fields. And these fields can be used to transfer the data. But why do we must include a field number after each field? Before we answer this question, let’s try to compile the following code,

If we compile this code, we will find the following error,

simple.proto:6:16: Field number 1 has already been used in "HelloRequest" by field "name".

This means that the field number must be unique for each field in each message. However, in two different messages, we can use the same field number. For example, in the helloworld example, field name and field message in two different messages can have the same field number.

But why should we include this field number? Let’s see an explanation from the StackOverflow. It says that the field numbers are used to match fields when serializing and deserializing the data. So if we don’t include this value, there will be ambiguity from both sides. For example, if we want to transfer both the string name and an int32 typed code, the client and the server will not know how to interpret the message data because we can have two different cases,

If we attach a field number for each field, then it will be easier because now both the server and the client know how to serialize and deserialize the data.

7. Proto Services

In the .proto file, we should also define our gRPC services. For example, in the helloworld example, both the server and the client stub have a SayHello() RPC method that takes a HelloRequest parameter from the client and returns a HelloResponse from the server, and that this method is defined like this,

service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}

You can add another service simply and now let’s have a try. Suppose after the greeter, we would like to send a password to the server and the server will send back the int32 type private key. So we can update the new .proto file as,

Before we can use the new service method, we need to recompile the updated proto file. From the directory …/examples/cpp/helloworld/cmake/build , we should run,

$ make -j4

This regenerates helloworld.pb.{h,cc} and helloworld.grpc.pb.{h,cc}, which contains the generated client and server classes, as well as classes for populating, serializing, and retrieving our request and response types.

Then, we still need to implement and call the new method in the human-written parts of our example application.

Finally, from the directory …/examples/cpp/helloworld/cmake/build , we should run,

$ make -j4

Then, from one terminal,

$ ./greeter_server
Server listening on 0.0.0.0:50051

From another terminal,

$ ./greeter_client
Greeter received: Hello world
Private key: 102513