RouteGuide Microservice on Kafka

This guide will walk through each unique gRPC message request and response design and how Zilla is configured to manage the connection for each.

Let's take a look at how Zilla would be configured with a full featured gRPC service. For this, we will use the route_guide.proto to define the gRPC service and the method request-response types. You can find examples of running this service in any language on the gRPC docs as well as example clients like this one implemented in java.

Step 1: Declaring the service

Zilla lets you use your service proto file(s) to specify the service definitions. This will let you create one or many grpc-kafka proxy handlers to be used by the whole service or individual methods.

Here is the service we will be enhancing with Zilla and Kafka.

service RouteGuide {

  // simple RPC
  rpc GetFeature(Point) returns (Feature) {}

  // server-side streaming RPC
  rpc ListFeatures(Rectangle) returns (stream Feature) {}

  // client-side streaming RPC
  rpc RecordRoute(stream Point) returns (RouteSummary) {}

  // bidirectional streaming RPC
  rpc RouteChat(stream RouteNote) returns (stream RouteNote) {}
}

Here is the simplest configuration for declaring the gRPC service. There is a server defining the service and methods which exits to the client which can be routed to the running gRPC instance.

grpc_server:
  type: grpc
  kind: server
  catalog:
    host_filesystem:
      - subject: route_guide
  routes:
    - when:
        - method: routeguide.RouteGuide/*
      exit: grpc_client
grpc_client:
  type: grpc
  kind: client

Define individual route handlers for each method or specify the * wildcard to handle all methods the same.

All messages handled by the same route

routes:
  - when:
      - method: routeguide.RouteGuide/*

Routes for individual service methods

routes:
  - when:
      - method: routeguide.RouteGuide/GetFeature

Step 2: Handling message routing onto Kafka

This maps the proto service method's request and response messages directly to Kafka topics defined by the topic and reply-to attributes respectively. The messages are linked by the zilla:correlation-id header for individual calls into the gRPC service. Read more about it and how the idempotency-key enables safe message replays in the grpc-kafka reference section.

Message routing for RPC request and response types

Let's look at some common service definitions and how Zilla can route their messages.

Simple RPC

For the GetFeature method, the Point request and Feature response create the stream of information Kafka can help manage.

routes:
  - when:
      - method: routeguide.RouteGuide/GetFeature
    with:
      capability: produce
      topic: request-topic
      reply-to: response-topic

Server-side streaming RPC

For the ListFeatures method, we can have a dedicated stream of Feature messages being recorded on a topic before being sent to the client. Kafka allows the server to produce all of the responses without considering the connection to the client.

routes:
  - when:
      - method: routeguide.RouteGuide/ListFeatures
    with:
      capability: produce
      topic: request-topic
      reply-to: feature-stream-topic

Client-side streaming RPC

For the RecordRoute method, the client sends a stream of Point messages to a dedicated stream topic before being sent to the server. Kafka allows the client to produce all of the requests without considering the connection to the server.

routes:
  - when:
      - method: routeguide.RouteGuide/RecordRoute
    with:
      capability: produce
      topic: point-stream-topic
      reply-to: response-topic

Bidirectional streaming RPC

For the RouteChat method, we can have all of the messages on the same topic since it is the history that both the client and the server will need.

routes:
  - when:
      - method: routeguide.RouteGuide/RouteChat
    with:
      capability: produce
      topic: chat-topic

Fanout streaming RPC

An additional method for getting messages from a service onto Kafka is using the grpc-kafka fetch capability. This enables message filtering and reliable message delivery. Read more about the fetch capability in the reference section.

The service will need a method that accepts the google/protobuf/empty.proto and produce the massage to be fanned out onto the Kafka topic.

import "google/protobuf/empty.proto";

service FanoutService
{
  rpc FanoutServerStream(google.protobuf.Empty) returns (stream FanoutMessage);
}

Here we set the fetch capability and a filter.

grpc_kafka:
  type: grpc-kafka
  kind: proxy
  routes:
    - when:
        - method: example.FanoutService/*
      with:
        capability: fetch
        topic: messages
        filters:
          key: custom-key
          headers:
            custom-text: custom-value

Step 3: Calling services from Kafka

Now that messages are in Kafka we need to send them to the gRPC services responsible for processing them. For this, we will be using the kafka-grpc binding.

remote_server:
  type: kafka-grpc
  kind: remote_server
  entry: <kafka_cache_client_name>
  options:
    acks: leader_only
  routes:
    - when:
        - topic: request-topic
          reply-to: response-topic
      exit: grpc_client
      with:
        scheme: http
        authority: ${{env.ENTRYPOINT_HOST}}:${{env.ENTRYPOINT_PORT}}
grpc_client:
  type: grpc
  kind: client
  options:
    services:
      - route_guide.proto
  routes:
    - when:
        - method: routeguide.RouteGuide/GetFeature
      exit: <http_client_name_for_grpc_host>

Other important elements

Other aspects of routing traffic through Zilla, Kafka, and gRPC services can be summed up with reusable config setups.

Connecting a gRPC service to Zilla

When a Zilla config refers to a grpc client the traffic will need to be routed through an http and tcp client down to the <grpc_host> <grpc_port>.

grpc_client:
  type: grpc
  kind: client
  exit: http_client
http_client:
  type: http
  kind: client
  options:
    versions:
      - h2
  exit: tcp_client
tcp_client:
  type: tcp
  kind: client
  options:
    host: <grpc_host>
    port: <grpc_port>

Proxy service entrypoint

Using a tcp server binding we can route the gRPC traffic through an http server, ${{env.ENTRYPOINT_HOST}}:${{env.ENTRYPOINT_PORT}}, to our desired gRPC server at the <grpc_server_binding_name>. The tls server binding secures the entrypoint by defining a tls certificate from the filesystem vault, which is used to send the traffic over https. Alternatively, if tls is not needed The tcp server can exit directly to the http server and the tls binding and filesystem vault can be removed.

  tcp_server:
    type: tcp
    kind: server
    options:
      host: 0.0.0.0
      port: ${{env.ENTRYPOINT_PORT}}
    exit: tls_server
  tls_server:
    type: tls
    kind: server
    vault: server
    options:
      keys:
        - ${{env.ENTRYPOINT_HOST}}
      sni:
        - ${{env.ENTRYPOINT_HOST}}
      alpn:
        - h2
    exit: http_server
  http_server:
    type: http
    kind: server
    options:
      versions:
        - h2
      access-control:
        policy: cross-origin
    routes:
      - when:
          - headers:
              :scheme: https
              :authority: ${{env.ENTRYPOINT_HOST}}:${{env.ENTRYPOINT_PORT}}
        exit: <grpc_server_binding_name>

Connecting to Kafka

A gRPC server can connect to Kafka the same as any other binding in Zilla. See the docs on the different connection options.

  kafka_cache_client:
    type: kafka
    kind: cache_client
    exit: kafka_cache_server
  kafka_cache_server:
    type: kafka
    kind: cache_server
    exit: kafka_client
  kafka_client:
    type: kafka
    kind: client
    options:
      servers:
        - ${{env.KAFKA_BOOTSTRAP_SERVER}}
    exit: tcp_client
  tcp_client:
    type: tcp
    kind: client

Try it out

Go check out the grpc.kafka.proxy example for a full implementation of an EchoService.