Building gRPC Services with Clojure and Leiningen

gRPC is an RPC framework open-sourced fairly recently, and initially developed by Google. There’s a bunch of support for several languages, and since I’ve been experimenting with learning a few, I’ve decided to write a service in Clojure, which I’ll call from a C++ client. gRPC doesn’t directly support Clojure, but we can manage because of Clojure’s great Java interop.

The first hurdle I encountered though was getting my service definition compiled into Java stubs. There are two official ways¹ to integrate the interface definition compilation into the build in Java: via a Gradle plugin, and via a Maven plugin. However, most Clojure projects either build via Leiningen or Boot; I’m more familiar with Leiningen, so I went with using it for the service. I could have used Boot though, seeing as it’s a little more flexible, but I’d rather not make things overly difficult for now.

So, first things first: generate an empty Leiningen project:

$ lein new app clj-grpc

This creates an empty project with the app template, which is appropriate for us here.

We’ll need to set up our project.clj to generate Java code from our service IDL. I initially thought of using a Leiningen plugin I had used before for integrating protoc into the build called lein-protobuf, but I soon found out that although protoc did run and did generate the Java source files, it didn’t parse the service definition, and only generated classes for the messages I defined. Thankfully, someone else did a similar plugin called lein-protoc that does support generating the necessary gRPC stubs as well.

So, integrating lein-protoc and adding the basic dependencies we’ll need for gRPC gives us the following project definition:

(defproject clj-grpc "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license
  {:name "Eclipse Public License"
   :url "http://www.eclipse.org/legal/epl-v10.html"}
  :main ^:skip-aot clj-grpc.core
  :target-path "target/%s"
  :profiles {:uberjar {:aot :all}}

  :plugins [[lein-protoc "0.4.2"]]

  :protoc-version "3.6.0"
  :protoc-grpc {:version "1.13.1"}
  :proto-target-path "target/generated-sources/protobuf"

  :java-source-paths
  ["target/generated-sources/protobuf"]

  :dependencies
  [[org.clojure/clojure "1.9.0"]
   [com.google.protobuf/protobuf-java "3.6.0"]
   [javax.annotation/javax.annotation-api "1.2"]
   [io.netty/netty-codec-http2 "4.1.25.Final"]
   [io.grpc/grpc-core "1.13.1"]
   [io.grpc/grpc-netty "1.13.1"
    :exclusions [io.grpc/grpc-core
                 io.netty/netty-codec-http2]]
   [io.grpc/grpc-protobuf "1.13.1"]
   [io.grpc/grpc-stub "1.13.1"]])

Note that the current version of the app template points to Clojure 1.8; I’ve decided to bump to the latest version, which provides support for Spec.

Aside from the dependencies on the protobuf and grpc-java libraries, I’ve also had to add the JSR-250 API separately as a workaround, as I’ve decided to run this on Java 9 – the generated Java class files use the @Generated annotation, which doesn’t compile under Java 9 because of Java 9’s modules, and because one of the gRPC dependencies is com.google.code.findbugs/jsr305, which clashes with the built-in JDK javax.annotation package due to it adding classes to the same javax.annotation package as the java.xml.ws.annotation module in the JDK.

Note that we also need to change the target path, as the default uses generated-sources/protobuf inside target-path, which we don’t want as our target path contains a specifier for what profiles are active (and there doesn’t seem to be a way to specify profiles in java-source-path).

Let’s also go with the service defined in the quick start; copy that over to src/proto.

Now let’s tell Leiningen to compile:

$ lein javac

which should yield class files in target/default/classes.

Now, let’s implement the Greeter service in Clojure; we’ll do it in the clj-grpc.service namespace for now:

(ns clj-grpc.service
  (:gen-class
   :name clj-grpc.service.GreeterServiceImpl
   :extends
   io.grpc.examples.helloworld.GreeterGrpc$GreeterImplBase)
  (:import
   [io.grpc.stub StreamObserver]
   [io.grpc.examples.helloworld
    HelloReply]))


(defn -sayHello [this req res]
  (let [name (.getName req)]
    (doto res
      (.onNext (-> (HelloReply/newBuilder)
                   (.setMessage (str "Hello, " name))
                   (.build)))
      (.onCompleted))))

and we’ll have to provide a suitable -main implementation, in clj-grpc.core:

(ns clj-grpc.core
  (:gen-class)
  (:require [clj-grpc.service])
  (:import
   [io.grpc
    Server
    ServerBuilder]
   [io.grpc.stub StreamObserver]
   [clj-grpc.service GreeterServiceImpl]))

(def SERVER_PORT 50051)

(defn start []
  (let [greeter-service (new GreeterServiceImpl)
        server (-> (ServerBuilder/forPort SERVER_PORT)
                   (.addService greeter-service)
                   (.build)
                   (.start))]
    (-> (Runtime/getRuntime)
        (.addShutdownHook
         (Thread. (fn []
                    (if (not (nil? server))
                      (.shutdown server))))))
    (if (not (nil? server))
      (.awaitTermination server))))

(defn -main
  [& args]
  (print "Now listening on port " SERVER_PORT)
  (start))

Put it all together into an uberjar, so we can run it:

$ lein uberjar

And then run the uberjar:

$ java -jar target/uberjar/clj-grpc-0.1.0-SNAPSHOT-standalone.jar

We now have a gRPC service implemented in Clojure, more or less. We can test it out by using the Java client, or any other client.

Previously: Polyglot