Going Serverless with Clojure and Terraform

code clojure devops

John Jacobsen
Wednesday, December 6, 2017

Home Other Posts

If you write some code that does a useful task, how do you make it available to others as cheaply as possible? Traditional cloud deployments involve managing virtual servers on cloud providers such as AWS. So-called “serverless” deployments, where you simply upload code for a single function into existing infrastructure, are an interesting recent development which have the potential to steamline operational workflows and lower costs.

This post discusses deploying serverless Clojure APIs using AWS Lambda and API Gateway. Lambda allows you to upload code for single functions (“lambdas”), rather than entire applications, into infrastructure that AWS manages transparently for you.

Lambdas are cheap, at least for code that is only occasionally used, when compared to managing your own virtual machine. Usage of your function can scale up or down automatically as needed – you only pay for the load you use. Having operated my own servers for many years now, I find not having to manage an entire VM for any given project especially appealing.

AWS Lambda supports Node.js, Python, and Java (and, by extension, JVM languages). Since I prefer to use Clojure for backend services, I wanted to verify for myself whether Clojure is a good fit for AWS Lambda (and vice versa). After some experimentation, I have become convinced that Clojure and Lambda can work well together, provided you avoid the pitfalls I describe here. Clojure lambdas, kept “warm,” execute quickly and cheaply.

Lambdas need a front end to be usable by systems outside of AWS, and this can be provided by API Gateway, which can provide highly configurable web interfaces to lambdas or serve as proxies to more traditional web servers. Also available via API Gateway but not covered here are automatic DDoS protection, rate limiting features, and a variety of authentication and authorization mechanisms.

You will probably also want to configure your lambdas to log their output somewhere, and CloudWatch can be put to use for this purpose.

Example Projects

I have two example repositories on GitHub: the first is a minimal example which does nothing useful other than wire all the required bits together, and the other is a small project for rolling up characters for a classic, 1980’s sci fi role playing game called Traveller, which I ported into AWS just for fun.

Ingredients

Here are the ingredients used:

Clojure

I have used Clojure 1.8 with no issues; because Clojure upgrades tend to be painless, I expect 1.9 to work fine as well. Helpful libraries:

  • The lambada library provides a small but convenient wrapper around the ceremony needed for a function to be called by the Java Lambda implementation.
  • If you’re deploying to AWS, chances are you’re using other AWS services as well. amazonica provides a very convenient wrapper around the massive AWS Java SDK. I use it daily and it works well for me most of the time. However: you will need to keep an eye on jar size, as described in the Pitfalls section, below.

Terraform

Terraform is a program and an accompanying declarative configuration language which allows one to specify all the various resources you need for a complex cloud deployment, and create or destroy them as needed. Terraform keeps track of your current deployment and, when you make changes to the config files, changes only those resources that need to be updated.

Much of the effort required for this work was in learning enough Terraform to get the various parts wired up together correctly.

Make

Since Terraform doesn’t compile my Clojure code, I find it convenient to use make to coordinate all my build steps.

AWS

  • AWS Lambda hosts function code and executes it on demand;
  • API Gateway can make your Lambda function accessible to the outside world;
  • CloudWatch can collect logs (basicaly, anything written to *out*) and also execution metrics for your Lambda code.
  • Not used by my example repos, but typically required for most useful projects, will be other pieces of the vast AWS ecosystem; these are typically well-supported by Terraform, except for absolutely bleeding-edge AWS products. (The pace at which AWS is rolling out new services and functionality is truly astonishing.)

Working code

We’ll talk through the minimal example in a little detail. The Clojure portion is quite simple: a basic project.clj, and the actual code per se (core.clj):

(ns example.core
  (:require [cheshire.core :as json]
            [clojure.java.io :refer [reader writer]]
            [uswitch.lambada.core :refer [deflambdafn]]))


(defn handle-event [event]                                  ;; 1
  (println "Your lambda is ready, madame.")                 ;; 2
  {"isBase64Encoded" false                                  ;; 3
   "statusCode" 200
   "headers" {}
   "body" (json/generate-string {:status "OK"               ;; 4
                                 :extra-stuff "I am lambda, hear me roar"})})


(deflambdafn example.core.LambdaFunction [is os ctx]         ;; 5
  (let [event (json/parse-stream (reader is))
        res (handle-event event)]
    (with-open [w (writer os)]
      (json/generate-stream res w))))
  1. I like to make an event handler that I can test independently from all the Lambda ceremony in the next fuction.
  2. println and writes to *out* will go to CloudWatch, via some AWS / Terraform magic we’ll touch upon in a moment.
  3. We’re using the API proxy integration, wherein the lambda returns all the fields API Gateway needs to complete the HTTP response.
  4. Here is where we get to actually return the useful part of our “API”, namely in JSON format. The default Content-Type is application/json, though that can be set in the headers value.
  5. The needed ceremony for the lambda invocation. deflambdafn is Lambada’s macro for defining the lambda function (though this library consists of nothing but this macro, and it hasn’t changed in over a year and a half, I still find it useful). is and os are, respectively, a Java InputStream and OutputStream, and reflect the inputs and outputs to the function. Cheshire (the JSON parsing library) conveniently handles the Java input and output streams. ctx is an AWS-defined Context object which provides (as of this writing) 11 pieces of metadata about the execution of your lambda, none of which I’ve needed to date.

The Terraform is more involved. Since there is a lot of code here, I will link to the various bits directly on GitHub and provide commentary here. First of all, in my examples all the terraform is in one file, for clarity; for any larger projects one will typically want to break it up into smaller files (modules).

  1. Variable definitions. These can be set by environment variables by appending TF_VAR_ to the variable name in your .bash_profile or equivalent.
  2. Jar file in S3. We’re uploading the Jar file to S3, and then telling Lambda where to find the code when the function is created.

Pitfalls, and how to avoid them

Large jar files

As of this writing, lambdas have a maximum jar (or zip) file size of 50 MB. When using amazonica (which itself uses the massive Java AWS SDK), you will exceed this unless you do something like the following in your project.clj or build.boot:

:dependencies [[amazonica "0.3.121" :exclusions [com.amazonaws/aws-java-sdk]]
               ;; Use only the packages you need:
               [com.amazonaws/aws-java-sdk-s3 "1.11.287"]
               [com.amazonaws/aws-java-sdk-dynamodb "1.11.287"]
               [com.amazonaws/aws-java-sdk-cloudwatch "1.11.287"]
               [com.amazonaws/aws-java-sdk-sqs "1.11.287"]
               ;; ...
              ]

The same approach will be needed for any other large dependencies. If your jar file is really large after pruning unneeded dependencies, then you don’t have a good fit for AWS Lambda.

Slow start times

Java (and especially Clojure) lambdas start slowly (multiple seconds) the first time they are launched or if they have been idle awhile. A simple solution to this is to keep the lambda “warm”, i.e. to trigger the lambda every, say, five minutes via a CloudWatch event. Your lambda should have a ’no-op’ option which does nothing other than return. The lambda will then continue to run quickly (well under half a second for lambdas doing things that are not themselves time-intensive).

Here is the sample Terraform from the minimal project:

# Keep Lambda "warm" to make startup much faster:
resource "aws_cloudwatch_event_rule" "example_5min_rule" {
    name = "example-every-five-minutes"
    description = "Fires Example lambda every five minutes"
    schedule_expression = "rate(5 minutes)"
}


resource "aws_lambda_permission" "cw_event_call_example_lambda" {
    statement_id = "AllowExecutionFromCloudWatch"
    action = "lambda:InvokeFunction"
    function_name = "${aws_lambda_function.example_lambda.function_name}"
    principal = "events.amazonaws.com"
    source_arn = "${aws_cloudwatch_event_rule.example_5min_rule.arn}"
}


resource "aws_cloudwatch_event_target" "target5min" {
    rule = "${aws_cloudwatch_event_rule.example_5min_rule.name}"
    target_id = "example_lambda"
    arn = "${aws_lambda_function.example_lambda.arn}"
}

Missing logs

Need to create an extra policy for the logs and attach them to the role.

Local testing

Cost overruns

Results

Speed

The API responds in a few hundred milliseconds.

$ export HOSTNAME=cbn1afevhh.execute-api.us-east-1.amazonaws.com
$ export URL="https://${HOSTNAME}/production?lang=english&n=1"
$ time curl -s $URL > /dev/null
real	0m0.384s
user	0m0.046s
sys	0m0.015s

CloudWatch tells me how long the lambda proper takes on average, over time (this is another nice feature of having the keepalive event).

lambda-execution-duration.png

The periodicity in this graph is interesting, and presumably someone at AWS can explain it to me, but the main point is that the average is below 100 msec.

Cost

With the keepalive, and after the AWS free tier is past, the Lambda Cost Calculator tells me my lambda cost will be US$0.03/month (so far it has been zero). If I have the math right, API Gateway should be about the same cost, so, six US cents per month. By contrast, my Digital Ocean droplet (which I love, BTW) is US$5/month.

Unavoidable(?) Drawbacks

It seems that breaking the traditional server model of deploying code also breaks down some of the barriers between development and operations: you can only test so much at the REPL; quite soon you have to try it in AWS itself. The pieces get smaller, so there is more gluing to do. As I’ve used more and more AWS features, my work life has become progressively more “DevOps-y.” It feels powerful to be able to deploy and operate my own infrastructure, but there is definitely a cost in time and effort to navigate the substantial AWS learning curve.

Then there is the Amazon vendor lock-in… but perhaps that’s a subject for another day.

See also

Portkey (under development) is a lower-level approach that allows one to send code into Lambda directly from the REPL. It looks like it still has rough edges, but I’ve been watching the project for some time.

Conclusion

There is some Terraform work (and very little Clojure code) involved with exposing Clojure functions via an API. Once the all the ceremony is figured out, it’s easy enough to roll out new microservices for very little cost.

Blog Posts (164)

Select from below, view all posts, or choose only posts for:art clojure code emacs misc orgmode physics python ruby sketchup southpole


Home


Painting and Time art

Learning Muscular Anatomy code clojure art emacs orgmode

Reflections on a Year of Daily Memory Drawings art

Repainting art

Daily Memory Drawings art

Questions to Ask art

Macro-writing Macros code clojure

Time Limits art

Lazy Physics code clojure physics

Fun with Instaparse code clojure

Nucleotide Repetition Lengths code clojure

Updating the Genome Decoder code clojure

Getting Our Hands Dirty (with the Human Genome) code clojure

Validating the Genome Decoder code clojure

A Two Bit Decoder code clojure

Exploratory Genomics with Clojure code clojure

Rosalind Problems in Clojure code clojure

Introduction to Context Managers in Python code python

Processes vs. Threads for Integration Testing code python

Resources for Learning Clojure code clojure

Continuous Testing in Python, Clojure and Blub code clojure python

Programming Languages code clojure python

Milvans and Container Malls southpole

Oxygen southpole

Ghost southpole

Turkey, Stuffing, Eclipse southpole

Wind Storm and Moon Dust southpole

Shower Instructions southpole

Fresh Air and Bananas southpole

Traveller and the Human Chain southpole

Reveille southpole

Drifts southpole

Bon Voyage southpole

A Nicer Guy? southpole

The Quiet Earth southpole

Ten southpole

The Wheel art

Plein Air art

ISO50 southpole art

SketchUp and MakeHuman sketchup art

In Defense of Hobbies misc code art

Closure southpole

Takeoff southpole

Mummification southpole

Eleventh Hour southpole

Diamond southpole

Baby, It's Cold Outside southpole

Fruition southpole

Friendly Radiation southpole

A Place That Wants You Dead southpole

Marathon southpole

Deep Fried Macaroni and Cheese Balls southpole

Retrograde southpole

Three southpole

Transitions southpole

The Future southpole

Sunday southpole

Radio Waves southpole

Settling In southpole

Revolution Number Nine southpole

Red Eye to McMurdo southpole

That's the Way southpole

Faults in Ice and Rock southpole

Bardo southpole

Chasing the Sun southpole

Downhole southpole

Coming Out of Hibernation southpole

Managing the Most Remote Data Center in the World code southpole

Ruby Plugins for Sketchup art sketchup ruby code

The Cruel Stranger misc

Photoshop on a Dime art

Man on Wire misc

Videos southpole

Posing Rigs art

Metric art

Cuba southpole

Wickets southpole

Safe southpole

Broken Glasses southpole

End of the Second Act southpole

Pigs and Fish southpole

Last Arrivals southpole

Lily White southpole

In a Dry and Waterless Place southpole

Immortality southpole

Routine southpole

Tourists southpole

Passing Notes southpole

Translation southpole

RNZAF southpole

The Usual Delays southpole

CHC southpole

Wyeth on Another Planet art

Detox southpole

Packing southpole

Nails southpole

Gearing Up southpole

Gouache, and a new system for conquering the world art

Fall 2008 HPAC Studies art

YABP (Yet Another Blog Platform) southpole

A Bath southpole

Green Marathon southpole

Sprung southpole

Outta Here southpole

Lame Duck DAQer southpole

Eclipse Town southpole

One More Week southpole

IceCube Laboratory Video Tour; Midrats Finale southpole

SPIFF, Party, Shift Change southpole

Good things come in threes, or 18s southpole

Sleepless in the Station southpole

Post Deploy southpole

Midrats southpole

IceCube and The Beatles southpole

Video: Flight to South Pole southpole

The Pure Land southpole

Almost There southpole

There are no mice in the Hotel California Bunkroom southpole

Short Timer southpole

Sleepy in MacTown southpole

Superposition of Luggage States southpole

Sir Ed southpole

Shortcut to Toast southpole

Pynchon, Redux southpole

Flights: Round 1 southpole

Packing for the Pole southpole

Goals for Trip southpole

Balaklavas southpole

Tree and Man (Test Post) southpole

Schedule southpole

How to mail stuff to John at the South Pole southpole

Summer and Winter southpole

Homeward Bound southpole

Redeployment southpole

Short-timer southpole

The Cleanest Air in the World southpole

One more day (?) southpole

One more week (?) southpole

Closing Softly southpole

More Photos southpole

Super Bowl Wednesday southpole

Night Owls southpole

First Week southpole

More Ice Pix southpole

Settling In southpole

NPX southpole

Pole Bound southpole

Bad Dirt southpole

The Last Laugh southpole

Nope southpole

First Delay southpole

Batteries and Sheep southpole

All for McNaught southpole

The Big (Really really big...) Picture southpole

t=0 southpole

Giacometti southpole

Descent southpole

Video Tour southpole

How to subscribe to blog updates southpole

What The Blog is For southpole

Auckland southpole

Halfway Around the World; Dragging the Soul Behind southpole

Launched southpole

Getting Ready (t minus 2 days) southpole

Home

Subscribe: RSS feed ... all topics) ... or Clojure only