Staying warm with docker-lambda

Sometime in the last few days, docker pulls of lambci/lambda hit 35 million.

Hockey anyone? 🏒

Which is more than twice what it was six months ago:

I don’t know where serverless is on the hype cycle these days, but if local testing and building of Lambda services is anything to go by, doubling every six months ain’t bad 📈

Strict growth

I have no doubt docker-lambda’s growth is fueled by Amazon’s decision to use it as the base of their AWS SAM CLI tool for local testing — as well as tools like Serverless Framework, localstack and many others.

Part of its appeal I think lies in the fact that it reproduces the Lambda environment incredibly strictly. This means that developers spend less time needing to deploy their software just to see how it will run—even if they have unusual requirements that go beyond simple hello-worlds.

But there is some friction with this approach. In my stringency, I’ve tried to add as little extra software as possible to the images, so developers don’t rely on things that just don’t exist in production. However, that means the images have been stuck in a “run-once” model: you run the docker container, it executes the runtime, including your Lambda handler, and exits. Every execution is a cold start, as there’s been no coordinating service to allow for multiple invocations.

…Until now

Well screw that stringency, I’m excited to announce that docker-lambda now supports warm invocations for all runtimes 🎉

You can run each runtime in a “stay-open” mode, and it will start an API server that listens for events in exactly the same way the production Lambda API does. You can use the AWS CLI to invoke them, or you can just use curl or any other http client (there’s no authentication). The first invocation will still be a “cold start”, but all subsequent invocations will be warm.

It still keeps the concurrency model that Lambda has — each container won’t handle more than one request at once — but testing on my MacBook Pro yields around 130 req/s for a fast handler. This should be fast enough for local testing, and certainly an order of magnitude faster than executing the docker container plus a cold start each time.

Internally it uses the same mechanism each runtime does in production, even the legacy ones. Each has a loop and a “receive-invoke” function that waits for incoming events, and I hook into this. The only difference is I’ve added a mock server (written in Go) responsible for dispatching the events, instead of the native socket methods used in production. I needed to add this as an extra piece of software on the older runtimes (at /var/runtime/mockserver), but aside from that, I kept the intrusion pretty minimal.

Along the way, I refactored a lot (especially the dotnetcore runtimes), and the behavior of the runtimes should be more standardized across each language. It’s become a veritable Pantone wheel of a repo.

Features, features

This feature follows a line of smaller ones I’ve been rolling out in my spare time over the past few weeks.

• Support for the new runtimes from day one: nodejs12.x, python3.8 and java11
• The images now support Docker Content Trust, so you can have some assurance about the provenance of the software you’re running
• I launched a related project, yumda — which makes it a lot easier to install system dependencies for your Lambdas, and is especially relevant for the newer runtimes running on Amazon Linux 2 (nodejs10.x, nodejs12.x, python3.8 and java11). I chatted with Jeremy Daly about this and other Lambda deep-dives on his Serverless Chats podcast here and here.

The future

The obvious use case for keeping runtimes warm is local API Gateway testing. I’m hoping support in AWS SAM CLI and Serverless Framework will be added for this soon (or maybe it’s the excuse I need to finally write a docker-lambda CLI).

Please test away and file issues if you run into any behavior that differs from production.

If anyone has suggestions around how the build images or yumda can reduce their development friction even further, I’m all ears!

Here’s to 70 million, see you in six months 🍸😸