This article will show you how to build docker images that contain a single akka cluster application. You will be able to run multiple seed nodes and multiple cluster nodes. The code can be found on Github and will be available as a Typesafe Activator.
If you don’t know docker or akka
Docker is the new shiny star in the devops world. It lets you easily deploy images to any OS running docker, while providing an isolated environment for the applications running inside the container image.
Akka is a framework to build concurrent, resilient, distributed and scalable software systems. The cluster feature lets you distribute your Actors across multiple machines to achieve load balancing, fail-over and the ability to scale up and out.
The big picture
This is what the running application will look like. No matter where your docker containers will run at the end of the day. The numbers at the top left describe the starting order of the containers.
First you have to start your seed nodes, which will „glue“ the cluster together. After the first node is started all following seed-nodes have to know the ip address of the initial seed node in order to build up a single cluster. The approach describe in this article is very simple, but easily configurable so you can use it with other provision technologies like chef, puppet or zookeeper.
All following nodes that get started need at least one seed-node-ip in order to join the cluster.
The application configuration
We will deploy a small akka application which only logs cluster events. The entrypoint is fairly simple:
object Main extends App { val nodeConfig = NodeConfig parse args // If a config could be parsed - start the system nodeConfig map { c => val system = ActorSystem(c.clusterName, c.config) // Register a monitor actor for demo purposes system.actorOf(Props[MonitorActor], "cluster-monitor") system.log info s"ActorSystem ${system.name} started successfully" } } |
The tricky part is the configuration. First the akka.remote.netty.tcp.hostname configuration needs to be set to the docker ip address. The port configuration is unimportant as we have unique ip address thanks to docker. You can read more about docker networking here. Second the seed nodes should add themselves to the akka.cluster.seed-nodes list. And at last everything should be configurable through system properties and environment variables. Thanks to the Typesafe Config Library this is achievable (even with some sweat and tears).
- Generate a small commandline parser with scopt and the following two parameters:
–seed flag which determines if this node starting should act as a seed node
([ip]:[port])… unbounded list of [ip]:[port] which represent the seed nodes - Split the configuration in three files
- application.conf which contains the common configuration
- node.cluster.conf contains only the node specific configuration
- node.seed.conf contains only the seed-node specific configuration
- A class NodeConfig which orchestrates all settings and cli parameters in the right order and builds a Typesafe Config object.
Take a closer look at the NodeConfig class. The core part is this
// seed nodes as generated string from cli (ConfigFactory parseString seedNodesString) // the hostname .withValue("clustering.ip", ipValue) // node.cluster.conf or node.seed.conf .withFallback(ConfigFactory parseResources configPath) // default ConfigFactory.load but unresolved .withFallback(config) // try to resolve all placeholders (clustering.ip and clustering.port) .resolve |
The part to resolve the IP address is a bit hacky, but should work in default docker environments. First the eth0 interfaces is searched and then the first isSiteLocalAddress is being returned. IP adresses in the following ranges are local: 172.16.xxx.xxx, 172.31.xxx.xxx , 192.168.xxx.xxx, 10.xxx.xxx.xxx.
The main cluster configuration is done inside the clustering section of the application.conf
clustering {
# ip = "127.0.0.1" # will be set from the outside or automatically
port = 2551
cluster.name = "application"
} |
The ip adress will be filled by the algorithm describe above if nothing else is set. You can easily override all settings with system properties.
E.g if you want to run a seed node and a cluster node inside your IDE without docker start both like this:
# the seed node -Dclustering.port=2551 -Dclustering.ip=127.0.0.1 --seed # the cluster node -Dclustering.port=2552 -Dclustering.ip=127.0.0.1 127.0.0.1:2551 |
For sbt this looks like this
# the seed node sbt runSeed # the cluster node sbt runNode |
The build
Next we build our docker image. The sbt-native-packager plugin recently added experimental docker support, so we only need to configure our build to be docker-ready. First add the plugin to your plugins.sbt.
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "0.7.4") |
Now we add a few required settings to our build.sbt. You should use sbt 0.13.5 or higher.
// adds start script and jar mappings packageArchetype.java_application // the docker maintainer. You could scope this to "in Docker" maintainer := "Nepomuk Seiler" // Short package description packageSummary := s"Akka ${version.value} Server" |
And now we are set. Start sbt and run docker:publishLocal and a docker image will be created for you. The Dockerfile is in target/docker if you want to take a closer look what’s created.
Running the cluster
Now it’s time to run our containers. The image name is by default name:version. For the our activator it’s akka-docker:2.3.4. The seed ip adresses may vary. You can read it out of the console output of your seed nodes.
docker run -i -t -p 2551:2551 akka-docker:2.3.4 --seed docker run -i -t -p 2551:2551 akka-docker:2.3.4 --seed 176.16.0.18:2551 docker run -i -t -p 2551:2551 akka-docker:2.3.4 176.16.0.18:2551 176.16.0.19:2551 docker run -i -t -p 2551:2551 akka-docker:2.3.4 176.16.0.18:2551 176.16.0.19:2551 |
What about linking?
This blog entry describes a different approach to build an akka cluster with docker. I used some of the ideas, but the basic concept is build ontop of linking the docker contains. This allows you to get the ip and port information of the running seed nodes. While this is approach is suitable for single host machines, it seems to get more messy when working with multiple docker machines.
The setup in this blog requires only one thing: A central way of assigning host ips. If your seed nodes don’t change their IP adresses you can basically configure almost everything already in your application.conf.
