If you have ever worked on designing and developing a cloud based application, you would know the complexity involved in the development process. It involves a lot of steps starting from managing codebases, injecting library dependencies, configuring system properties and leading into a proper production deployment at scale.
If you are starting out to build a cloud app on your own, you would find it difficult to manage this at first. Not surprisingly though, a number of identified patterns have been used to come up with a design methodology to properly build and manage so called ‘cloud-native’ apps. Let’s take a look at what is commonly referred to as the 12 Factor App design methodology, written by Heroku co-founder Adam Wiggins.
When implemented correctly, this design methodology is expected to help
- Automate various development or deployment setups and minimize time/cost for new engineers joining the project
- Port applications across multiple execution environments by maintaining a clean contract with the underlying operating systems
- Deploy apps on to modern cloud platforms (Google cloud, Heroku, AWS etc..)
- Minimize divergence between deployment and production
- Scale up without much changes in the architecture, tools or practices.
So let’s get in to the details now. Listed below are the 12 factors which you should consider when developing or deploying apps.
First, a twelve factor app is always tracked in a version control system like Git or SVN in the form of a code repository. It’s important to note that there is a one to one relationship between a 12 factor app and the codebase repo. If there are multiple repos, then it needs to be considered as a distributed system consisting of multiple 12 factored apps. Even though all the shared code between apps would be injected in the form of dependency libraries, there would still be multiple deployments for an app. It could be in the development, staging or production setups with each running different versions of the app at any given point of time.
Second, managing app dependencies should be done in a better way, possibly by creating a manifest file, which explicitly declares all the dependent library packages. It’s also a good idea to use a dependency isolation tool (ex VirtualEnv for Python) during execution to ensure no implicit dependencies from the surrounding system. This explicit dependency specification should be applied uniformly to both production and development environments
Third factor is about the managing the configuration information for the app. It can include database resource handles as well as credentials for accessing various external services. These can be stored in a config file, however it is prudent to use environment variables for this purpose mainly because they are a language and OS agnostic standard.
Fourth factor constitutes the backing services management. Backing services are those services which an app depends on over a network connection. This can be a local database service or any other 3rd party service. For a 12 factor app, the interface to connect to these various services should be defined in a standard way. In most cases it would be accessed over a URL with credentials stored in the config. There should not be any code changes when switching between these services. For example, a 12 factor app should be able to switch between a local MySQL database and a remote one by simply changing the config.
Fifth factor is about handling the build and deployment process. Build, release and run stages should be treated separately. Use necessary automation and tools to generate build and release packages with proper tags. This is followed by running the app in the execution environment while using proper release management tools like Capistrano for ensuring timely rollback.
Sixth and seventh factors are about running apps as stateless processes and exporting services over port-binding. Under normal cases web apps are sometimes run inside a web server container (For ex. PHP/Apache, PHP apps assume that Apache webserver is already installed on the system and run on top of it). But twelve factor apps are self-contained, stateless and share-nothing processes and don’t depend on any runtime injection for creating web-facing services. The only thing they should do is to bind to a port on the underlying execution environment and the app services are exported over that port.
Eighth factor looks in to the best practices for scaling the app. It recommends process model and treating each process in the app as a first-class citizen. This simply means that each process can be managed independently (start/stop, clone to different machines etc.). Also, design a process formation for the app which details the process types (ex. web, workers) and number of processes for each type to handle different kinds of workload.
Based on the above we have multiple processes handling different jobs. So 9th factor looks in to the robustness of the app with fast startup and shutdown methods. Always try to minimize the startup time for each process. This gives more agility to the overall release process. Processes should shutdown gracefully and also robust against sudden death in the case of hardware failures. We can use some high quality robust queuing backend (Beanstalk, RabbitMQ etc.) that would help return unfinished jobs back to the queue in the case of a failure.
Tenth factor is to consider dev/prod parity. This is all about keeping development, staging and productions setups as similar as possible. Implement a continuous deployment strategy and deploy code on demand instead of sticking to a schedule. This would help catch issues more easily and at an early stage of development.
Eleventh factor is about logging mechanisms. It is critical for debugging. So it should be handled properly. Instead of the app managing the datastore for log files, it should output the log info as a continuous stream. Then some other separate services should pick up this stream and do the archiving/analysis activities.
Finally the last is to create one-off admin processes to collect data from the running application. These processes should be part of the all deploys to avoid any synchronization issues.
So that’s a quick review of the 12 factor design methodology. It would help build a cloud application system with a strong architecture at the core which is reliable and easier to manage. In some cases it may not be possible to implement all these design patterns at once. Nonetheless, it would be a good idea to start discussions around these items with development teams and solve the architectural problems one at a time.
For a detailed information on the 12 factor design, please have a look at the official spec here: https://12factor.net
Thanuj has around 7 years of experience in designing software systems ranging from small scale embedded system software to large scale cloud applications and services. Currently he is engaged in projects related to networking and cloud while he keeps himself abreast with the latest developments in SDN and NFV. In his free time, he is a hobbyist photographer trying to find his niche at the intersection of art and technology.
