5 Common Mistakes When Automating Application Deployment
I have been automating the deployment of applications to physical and cloud-based servers for many years. Here are five mistakes that I commonly find when helping teams with their deployment automation.
1. Assuming everything will be deployed on one server
Most applications begin with a simple deployment model that includes a single app server and a database server (often on the same instance). Over time, the need to add more horsepower, background processing, and file system storage will require moving your application to multiple servers. No longer will files exist on the local filesystem or all queries execute on the same machine. More application configuration will be required, as will automated server setup. Usually at the worst time.
Don’t assume everything will be deployed to the same server. Instead, externalize all integration points into configuration files or use environment variables. This will allow you to move parts of your application around as needed with minimal impact to the application.
Also, select a file storage strategy that will scale beyond a single server. Solutions such as Amazon S3 or Rackspace Cloud Files are common, but will likely have slower performance than a local file system. Identifying the solution up-front that fits the needs of your application will save you from the time required to move files around from one service to another. Alternatively, consider a distributed file system like Ceph or utilizing features such as GridFS from MongoDB – just be aware of their intended use cases and limitations.
Finally, build your application with the assumption that multiple database replicas will exist in the future. There are often libraries for enabling frameworks like Ruby on Rails to round robin to read-only replicas, while sending all INSERTs and UPDATEs to the master database. Introducing this complexity up-front is much easier than later in the life of the application, when the assumption of a single database server exists throughout the code.
2. The Lack of ability to scale up and down
Part of the automation process is understanding how your application will behave as new servers are added or removed. If the list of known servers must exist in a configuration file, removed servers must be deleted from the file and the processes notified of the change (if the contents of the config file is cached in memory). This introduces greater complexity within your application and makes it more difficult to add or remove servers as needed.
Whenever possible, group servers behind a load balancer or utilize round robin DNS to rotate between servers internally. This isolates the knowledge of the network topology and deployment configuration from the application and enables a load balancer or internal DNS server to be updated quickly with new or outdated servers.
Finally, keep in mind that autoscaling introduces new issues into the deployment process. Instead of manually running automation scripts to setup a new server with the latest production code, autoscalers start and stop machines on their own. This means that servers must be capable of “pulling” their configuration scripts and executing them prior to being added to the server pool. This also has an impact on server group configuration, as previously mentioned. This is a vastly different approach than assuming that a human has provisioned or removed a machine manually.
3. Creating custom deployment scripts
Unless you and your team are experienced in building modular, easily understood shell scripts that manage the deployment and configuration of servers, then stick with a configuration management framework. A variety of solutions exist, including Chef, Puppet, and CFEngine, that should meet your needs. Plus, new team members can easily obtain training and support for these CM tools, without having to understand about that bad night you had writing and fixing your shell scripts.
4. Requiring manual steps to complete a deployment
Deployment automation means automating 100% of the deployment process. If you require a manual step to complete the deployment, you open yourself up to missing a step in the future and creating downtime after a production deployment. The best case is that other members of your team lose productivity while trying to deploy the latest version of the application into a development or staging environment. Ensure that all steps necessary to deploy and configure your application are automated and fully understood.
5. Not deploying often enough
If you have automated your deployment, why not deploy more often? The longer you wait to deploy, the more risk that builds due to forgotten configuration changes and hidden issues. Plus, you begin to deliver more value to your customers and your business sooner by deploying more often.