Sorry, the language specified is not available for this page

    CloudFormation Scoping for Beginners

    August 1, 2018

    When most people begin working with CloudFormation, they usually start with examples or tutorials they find online. After that, they quickly start combining and adding their own content. Very ­soon, if not immediately, they face the decision of nesting and how much to include in a single template, not to mention the overall scope of a group of nested stacks. The obvious goal is to create as reusable a sub-template as possible.

    However, I've noticed two main pieces that most tutorials do not mention -- although everyone figures this out eventually. I hope to speed up that learning curve in this short post.

    You’ve probably created or deployed a few templates and know what the parameters are, and how you pass in values to your templates that are specific to the deployment you hope to accomplish. You might start out hard coding values specific to that deployment, but if you made that value into a parameter, you would want to make that template more reusable. This is pretty obvious, though not always followed, although this is what people do to test things out. A reusable template is the goal.

    One constraint of CloudFormation is the limitation in the complexity of the data you can implement within a parameter. One way around this is to standardize around an external datastore for those values and have a parameter that identifies that datastore and any sort of key for the resource you are creating in your template.

    For example, for ECS you could pass on the parameter store information to allow your container to access that parameter store, and then lookup your parameters as needed within your container. You would do this as opposed to passing environment variables as parameters within your CloudFormation to your container/task definition. You will most likely always need different key/value parameters per ECS service, so altering your CloudFormation templates per service goes against having generic reusable templates.

    There are many similar systems, such as etcd, consul, or any datastore like DynamoDB, but this varies depending on your comfort level. This includes your ability to run your own third-party key/value stores, possibly doing so if you are already using those system in a different part of the same environment

    The key thing most people don’t realize or think about when first coming to CloudFormation is the idea of keeping everything linked to a master CloudFormation template that must be kept consistent, upgraded, and rolled back atomically. Otherwise, you will be required to add additional logic to keep things in sync – and I can almost guarantee that your environment will be out of sync. Or, if you group unrelated things in a master template, you will eventually rollback resources that don’t need to rollback because of an unrelated failure.

    Again, the main idea with a master template (and nested templates linked to that master) is that if anything fails within a run, everything that needs to be kept in sync will be rolled back consistently. This is always important, but especially so when adding CloudFormation Actions to any sort of CI/CD Pipeline.

    It may seem easier in the moment to run your templates separately from within your pipeline, but in the long run everything will be much cleaner if your related resources that are created/updated/deleted from your templates are kept inside of a single master template. This could also result in running into CloudFormation limits related to number of phases, stages, parallel or total actions.

    As you progress with CodePipeline, at the beginning you will want to just get things working. But you will eventually find CodePipeline limits, or your pipeline just won't perform according to the goals you set for time, speed or just logical groupings. Think about how those groupings are for an efficient Development/Deployment workflow with the ability to rollback cleanly, and you will save yourself headaches in the future.

    In the end, CloudFormation is all about helping you set up AWS resources so you can focus on your apps that run in AWS, vs. managing those resources. Good luck with your CloudFormation efforts; I hope I've been able to save you a little time and eliminate a little confusion in the process.

    Other Posts You Might Be Interested In

    Run Lambda, Run

    Lambda has quickly become a core service in the AWS lineup. A utility knife for developers, Lambda can be as simple or complex as needed. It is the glue between AWS... Learn More

    How To Build A Serverless Alert Email Notification System Using Machine Learning

    As an AWS partner, we receive a lot of email messages from AWS including many that need prompt attention. But with so many emails, it is very hard to detect which emails... Learn More

    Governance at Different Scales – Part 1

    The Problem As organizations shift from traditional on-premise or dedicated hosted solutions to public cloud providers like Amazon Web Services (AWS), many are trying to... Learn More