From tag compliance to garbage collection, Cloud Custodian does more than AWS resource management, all through a CLI and using YaML files to define policies.
AWS allows you to build enormous and complex cloud infrastructures in a matter of hours. With the ability to create resources so easily, sometimes it can be hard to manage all those resources. If only there were a simple but powerful tool that could manage it all. Cloud Custodian (a.k.a C7N) is a Python CLI tool that gives you powerful account management capabilities with a simple config file. Cloud Custodian can help you manage your AWS account using a simple policy config file and time-based or event-based Lambdas. The config files (YAML formatted) allow you to define policies for everything from tag compliance to backups. Define policies for a wide variety of management activities, including garbage collection to encryption.
Cloud computing has made creating and managing web resources insanely easy, quite possibly too easy. You can now spin up quite a few computing, database, and storage resources with the click of a button or the stroke of a return key. However, if you use a company account, you likely spin up those resources often for demonstration and testing purposes, without considering the cost or clutter you might be creating along with it. This was “the problem” at Capital One when they created this very powerful tool for managing the cleanup of your superfluous cloud resources. Capital One started developing Cloud Custodian in July 2015 and open-sourced the tool in April 2016.
Cloud Custodian’s feature-set has grown exponentially with its popularity because they’ re very good about responding to feature requests. It’s now grown to the point where there’s not much in the AWS world you can’ t do with it. Here’s a short list of some things you might be surprised it can do:
Odds are though, you’ re considering Cloud Custodian for its namesake: cleaning up your AWS account; resource/cost management during off-hours; and overall garbage collection. True to its name, this is where Cloud Custodian excels. With a relatively simple configuration file, you can tidy and trim your AWS account and keep it that way as you grow your business.
Cloud Custodian is great for mid-to-large sized companies that give a large number of their employees full access to a company AWS account. Naturally, their account quickly becomes cluttered with dozens of CloudFormation stacks, VPCs, old test instances, and Lambda functions. Here at Stelligent, we have an AWS Labs account for exploring and testing in AWS. We use Cloud Custodian to clean up old testing resources based on age and resource tags.
Cloud Custodian is very well documented, so if you’ re excited to start taking out the digital trash in your AWS account there’s no better place to start than their website and documentation. There are a few things to keep in mind before diving head-first into the cloud equivalent of the custodial arts:
The easiest way to get started cleaning up your AWS account with Custodian is to go through your account and tag everything you want to keep with something like “NoCustodian”. Then, set
Click the button below to launch an example CloudFormation Stack that boots an EC2 instance and then uses Custodian to stop the instance.
The first thing you’ ll be tempted to do when implementing Cloud Custodian is terminate all the old and unused resources in your account. Just be sure all the relevant parties in your company know what you’ ll be terminating and when.
If you need better visibility and automated management of your AWS account, Cloud Custodian has lots of helpful features that are easy to manage in a single config file. If you aren’ t already a python developer, I recommend setting up a CloudFormation template or Automated Pipeline to manage changes to yours. You can use the launch button in Pro Tip #1 to see an example of Custodian in a CloudFormation template. Keep a look out for a future blog post with a detailed example of a fully-automated Custodian Pipeline.
Custodian Website
Custodian Docs
Custodian GitHub
AWS This Is My Architecture: Cloud Custodian
Cloud Custodian @ AWS re: Invent
Cloud Custodian @ Serverlessconf
