Photo by Fabian Blank on Unsplash
Infracost and Terraform: Giving Developers the Tools to Manage Cloud Costs
Managing the cost of your cloud infrastructure can be a complex and daunting task, especially as your organization grows and the number of resources increases. One way to handle this challenge is to use a tool like Infracost, which allows you to see the cost of your resources in real-time and make more informed decisions about where to optimize.
In this post, we'll take a look at how to use Infracost with Terraform to gain greater visibility into your infrastructure costs and make cost control a shared responsibility across your organization.
First, let's talk about what Infracost is and how it works. Infracost is an open-source tool that generates cost estimates for your cloud resources by integrating with your infrastructure as code repository. It supports multiple cloud providers, including AWS, Azure, and GCP, and can be used with Terraform and soon with CloudFormation and Pulumi.
The most fascinating aspect here is that Infracost empowers developers to take the driver's seat in cloud cost control by providing them with real-time visibility into the costs of their resources and the ability to see the cost impact of changes before they are applied.
This gives developers the tools they need to identify high-cost resources and make adjustments to them, such as scaling down or deleting them, to reduce costs. Additionally, the ability to see cost savings opportunities and include cost estimates in their pull requests enables developers to make more informed decisions about which resources to create, modify or delete and share their cost-saving measures with other stakeholders.
With Infracost, developers can take ownership of their resources costs and take action to optimize them, and also, they can assist in the budgeting process and make sure that they are aligned with the organization's financial goals.
This way, Infracost can be a valuable tool that empowers developers to help with cost control, promoting a culture of cost awareness, and encouraging collaboration across the organization to optimize the cloud infrastructure costs.
A word about cost estimate reports
It is important to note that Infracost cost estimates are based on the standard public pricing provided by cloud providers such as AWS, GCP, or Azure and do not take into account any discounts that your project or organization may have.
However, even with standard public prices, Infracost can still provide a good estimate of your project's costs.
By using Infracost Cloud, the SaaS product that builds on top of the Infracost open-source tool, you can create custom price books that reflect any discounts you may have, resulting in more accurate cost estimates that align with the invoices you receive at the end of each month.
How to use
To use Infracost, you can easily install it by following the instructions provided on the Infracost website. Once installed, it can be integrated into any Terraform project.
In this post, we will be using a simple Terraform project as an example. You can access this example project on GitHub at the following link: https://github.com/devrealm/terraform-infracost
provider "aws" {
region = "us-east-1"
skip_credentials_validation = true
skip_requesting_account_id = true
access_key = "mock_access_key"
secret_key = "mock_secret_key"
}
resource "aws_instance" "ec2_instance_example" {
ami = "ami-0abe92d15a280b758"
instance_type = "t2.micro"
tags = {
Name = "example-instance"
}
}
resource "aws_s3_bucket" "s3_example_bucket" {
bucket = "example-bucket"
}
resource "aws_db_instance" "rds_example" {
engine = "postgres"
instance_class = "db.t2.micro"
name = "example-db"
allocated_storage = 20
storage_type = "gp2"
tags = {
Name = "example-rds-instance"
}
}
After installing Infracost, we can start getting cost estimates for our infrastructure:
$ cd terraform-infracost
$ infracost breakdown --path .
Evaluating Terraform directory at .
Project: devrealm/terraform-infracost
Name Monthly Qty Unit Monthly Cost
aws_db_instance.rds_example
├─ Database instance (on-demand, Single-AZ, db.t2.micro) 730 hours $13.14
└─ Storage (general purpose SSD, gp2) 20 GB $2.30
aws_instance.ec2_instance_example
├─ Instance usage (Linux/UNIX, on-demand, t2.micro) 730 hours $8.47
└─ root_block_device
└─ Storage (general purpose SSD, gp2) 8 GB $0.80
aws_s3_bucket.s3_example_bucket
└─ Standard
├─ Storage Monthly cost depends on usage: $0.023 per GB
├─ PUT, COPY, POST, LIST requests Monthly cost depends on usage: $0.005 per 1k requests
├─ GET, SELECT, and all other requests Monthly cost depends on usage: $0.0004 per 1k requests
├─ Select data scanned Monthly cost depends on usage: $0.002 per GB
└─ Select data returned Monthly cost depends on usage: $0.0007 per GB
OVERALL TOTAL $24.71
──────────────────────────────────
3 cloud resources were detected:
∙ 3 were estimated, all of which include usage-based costs, see https://infracost.io/usage-file
It's worth noting that for certain services that cost is based on usage, such as S3, Infracost may not be able to provide cost estimates by default. However, Infracost has a solution for this by allowing you to provide usage information.
In this example, you can create a file named infracost-usage.yml that contains the estimations of your S3 bucket usage. This file would look something like this:
# You can use this file to define resource usage estimates for Infracost to use when calculating
# the cost of usage-based resource, such as AWS Lambda.
# `infracost breakdown --usage-file infracost-usage.yml [other flags]`
# See https://infracost.io/usage-file/ for docs
version: 0.1
resource_usage:
aws_s3_bucket.s3_example_bucket:
standard: # Usages of S3 Standard:
storage_gb: 10000 # Total storage in GB.
monthly_tier_1_requests: 1000000 # Monthly PUT, COPY, POST, LIST requests (Tier 1).
monthly_tier_2_requests: 100000 # Monthly GET, SELECT, and all other requests (Tier 2).
monthly_select_data_scanned_gb: 10000 # Monthly data scanned by S3 Select in GB.
monthly_select_data_returned_gb: 1000 # Monthly data returned by S3 Select in GB.
You can provide this usage file as an input to Infracost and it will use this information to generate cost estimates for your S3 bucket.
Now, with the usage information provided in the infracost-usage.yml file, Infracost will be able to calculate the cost for the S3 service and provide an estimate of, for example, $280.45.
$ infracost breakdown --path . --usage-file infracost-usage.yml
Project: devrealm/terraform-infracost
Name Monthly Qty Unit Monthly Cost
aws_db_instance.rds_example
├─ Database instance (on-demand, Single-AZ, db.t2.micro) 730 hours $13.14
└─ Storage (general purpose SSD, gp2) 20 GB $2.30
aws_instance.ec2_instance_example
├─ Instance usage (Linux/UNIX, on-demand, t2.micro) 730 hours $8.47
└─ root_block_device
└─ Storage (general purpose SSD, gp2) 8 GB $0.80
aws_s3_bucket.s3_example_bucket
└─ Standard
├─ Storage 10,000 GB $230.00
├─ PUT, COPY, POST, LIST requests 1,000 1k requests $5.00
├─ GET, SELECT, and all other requests 100 1k requests $0.04
├─ Select data scanned 10,000 GB $20.00
└─ Select data returned 1,000 GB $0.70
OVERALL TOTAL $280.45
──────────────────────────────────
3 cloud resources were detected:
∙ 3 were estimated, all of which include usage-based costs, see https://infracost.io/usage-file
It's important to note that the usage file can be tailored to the specific resources in your project, and Infracost also provides a reference usage file that can serve as a guide for providing usage information for different supported resources.
Show cost estimate diff
Infracost also allows you to see the impact of changes to your resources on your cost estimates.
For instance, you can modify the EC2 and RDS instance types and see how it would affect the overall cost.
First, let's start by getting a baseline cost estimate by running the infracost breakdown command and save the output to a file.
infracost breakdown --path . --format json --out-file infracost-base.json --usage-file infracost-usage.ym
It's important to remember that when creating the baseline you need to include the usage file by passing the flag --usage-file infracost-usage.yml in the infracost breakdown command.
After that, we can make changes in the Terraform project by updating the EC2 and RDS instance types to "m5.4xlarge" and "m5.8xlarge" respectively. You can find the updated Terraform project in a new branch in the project's repository.
To see the impact of these changes on the cost estimate, we can then run the Infracost diff command to generate a new cost estimate report.
$ infracost diff --path . --compare-to infracost-base.json --usage-file infracost-usage.yml
Project: devrealm/terraform-infracost
~ aws_db_instance.rds_example
+$2,066 ($15.44 → $2,081)
~ Database instance (on-demand, Single-AZ, db.t2.micro → db.m5.8xlarge)
+$2,066 ($13.14 → $2,079)
~ aws_instance.ec2_instance_example
+$552 ($9.27 → $561)
~ Instance usage (Linux/UNIX, on-demand, t2.micro → m5.4xlarge)
+$552 ($8.47 → $561)
Monthly cost change for devrealm/terraform-infracost
Amount: +$2,618 ($280 → $2,899)
Percent: +934%
──────────────────────────────────
Key: ~ changed, + added, - removed
3 cloud resources were detected:
∙ 3 were estimated, all of which include usage-based costs, see https://infracost.io/usage-file
As you can see, the cost for RDS service has increased dramatically from $13.14 to $2,079 and the cost for EC2 service has increased from $280 to $2,899.
With Infracost, you can also see the cost estimate difference as a comment on a pull request or a commit, which can be useful in a CI/CD pipeline. You can learn more about this feature by reading the Infracost documentation.
Infracost Cloud
Infracost Cloud is a SaaS product that builds on top of the Infracost open-source tool, providing an easy-to-use dashboard and detailed reports for controlling costs in one central location.
To set up Infracost Cloud, you can follow the instructions provided and link your Terraform project repository. Once setup is completed, you will have access to your repositories and their associated cost estimate reports.
For example, in the screenshot below, you can see the main screen of the devrealm/terraform-infracost repository, which shows a pull request to upgrade the EC2 and RDS instance types, and the resulting increase in costs.
By navigating to the "All estimates" tab, you can see the cost estimate report for the initial commit in the main branch.
Finally, by checking the "Pull requests" tab, you can view the cost estimate report for the PR and see the details of the difference compared to the main branch or the total cost in a tabular format.
In conclusion, Infracost is a powerful tool that allows developers to take a proactive approach to manage cloud infrastructure costs.
By providing real-time cost estimates and alerts, developers can identify and address cost issues before they become significant problems. This not only helps to keep costs under control but also promotes a culture of cost awareness and responsibility within development teams. By integrating Infracost into your development workflow, you can gain greater visibility and control over your cloud infrastructure costs, enabling you to make more informed decisions and optimize your resources.
To start using Infracost, you can visit their website at https://www.infracost.io and check their documentation.
Get in touch
Your feedback is appreciated, please leave a comment on the post to let me know what you think.
If you're ready to take the next step in cost-effective cloud management with Infracost and Terraform, reach out at george@devrealm.org or connect on LinkedIn at https://www.linkedin.com/in/georgevagenas/ to see how I can help.