Automating Cloud Cost Optimization with Make and AWS CloudWatch: Implementing Real-time Monitoring and Automated Resource Adjustment

Cloud cost management means more than just budget savings. By combining Make (formerly Integromat) with AWS CloudWatch, you can accurately identify resource usage through real-time monitoring, build automated workflows to adjust resources as needed, thereby maximizing operational efficiency and preventing unnecessary cost expenditures. In this article, we will demonstrate the effectiveness of this powerful combination through a real implementation case.

1. The Challenge / Context

Cloud costs are difficult to predict and tend to increase continuously. Many companies fail to properly understand resource usage and pay for over-provisioned instances or idle resources. While CloudWatch is a powerful monitoring tool, manually setting up alarms and responding to them requires significant time and effort. Especially for small teams or individual developers, automating these tasks is crucial. To address these issues, we introduce a method to implement real-time monitoring and automated resource adjustment by integrating Make, a no-code automation platform, with AWS CloudWatch.

2. Deep Dive: Make and AWS CloudWatch

Make (formerly Integromat) is a powerful platform that allows you to automate workflows by connecting various web services and APIs without coding. Its intuitive visual interface makes it easy to implement complex logic, and it can integrate with various AWS services like AWS CloudWatch. Make enables the automation of various actions, such as invoking Lambda functions, resizing EC2 instances, and scaling RDS instances, by using CloudWatch alarms as triggers. AWS CloudWatch is a service used to monitor AWS resources and applications. It collects various metrics such as CPU utilization, memory usage, network traffic, and disk I/O, and allows you to create and monitor custom metrics. Furthermore, if a specific metric exceeds a defined threshold, it can trigger an alarm to notify administrators or perform automated actions in conjunction with other AWS services. The key is that CloudWatch provides the data, and Make performs automated actions based on that data.

3. Step-by-Step Guide / Implementation

The following is a step-by-step guide to implementing a workflow that monitors the CPU utilization of an EC2 instance using Make and AWS CloudWatch, and automatically scales out the EC2 instance if CPU utilization exceeds a specific threshold.

Step 1: Configure AWS CloudWatch Alarm

First, configure an alarm for the CPU utilization of an EC2 instance in the AWS CloudWatch console. When setting up the alarm, consider the following settings:

  • Metric Name: CPUUtilization
  • Namespace: AWS/EC2
  • Statistic: Average
  • Period: 1 minute (or 5 minutes, depending on your needs)
  • Threshold: 70% (Alarm triggers if CPU utilization exceeds 70%)
  • Evaluation Period: 5 minutes (Alarm triggers if CPU utilization exceeds 70% for 5 minutes)
  • Actions: Create SNS topic and subscribe an email address to it for testing.

SNS topic created will be used to trigger Make scenario.


    # AWS CLI Example (for reference - you'll likely use the AWS Console initially)
    aws cloudwatch put-metric-alarm \
        --alarm-name "HighCPUUtilizationAlarm" \
        --metric-name CPUUtilization \
        --namespace AWS/EC2 \
        --statistic Average \
        --period 60 \
        --unit Percent \
        --evaluation-periods 5 \
        --threshold 70 \
        --comparison-operator GreaterThanThreshold \
        --dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx \
        --alarm-actions arn:aws:sns:your-region:your-account-id:YourSNSTopic

Step 2: Create Make Scenario

Log in to Make and create a new scenario.

  • Module 1: Webhooks - Custom Webhook. This will receive the notification from the SNS topic you configured in Step 1. After adding the webhook, copy the provided URL.
  • Module 2: JSON - Parse JSON. Parse the JSON data received from the Webhook. This allows you to easily access the data within the SNS message. Example: {{body.Message}}
  • Module 3: AWS EC2 - Run Instances. This module will create a new EC2 instance. Configure it with the appropriate AMI, instance type, subnet, security group, and other necessary settings. This scales OUT a new instance.
  • (Optional) Module 4: Email - Send an Email. Send an email notification to alert you that a new EC2 instance has been created.

Important: Before running the Make scenario, you must establish a connection with your AWS account. In Make, select the AWS module and enter your IAM user credentials (access key ID and secret access key) to set up the connection. **The IAM user must have permissions to start and stop EC2 instances.**


    # Example Make scenario configuration (pseudocode)
    Module 1: Webhook (Custom Webhook)
      - Listener for CloudWatch SNS Notifications
      - Webhook URL: [Generated by Make]

    Module 2: JSON Parse
      - Data: {{Webhook.body}}
      - JSON Structure: Automatic

    Module 3: AWS EC2 - Run Instances
      - Connection: [Your AWS Connection]
      - Image ID: [Your AMI ID]
      - Instance Type: t2.micro (or desired instance type)
      - ... other EC2 configuration ...

Step 3: Test Make Scenario

Test the Make scenario by manually triggering an alarm in the CloudWatch console. When the CloudWatch alarm is triggered, verify that the Make scenario runs and a new EC2 instance is created. If an error occurs, check the Make scenario's logs to troubleshoot the issue.

Step 4: Scaling In (Optional)

If CPU utilization decreases, you can also implement a scale-in workflow that automatically terminates EC2 instances. This requires setting up additional CloudWatch alarms and modifying the Make scenario to include the functionality to terminate EC2 instances.


    # Example Make scenario configuration for Scale-In (pseudocode)

    # Similar to Step 2, but with different CloudWatch alarm and EC2 module configuration

    Module 1: Webhook (Custom Webhook)
      - Listener for CloudWatch SNS Notifications (low CPU utilization)
      - Webhook URL: [Generated by Make]

    Module 2: JSON Parse
      - Data: {{Webhook.body}}
      - JSON Structure: Automatic

    Module 3: AWS EC2 - Terminate Instances
      - Connection: [Your AWS Connection]
      - Instance ID: [Retrieved from Webhook data]
      - ... other EC2 configuration ...

4. Real-world Use Case / Example

I operate a small-scale web application for a personal project. In the past, the server would often go down whenever traffic surged. So, I integrated Make and CloudWatch using the method described above to automatically scale out EC2 instances when traffic exceeded a certain threshold. As a result, I was able to completely eliminate server downtime and reduce unnecessary resource waste, saving approximately 30% on cloud costs. It was especially convenient to have the system automatically scale even while I was sleeping. By saving the time previously spent on manual server management, I could focus more on application development.

5. Pros & Cons / Critical Analysis

  • Pros:
    • Cost Savings: Automatically adjusts resources as needed to prevent unnecessary expenditures.
    • Automation: Saves time and effort spent on manual resource management.
    • Scalability: Automatically responds to traffic surges, increasing application availability.
    • Flexibility: Can integrate with various AWS services to build complex workflows.
    • No-Code: Allows building automation workflows without coding knowledge.
  • Cons:
    • Initial Setup Complexity: Setting up Make and AWS CloudWatch for the first time can take some time. IAM permission configuration is crucial.
    • Make Pricing: Fees may be charged based on Make usage. (There is a free plan, but it is limited.)
    • Importance of Alarm Configuration: Incorrect alarm settings can lead to unnecessary scale-out or scale-in.
    • Monitoring Required: The automated system needs continuous monitoring to ensure it is functioning correctly.

6. FAQ

  • Q: Can I implement this workflow with Make's free plan?
    A: Yes, it's possible, but the free plan has a monthly limit on the number of operations. Therefore, for applications with high traffic, a paid plan should be considered.
  • Q: What metrics should I monitor when configuring CloudWatch alarms?
    A: It depends on the application's characteristics, but generally, it's recommended to monitor CPU utilization, memory usage, network traffic, disk I/O, and so on.
  • Q: Can Make also invoke AWS Lambda functions?
    A: Yes, you can invoke Lambda functions using Make's AWS Lambda module. This allows you to build more complex automation workflows. For example, you can automate database backups, log analysis, and more.
  • Q: What should I do if Make is not receiving CloudWatch alarms?
    A: First, ensure that the CloudWatch alarm is configured correctly. Also, verify that the Webhook URL in your Make scenario is correctly set in the CloudWatch alarm. Finally, check the Make scenario's logs for any errors. IAM permission issues are also a high possibility.

7. Conclusion

Automating cloud cost optimization using Make and AWS CloudWatch provides value beyond simple cost savings. It maximizes system operational efficiency and enables developers to focus on application development. Leverage this powerful combination now to optimize your cloud costs and enhance your application's performance. We recommend referring to Make's official documentation to build workflows and exploring AWS CloudWatch's various metrics to establish an automation strategy that suits your needs.