Get started with AWS CloudFormation CFN Helper Scripts

AWS CloudFormation CFN helper scripts are a collection of scripts provided by Amazon Web Services (AWS) to assist in the deployment and management of resources using AWS CloudFormation. CloudFormation is a service that allows you to describe and provision AWS resources in a declarative manner, using JSON or YAML templates.

The CFN helper scripts are designed to be used within the EC2 instances or other resources that you create or manage using CloudFormation. These scripts facilitate common tasks such as configuring instances, installing software, and executing custom scripts during the resource provisioning process.

The CFN helper scripts consist of the following components:

  1. cfn-init: This script is used to retrieve and interpret metadata from the CloudFormation template. It enables you to define configuration data, packages, files, services, and commands that should be applied to the instance during initialization. The script supports pulling configuration from various sources such as Amazon S3, Amazon SSM Parameter Store, or inline within the CloudFormation template itself.

  2. cfn-signal: After the resource is successfully provisioned, cfn-signal is used to send a success signal back to CloudFormation. This signal confirms that the resource has been provisioned and is ready to be used. CloudFormation waits for the signal before proceeding with the creation of dependent resources, ensuring that resources are created in the correct order.

  3. cfn-get-metadata: This script retrieves metadata associated with the EC2 instance. It allows you to fetch metadata about the instance, stack, or any other resource-specific metadata defined in the CloudFormation template.

  4. cfn-hup: The cfn-hup script monitors changes to the specified configuration files and triggers custom actions whenever changes are detected. This allows you to automate tasks based on changes to the configuration of your resources.

These helper scripts can be embedded within your EC2 instances or other resources by leveraging user data or the CloudFormation template's metadata section. They provide a convenient way to automate tasks and ensure proper configuration of your resources during their provisioning and lifecycle management.

By using the CFN helper scripts, you can enhance the functionality and automation capabilities of your CloudFormation templates, making it easier to manage your AWS infrastructure as code.

Amazon Linux AMI images

The CloudFormation helper scripts are preinstalled on Amazon Linux AMI images that have bootstrap scripts installed.

  • On the latest Amazon Linux AMI version, the scripts are installed in /opt/aws/bin.

  • On previous Amazon Linux AMI versions, the aws-cfn-bootstrap package that contains the scripts is located in the Yum repository.

The helper scripts are pre-installed on the latest versions of the Amazon Linux AMI and not on Optimized AMIs, such as ECS Optimized Image that uses Amazon Linux as base.

Using the latest version of helper scripts

The helper scripts are updated periodically. If you use the helper scripts, ensure that your launched instances are using the latest version of the scripts:

  • Include the following command in the UserData property of your template before you call the scripts. This command ensures that you get the latest version:

    yum install -y aws-cfn-bootstrap

  • If you don't include the yum install command and you use the cfn-init, cfn-signal, or cfn-get-metadata scripts, then you'll need to manually update the scripts in each Amazon EC2 Linux instance using this command:

    sudo yum install -y aws-cfn-bootstrap

    Running sudo yum install -y aws-cfn-bootstrap installs the helper scripts from the yum repository.

  • If you don't include the yum install command and you use the cfn-hup script, then you'll need to manually update the script in each Amazon EC2 Linux instance using these commands:

    sudo yum install -y aws-cfn-bootstrap

    sudo /sbin/service cfn-hup restart

    Running sudo yum install -y aws-cfn-bootstrap installs the helper scripts from the yum repository.

  • If you use the source code for the scripts to work with another version of Linux or a different platform, and you have created your own certificate trust store, you'll also need to keep the trust store updated.

For the version history of the aws-cfn-bootstrap package, see Release history for AWS CloudFormation helper scripts.

cnf-init script

The cfn-init script is one of the AWS CloudFormation helper scripts that is used to retrieve and interpret metadata from the CloudFormation template. It enables you to define configuration data, packages, files, services, and commands that should be applied to an instance during initialization.

The cfn-init helper script reads template metadata from the AWS::CloudFormation::Init key and acts accordingly to:

  • Fetch and parse metadata from CloudFormation

  • Install packages

  • Write files to disk

  • Enable/disable and start/stop services

Here's an example of how to use the cfn-init script in a CloudFormation template:

Resources:
  MyKeyPair:
    Type: "AWS::EC2::KeyPair"
    Properties:
      KeyName: "my-keypair"
      PublicKeyMaterial: "ssh-rsa AAAAB3N==... user@host"

  MySecurityGroup:
    Type: "AWS::EC2::SecurityGroup"
    Properties:
      GroupDescription: "Allow SSH access"
      SecurityGroupIngress:
        - CidrIp: "<your_ip>/32"
          FromPort: 22
          ToPort: 22
          IpProtocol: "tcp"

  MyEC2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      ImageId: "ami-06ca3ca175f37dd66"
      InstanceType: "t2.micro"
      KeyName: !Ref MyKeyPair
      SecurityGroupIds:
        - !Ref MySecurityGroup
      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash -ex

          # Install and configure cfn-init script
          yum install -y aws-cfn-bootstrap
          /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource MyEC2Instance --region ${AWS::Region}

          # Install additional packages and configure the instance
          # yum install -y <package1> <package2>

          # Perform other instance setup tasks

    Metadata:
      AWS::CloudFormation::Init:
        config:
          packages:
            yum:
              httpd: []

          files:
            /var/www/html/index.html:
              content: |
                <html>
                  <body>
                    <h1>Hello World!</h1>
                  </body>
                </html>

          services:
            sysvinit:
              httpd:
                enabled: true
                ensureRunning: true

In the above example:

  • Security Group and SSH keys are created to connect to EC2 instance and check the execution of the cnf-init script. To check the output logs use the following command:

      cat /var/log/cloud-init-output.log
    

  • The UserData property is used to specify a bash script that will be executed when the EC2 instance is launched. It installs the aws-cfn-bootstrap package, which includes the cfn-init script. The cfn-init command is then invoked, passing the stack name, resource logical ID, and region as parameters.

  • The Metadata section includes the AWS::CloudFormation::Init key, which contains a config section. Within the config section, you can define files that should be created or modified on the instance using the files key. In this example, a configuration file is defined at /path/to/config-file.

When CloudFormation creates or updates the stack, it will launch the EC2 instance and execute the cfn-init script. The script will retrieve the metadata defined in the template, create or modify files specified in the files section, and installs packages from packages section. This allows you to perform custom configurations and execute scripts during the provisioning process.

To deploy the stack use the following command:

aws cloudformation create-stack --stack-name MyEC2Stack --template-body file://ec2-userdata.yaml

To delete stack:

aws cloudformation delete-stack --stack-name MyEC2Stack

Files and Command config parameters

The files and commands sections are part of the AWS CloudFormation metadata under the AWS::CloudFormation::Init key. They allow you to define and configure files and commands that should be executed on your EC2 instances or other resources during the provisioning process.

  1. Files Section: The files section allows you to specify files that should be created or modified on your instances. You define the file paths and their associated content within this section. The content can be provided inline within the CloudFormation template or fetched from external sources like Amazon S3.

For example, to create an index.html file with specific content, you can use the following configuration:

files:
  /var/www/html/index.html:
    content: |
      <html>
        <body>
          <h1>Hello World!</h1>
        </body>
      </html>

This configuration creates the /var/www/html/index.html file with the specified HTML content on the instance.

You can also define files that are sourced from external locations. For example, to fetch a configuration file from an S3 bucket, you can use the source attribute:

files:
  /path/to/config-file:
    source: s3://my-bucket/config-file

This configuration fetches the config-file from the my-bucket S3 bucket and places it at /path/to/config-file on the instance.

  1. Commands Section: The commands section allows you to specify commands that should be executed on your instances during the provisioning process. You can define shell commands, scripts, or call external scripts hosted in various locations.

For example, to run a custom script located at /path/to/custom-script.sh, you can use the following configuration:

commands:
  01_run_script:
    command: "/path/to/custom-script.sh"

This configuration executes the custom-script.sh located at /path/to/custom-script.sh on the instance.

You can also define commands inline using multiple lines:

commands:
  02_another_command:
    command: |
      echo "Hello"
      echo "World"

This configuration executes two echo commands on the instance.

By utilizing the files and commands sections in the CloudFormation metadata, you can automate the creation/modification of files and execute commands on your instances during the provisioning process. This allows you to perform custom configurations, install software, and run specific tasks to set up your resources as desired.

Services and Packages config parameters

The services and packages sections are part of the AWS CloudFormation metadata under the AWS::CloudFormation::Init key. They allow you to define and configure services and packages that should be installed and managed on your EC2 instances or other resources during the provisioning process.

  1. Services Section: The services section allows you to specify services that should be enabled and running on your instances. You can define services using different service managers, such as sysvinit (SysVinit), upstart (Upstart), or systemd (systemd init system). Within the services section, you can specify service names, configuration options, and their desired state (enabled/disabled, running/stopped).

For example, to enable and ensure the Apache HTTP server (httpd) is running, you can use the following configuration:

services:
  sysvinit:
    httpd:
      enabled: true
      ensureRunning: true

This configuration enables the httpd service and ensures it is running on the instance.

  1. Packages Section: The packages section allows you to specify the packages (software) that should be installed on your instances. You can define packages using the package manager relevant to your operating system, such as yum (for Amazon Linux, CentOS, and RHEL) or apt (for Ubuntu and Debian). Within the packages section, you specify the package names to be installed.

For example, to install Apache HTTP server (httpd) and MySQL client (mysql), you can use the following configuration:

packages:
  yum:
    httpd: []
    mysql: []

This configuration ensures that the httpd and mysql packages are installed using yum package manager.

By utilizing the services and packages sections in the CloudFormation metadata, you can automate the installation and management of services and packages on your instances during the provisioning process. This ensures that your instances have the necessary software and services in the desired state when the CloudFormation stack is created or updated.

cfn-signal script

The cfn-signal script is used to send a signal to AWS CloudFormation to indicate the status of a resource or instance. This helps CloudFormation wait for a specific resource to be ready before proceeding with the rest of the stack creation/update.

Here's an example code snippet demonstrating the usage of the cfn-signal script:

Resources:
  MyKeyPair:
    Type: "AWS::EC2::KeyPair"
    Properties:
      KeyName: "my-keypair"
      PublicKeyMaterial: "ssh-rsa AAAAB3N==... user@host"

  MySecurityGroup:
    Type: "AWS::EC2::SecurityGroup"
    Properties:
      GroupDescription: "Allow SSH access"
      SecurityGroupIngress:
        - CidrIp: "<your_ip>/32"
          FromPort: 22
          ToPort: 22
          IpProtocol: "tcp"

  MyEC2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      ImageId: "ami-06ca3ca175f37dd66"
      InstanceType: "t2.micro"
      KeyName: !Ref MyKeyPair
      SecurityGroupIds:
        - !Ref MySecurityGroup
      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash -ex

          # Install and configure cfn-init script
          yum install -y aws-cfn-bootstrap
          /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource MyEC2Instance --region ${AWS::Region}

          # Install additional packages and configure the instance
          # yum install -y <package1> <package2>

          # Perform other instance setup tasks

          # Signal CloudFormation that instance is ready
          /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource MyEC2Instance --region ${AWS::Region}

In this example:

  • After the instance setup tasks, the cfn-signal script is invoked to send a success signal to CloudFormation. The -e flag indicates the exit status of the instance setup tasks, which is denoted by $?. The script uses the --stack and --resource options to specify the stack and resource logical ID, and the --region option to specify the AWS region.

When the instance setup tasks complete successfully, the cfn-signal script will send a success signal to CloudFormation. CloudFormation will then proceed with the creation/update of other resources dependent on the signaled instance.

Note: It's important to ensure that the exit status ($?) passed to the cfn-signal script accurately reflects the success or failure of the instance setup tasks. This helps CloudFormation determine the status of the resource correctly.

You can check the execution of the script in the logs:

cfn-get-metadata script

The cfn-get-metadata script is one of the AWS CloudFormation helper scripts used to retrieve metadata associated with an EC2 instance or other resources provisioned through CloudFormation. It allows you to fetch metadata about the instance, stack, or any other resource-specific metadata defined in the CloudFormation template.

Here's an explanation of the cfn-get-metadata script and a code example to demonstrate its usage:

  1. Explanation: The cfn-get-metadata script is designed to retrieve metadata from the CloudFormation template and make it available to the EC2 instance or resource where it is executed. Metadata can be defined in the template at different levels, such as stack-level metadata or resource-level metadata. The script provides a convenient way to access and utilize this metadata within your instance or resource.

  2. Code Example: To demonstrate the usage of the cfn-get-metadata script, consider the following example:

Resources:
  MyKeyPair:
    Type: "AWS::EC2::KeyPair"
    Properties:
      KeyName: "my-keypair"
      PublicKeyMaterial: "ssh-rsa AAAAB3...  == user@host"

  MySecurityGroup:
    Type: "AWS::EC2::SecurityGroup"
    Properties:
      GroupDescription: "Allow SSH access"
      SecurityGroupIngress:
        - CidrIp: "<your_ip>/32"
          FromPort: 22
          ToPort: 22
          IpProtocol: "tcp"

  MyEC2Instance:
    Type: "AWS::EC2::Instance"
    Properties:
      ImageId: "ami-06ca3ca175f37dd66"
      InstanceType: "t2.micro"
      KeyName: !Ref MyKeyPair
      SecurityGroupIds:
        - !Ref MySecurityGroup
      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash -ex

          # Install and configure cfn-init script
          yum install -y aws-cfn-bootstrap
          /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource MyEC2Instance --region ${AWS::Region}

          # Install additional packages and configure the instance
          # yum install -y <package1> <package2>

          # Perform other instance setup tasks

          # Retrieve metadata and output to a file
          /opt/aws/bin/cfn-get-metadata --stack ${AWS::StackName} --resource MyEC2Instance --region ${AWS::Region} > /home/ec2-user/metadata.txt


    Metadata:
      AWS::CloudFormation::Init:
        config:
          packages:
            yum:
              httpd: []

          files:
            /var/www/html/index.html:
              content: |
                <html>
                  <body>
                    <h1>Hello World!</h1>
                  </body>
                </html>

          services:
            sysvinit:
              httpd:
                enabled: true
                ensureRunning: true

In this example:

  • After installing the package, the cfn-get-metadata script is invoked with the appropriate parameters. The --stack option is used to specify the stack name, the --resource option is used to specify the resource logical ID, and the --region option is used to specify the AWS region.

  • The metadata retrieved by the cfn-get-metadata script is redirected to a file (/path/to/metadata.txt) using the > operator.

By executing this script within the instance or resource, you can retrieve and store the metadata associated with the stack or specific resources in the /path/to/metadata.txt file. You can then process or utilize this metadata as needed within your instance.

Note: The cfn-get-metadata script provides various options to fetch specific types of metadata or filter the metadata based on keys or namespaces. You can refer to the AWS CloudFormation documentation for more details on advanced usage of this script.

To deploy the stack use the following command:

aws cloudformation create-stack --stack-name CFNMetadata --template-body file://ec2-cfn-get-metadata.yaml

After deployment at the home directory you will find the metadata.txt file:

cfn-hup script

The cfn-hup helper is a daemon that detects changes in resource metadata and runs user-specified actions when a change is detected. This allows you to make configuration updates on your running Amazon EC2 instances through the UpdateStack API action.

Here's an explanation of the cfn-hup script and a code example to demonstrate its usage:

Example usage of cfn-hup script can be found here: https://gist.github.com/libert-xyz/d1d7186ef3dc7ad23cbfe46eaf3fb27b

References

  1. CloudFormation helper scripts reference

  2. cfn-init

  3. cfn-signal

  4. cfn-get-metadata

  5. cfn-hup

  6. https://gist.github.com/libert-xyz/d1d7186ef3dc7ad23cbfe46eaf3fb27b