Understanding Instance MetaData Service (IMDS)

‍
ould you leave your Strongbox open just because you locked the house door?” - “No, right?”.
IMDS is like a strongbox containing precious information about your cloud. It is important that you understand IMDS, and how to secure it. It is time you take Instance metadata service (IMDS) seriously.

What is EC2 Instance Metadata Service (IMDS)?

The Instance Metadata Service (IMDS) helps code on EC2 instance access instance metadata. IMDS provides a great amount of information about instances. This includes hostname, security group, MAC address and much more. It also hosts user-data, that you specified when launching your instance. For an attacker, this is a gold mine. You should understand the security implications of instance metadata service. This would help you to better protect your cloud.

The IMDS exposes this instance metadata through a special “link-local” IP address of 169.254.169.254. Hence only code running on the instance would be able to access the metadata. Securing IMDS becomes critical due to its ability to access the AWS access credentials for the IAM role attached to the instance.
‍

AWS Credentials on the EC2 instance

The applications running on EC instances must sign their requests with AWS credentials. One way is to provide the credentials in the form of AWS access keys/secrets. For various reasons, this is not a recommended approach.

It is cumbersome to distribute/rotate the access credentials on each instance. Even more challenging is to distribute and rotate the AWS created access credentials to the instances. eg. Spot instances or auto-scaling groups (ASGs).

Instead, you can attach an IAM role to the EC2 instance. The advantage is that you as an application developer don’t need to bother distributing and rotating the access credentials. The IAM role helps to enforce the least privilege principle, as another advantage.

The resource-level permissions in IAM policies help you decide on a granular level what IAM roles users can attach, detach or replace for an instance. In general, using IAM improves the cloud security posture.

What is an IAM InstanceProfile?

To pass an IAM role to an EC2 instance you need to use the instance profile. You can think of an instance profile as a container for the IAM role. You can attach or detach The IAM role to the instance profile. The instance profile itself can be attached or detached from the EC2 instance.

The question arises what is the difference between IAM role and instance profile. Why do we need a container around the IAM role?

Only IAM principals/identities such as users, groups, AWS services etc can assume the IAM role. The users, group, AWS services represent who assumed the role. The EC2 instance itself cannot assume a role. Hence AWS created Instance Profile, as principal to assume the role for an EC2 instance. The only thing the instance profile does is to assume the role attached to it. Beyond that instance, the profile has no other real power. The instance profile derives all its power from the assumed IAM role.

Retrieving IAM role credentials from IMDS

To retrieve the security credentials:

• Send a request to http://169.254.169.254/latest/meta-data/iam. The HTTP status code 404 indicates that no IAM role is attached to the EC2 instance. An HTTP status code 200 with an empty response indicates currently no IAM role is attached but there was an IAM role attached earlier. When no IAM role is attached to the EC2 instance, access credentials would not be available with IMDS.
• Use the IMDS request http://169.254.169.254/latest/meta-data/iam/security-credentials/ to get the associated IAM role name.
• The request http://169.254.169.254/latest/meta-data/iam/security-credentials/<iam_role_name>/ return the access credentials.

All these calls work without any authentication. It’s like keeping the strongbox open thinking no outsider can get in your house.

What makes IMDSv1 risky?

The instance metaservice is accessible from the EC2 instance only.  Once you have access to an EC2 instance you wouldn’t need any further authentication. You have already proved the ownership and no extra authentication is thus needed.

‍
In theory, IMDS metadata will not be available outside the instance. If the EC2 instance is open to the outside world, attackers have a chance to steal credentials. Attackers use Server-Side Request Forgery (SSRF), XML External Entity Injection vulnerabilities to steal. Even shell access to the instance may expose access credentials.

‍
The SSRF attack happens through a vulnerable web server running on the instance. For example, a web server running on the instance may accept GET requests which may get redirected as a request to the IMDS endpoint. The response would in turn return the IAM credentials. The attackers sure enjoy using these credentials.

‍
In most such cases the reverse proxies such as Apache HTTPd or Squid or Web Application Firewall (WAFs) are misconfigured. These misconfigurations expose the metadata service to the outside world. At RSA Conference, 2020, Steve Grobman describes an example of such an attack in quite a detail. Many times EC2 instances host the open routers, layer 3 firewalls, VPNs, tunnels or NAT devices. Many times this software is not properly configured. The IMDSv1 cannot stop the IP packets containing the response to crossing out of such EC2 instances. The IMDSv2 employs an ingenious trick. This trick prevents these packets from crossing out of the EC2 instance.

IMDSv2

The IMDSv1 requests are not authenticated and are susceptible to SSRF attacks.The newer version, IMDSv2 adds protections against SSRF. IMDSv2 also requires users to create and use session tokens.

The 2019 CapitalOne breach exploited IMDSv1 weakness. After this incident, AWS took corrective steps and introduced IMDSv2.

While establishing an IMDSv2 session, users can specify the session duration. The longest allowed session duration is of six hours. These session tokens are valid only for that specific EC2 instance.

The IMDSv2 session always starts with an HTTP PUT request. This is an important improvement over IMDSv1 as we will see shortly. The request also has a mandatory HTTP header: X-aws-ec2-metadata-token-ttl-seconds. This header specifies the maximum time (TTL) the session can be valid.

TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
curl http://169.254.169.254/latest/meta-data/profile -H "X-aws-ec2-metadata-token: $TOKEN"

How IMDSv2 improves security

Session initiation through PUT requests

The GET requests in IMDSv1 is one major reason for its security weakness. Instead, IMDSv2 chose the PUT request, as most WAF and reverse proxies do not support the PUT requests.

In addition, IMDSv2 requires the session to begin with a PUT request only. The session initiation request handles generating the token as seen earlier. Without a token, IMDSv2 access is not granted thus securing the instance metadata.

Blocking X-Forwarded-For header

Some reverse proxies may still allow PUT requests. Many of these proxies use the X-Forwarded-For header to pass the IP of the original caller. IMDSv2 rejects all requests set with this header. This helps in blocking all mischievous requests from the misconfigured open reverse proxy.

Ingenious IP Packet TTL reduction trick

Sometimes the EC2 instances work as open routers, NAT devices etc. This opens a communication path to the outside world. This communication happens over IP packets.

IMDSv2 sets the token response’s lower-level IP packet’s TTL value to 1. Each IP packet crosses the hardware or software layers. Every time an IP packet crosses such a layer, the layer subtracts the TTL time by 1. The software or hardware layer discards the IP packet if the TTL is 0. Thus the IMDSv2 instance metadata is safely returned to the caller application on the instance. But well before the metadata IP packet crosses the EC2 instance boundary the packet is dropped.Thus an attacker outside the EC2 instance is unable to get the access credentials.

What IMDS version to use?

It is almost certain that you should choose the IMDSv2 as the default version on your instances.

The IMDSv2 enforces restrictions on HTTP verbs or HTTP headers. In some cases, this would break your existing scripts or applications. In such cases, you may still want to continue to use IMDSv1.

For these reasons AWS still allows users to select the IMDSv1, even though the IMDSv2 is more secure. AWS has committed to supporting IMDSv1 indefinitely. So you now have a choice to completely disable IMDS or enable either IMDSv1 or IMDSv2.

By default, the IMDS is always ON. If you have not explicitly disabled it your EC2 instance would be running IMDSv1.

Identifying IMDSv1 instances

Using AWS CLI

Use AWS EC2 CLI describe-instances to pull the instance metadata for each instance. For IMDSv1, the instance metadata option HttpTokens is set to optional.

aws ec2 describe-instances 
--region us-east-1 
--instance-id i-0123456789abcdef 
--query "Reservations[0].Instances[0].MetadataOptions" 

Output:
{ 
	"State": "applied", 
	"HttpTokens": "optional", 
  "HttpPutResponseHopLimit": 1, 
	"HttpEndpoint": "enabled" 
 }

Using CloudWatch

You can use CloudWatch metric MetadataNoToken to track the number of calls to IMDSv1. The zero counts for this metric mean all your software is using IMDSv2. For more information, see Instance metrics.

Using Security Compliance Feature (Update: 14 March 2023)

CloudYali recently launched AWS Security Compliance check. We now support AWS Foundational Security Best Practices controls and

CIS Amazon Web Services Foundations Benchmark 1.4.0. The AWS Foundation Security Best Practices rule: [EC2.8] EC2 instances should use IMDSv2 now automatically detects EC2 instances which are not using IMDSv2 and report these EC2 instances as failures. You may directly visit the security dashboard and look at the list of failed resources for such instance ids.

Using CloudYali Resource Attribute Search

For a large cloud footprint that spans across many regions or/and many AWS accounts using AWS CLI can be difficult. CloudYali provides a simple way to identify all IMDSv1 instances in a single window.

To find all EC2 instances which have IMDSv1 use the below steps in the CloudYali console.

• Select the AWS accounts and regions into which you want to search. By default, the search would include all AWS accounts and regions.
• Select the resource type AWS::EC2::Instance from the resource type dropdown.
• Now select the Resource attributes dropdown. Selecting this dropdown would show you all the attributes for the EC2 instance to search.
• Select MetadaOptions.HttpTokens attribute and in the next input box type in "optional".
• Press enter to start the search. The result should list all EC2 instances with IMDSv1.

The below 20-second screen recording shows how CloudYali simplifies IMDSv1 instances search.

Disabling IMDSv1

Security-wise IMDSv1 is not a great choice. But, we live in an imperfect world. Many third-party libraries may have dependencies on IMDSv1. Make sure that your code, the libraries the code uses or the tech stack has no dependency on the IMDSv1. Disable IMDSv1 if no such dependencies are found. If you’re using the latest versions of AWS SDK to interact with AWS APIs, you should be good to move ahead.

We learned from a few of our customers, disabling IMDSv1 caused the software to fail. In some cases, the third-party vendors have not enabled their products for IMDSv1.

In such cases, you should keep IMDSv1 running until you update your code. We recommend that you add an “IMDSv1Exception=true” tag to the instance. The tags help you to document the weak spots in your infrastructure and address them with a proper plan. CloudYali Global Tag Search instantly find such tagged instances across your cloud.

Enabling IMDSv2

While launching your EC2 instances from the console select the option below.

If you’re using IaC such as terraform, use the recommended options.
‍

resource "aws_instance" "imdsv2_enabled_instance" {
  ami = "ami-00fe44dee72dc1de0"
  instance_type = "t2.micro"

metadata_options {
    http_endpoint = "enabled"
    http_tokens = "required"
  }
}

If you already have EC2 instances running in your environment, you may use AWS CLI.

aws ec2 modify-instance-metadata-options

Identifying IMDSv2 instances

Using AWS CLI

The AWS CLI describe-instances provides the metadata options information. The instance metadata HttpTokens is set to required for IMDSv2.
‍

aws ec2 describe-instances --region us-east-1 --instance-id i-0123456789abcdef --query "Reservations[0].Instances[0].MetadataOptions" 
Output:
{ 
	"State": "applied", 
	"HttpTokens": "required", 
	"HttpPutResponseHopLimit": 1, 
	"HttpEndpoint": "enabled" 
}

Using CloudYali Resource Attribute Search

For a large cloud footprint that spans across many regions or/and many AWS accounts using AWS CLI can be difficult. CloudYali provides a simple way to identify all IMDSv2 instances in a single window.

To find all EC2 instances which have IMDSv2 use the below steps in the CloudYali console.

• Select the AWS accounts and regions into which you want to search. By default, the search would include all AWS accounts and regions.
• Select the resource type AWS::EC2::Instance from the resource type dropdown.
• Now select the Resource attributes dropdown. Selecting this dropdown would show you all the attributes for the EC2 instance to search.
• Select MetadaOptions.HttpTokens attribute and in the next input box type in "required".
• Press enter to start the search. The result should list all EC2 instances with IMDSv2.

The below 20-second screen recording shows exactly how CloudYali simplifies IMDSv2 instances search.

Different ways to implement and enforce IMDSv2

Enforcing IMDSv2 with service control policies is the most natural way to ensure that all EC2 are following safe practices. Apart from using SCPs, other ways include, using AWS CLI, IaC tools such as Terraform or CloudFormation, and EC2 Launch Templates.

Implementing IMDSv2 with AWS CLI

All latest AWS CLI and AWS SDKs support IMDSv2 option natively. To launch EC2 instances with IMDSv2 you may use run-instances CLI or RunInstances API. You may refer to AWS doc for additional information.

aws ec2 run-instances \
    --image-id ami-0abcdef1234567890 \
    --instance-type c3.large \
	...
    --metadata-options "HttpEndpoint=enabled,HttpTokens=required"

For existing running instances you may use modify-instance-metadata-options CLI

aws ec2 modify-instance-metadata-options \
    --instance-id i-1234567898abcdef0 \
    --http-tokens required \
    --http-endpoint enabled

Implementing IMDSv2 with CloudFormation

The CloudFormation path however is not straightforward. A great blog post by Karim El-Melhaoui provide the finer details.

Implementing IMDSv2 with Terraform

The Terraform AWS provider v4.34.0 onward support aws_ami resource type with a new attribute imds_support for enabling IMDSv2.

resource "aws_ami" "imdsv2_ami" {
  name                = "terraform-imdsv2"
  virtualization_type = "hvm"
  root_device_name    = "/dev/xvda"
  imds_support        = "v2.0" # Enforce usage of IMDSv2. 
  ebs_block_device {
    device_name = "/dev/xvda"
    volume_size = 8
  }
}

Terraform resource type instance support attribute http_tokens for IMDSv2.

resource "aws_instance" "imdsv2_instance" {
        // ...
        metadata_options {
            http_endpoint = "enabled"
            http_tokens = "required"
        }
    }

Implementing IMDSv2 with SCP

Scott Piper's blog document the SCPs required for enforcing IMDSv2. These SCPs are documented by AWS as IAM policies. You can create these SCPs and attach to the AWS accounts where you want to enforce IMDSv2. Such SCP look like:

{
    "Version": "2012-10-17",
    "Statement": {
        "Sid": "RequireImdsV2",
        "Effect": "Deny",
        "Action": "ec2:RunInstances",
        "Resource": "arn:aws:ec2:*:*:instance/*",
        "Condition": {
            "StringNotEquals": {
                "ec2:MetadataHttpTokens": "required"
            }
        }
    }
}

To ensure no one is able to revert back to IMDSv1, you may use another SCP.

{
    "Version": "2012-10-17",
    "Statement": {
        "Effect": "Deny",
        "Action": "ec2:ModifyInstanceMetadataOptions",
        "Resource": "*"
    }
}

AWS credentials provided by the IMDS  include an ec2:RoleDelivery IAM context key. Credentials provided by the older IMDSv1 have an ec2:RoleDelivery value of “1.0,” and credentials using the new scheme have an ec2:RoleDelivery value of “2.0.”. Thus we can write another SCP, which enforces all resources to only allow access to role-account credentials that have the “2.0” value (or greater) for the context key.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "RequireAllEc2RolesToUseV2",
            "Effect": "Deny",
            "Action": "*",
            "Resource": "*",
            "Condition": {
                "NumericLessThan": {
                    "ec2:RoleDelivery": "2.0"
                }
            }
        }
    ]
}

We know IMDSv2 uses only one hop, and thus the last SCP ensures that no EC2 instance can be launched with IMDSv1.

{
    "Version": "2012-10-17",
    "Statement": {
        "Sid": "MaxImdsHopLimit",
        "Effect": "Deny",
        "Action": "ec2:RunInstances",
        "Resource": "arn:aws:ec2:*:*:instance/*",
        "Condition": {
            "NumericGreaterThan": {"ec2:MetadataHttpPutResponseHopLimit": "1"}
        }
    }
}

Should you disable IMDS completely?

Disable the IMDS if your application on EC2 doesn’t need instance metadata (at all). This would cut the active attack surface. Our advice is to perform due diligence before you take this action.

Disabling the IMDS is a reasonable step, but, you should be careful about disabling IMDS. Many libraries or AWS tools/Services such as CloudWatch or SSM need access to IMDS. The IMDS helps these tools/services to get the credentials. These tools/services use access credentials to access various AWS services. The attached instance role control the permissions. IMDS also provides other information such as Availability Zone or regions. Many applications use this information. Hence before you disable IMDS completely, do a proper due diligence. Note that all Amazon Linux 2 software packages now support IMDSv2 and thus if you're using Amazon Linux 2 it may be safe to transition to IMDSv2.

To completely disable IMDS use the below command line.

aws ec2 modify-instance-metadata-options 
–instance-id '' –http-endpoint disabled

‍

Remember:

• IMDS comes into picture when you start workloads on EC2 instances.
• EC2 instances can be launched via AWS Console, IaC, as part of Autoscaling Groups (ASG) and ECS.
• IMDSv1 is running by default on EC2 instances, AWS has no intention to change this behavior in near future.
• In most cases moving to IMDSv2 is a viable option. IMDSv2 is more secure option.
• Perform due dilligence before you either disable IMDS or move to IMDSv2.
• Check if you use any third party software which does not have IMDSv2 support. Refer to IMDSv2 Wall of Shame maintained by Scott Piper.
• Maintain AWS asset inventory.
• Run periodic checks on AWS asset inventory for IMDS disabled, IMDSv1 enabled and IMDSv2 enabled EC2 instances.
• Enforing IMDSv2 needs to be handled for multiple scenarios and in multiple ways.
  ‍

References:

• Transition to using Instance Metadata Service Version 2
• AWS Security Blog