W


ould you leave your Strongbox open just because you locked the house door?” - “No, right?”.
IMDS is like a strongbox containing precious information about your cloud. It is important that you understand IMDS, and how to secure it. It is time you take Instance metadata service (IMDS) seriously.

What is EC2 Instance Metadata Service (IMDS)?

The Instance Metadata Service (IMDS) helps code on EC2 instance access instance metadata. IMDS provides a great amount of information about instances. This includes hostname, security group, MAC address and much more. It also hosts user-data, that you specified when launching your instance. For an attacker, this is a gold mine. You should understand the security implications of instance metadata service. This would help you to better protect your cloud.

The IMDS exposes this instance metadata through a special “link-local” IP address of 169.254.169.254. Hence only code running on the instance would be able to access the metadata. Securing IMDS becomes critical due to its ability to access the AWS access credentials for the IAM role attached to the instance.

AWS Credentials on the EC2 instance

The applications running on EC instances must sign their requests with AWS credentials. One way is to provide the credentials in the form of AWS access keys/secrets. For various reasons, this is not a recommended approach.

It is cumbersome to distribute/rotate the access credentials on each instance. Even more challenging is to distribute and rotate the AWS created access credentials to the instances. eg. Spot instances or auto-scaling groups (ASGs).

Instead, you can attach an IAM role to the EC2 instance. The advantage is that you as an application developer don’t need to bother distributing and rotating the access credentials. The IAM role helps to enforce the least privilege principle, as another advantage.

The resource-level permissions in IAM policies help you decide on a granular level what IAM roles users can attach, detach or replace for an instance. In general, using IAM improves the cloud security posture.

What is an IAM InstanceProfile?

To pass an IAM role to an EC2 instance you need to use the instance profile. You can think of an instance profile as a container for the IAM role. You can attach or detach The IAM role to the instance profile. The instance profile itself can be attached or detached from the EC2 instance.

The question arises what is the difference between IAM role and instance profile. Why do we need a container around the IAM role?

Only IAM principals/identities such as users, groups, AWS services etc can assume the IAM role. The users, group, AWS services represent who assumed the role. The EC2 instance itself cannot assume a role. Hence AWS created Instance Profile, as principal to assume the role for an EC2 instance. The only thing the instance profile does is to assume the role attached to it. Beyond that instance, the profile has no other real power. The instance profile derives all its power from the assumed IAM role.

Retrieving IAM role credentials from IMDS

To retrieve the security credentials:

All these calls work without any authentication. It’s like keeping the strongbox open thinking no outsider can get in your house.

What makes IMDSv1 risky?

The instance metaservice is accessible from the EC2 instance only.  Once you have access to an EC2 instance you wouldn’t need any further authentication. You have already proved the ownership and no extra authentication is thus needed.
In theory, IMDS metadata will not be available outside the instance. If the EC2 instance is open to the outside world, attackers have a chance to steal credentials. Attackers use Server-Side Request Forgery (SSRF), XML External Entity Injection vulnerabilities to steal. Even shell access to the instance may expose access credentials.
The SSRF attack happens through a vulnerable web server running on the instance. For example, a web server running on the instance may accept GET requests which may get redirected as a request to the IMDS endpoint. The response would in turn return the IAM credentials. The attackers sure enjoy using these credentials.
In most such cases the reverse proxies such as Apache HTTPd or Squid or Web Application Firewall (WAFs) are misconfigured. These misconfigurations expose the metadata service to the outside world. At RSA Conference, 2020, Steve Grobman describes an example of such an attack in quite a detail. Many times EC2 instances host the open routers, layer 3 firewalls, VPNs, tunnels or NAT devices. Many times this software is not properly configured. The IMDSv1 cannot stop the IP packets containing the response to crossing out of such EC2 instances.  The IMDSv2 employs an ingenious trick. This trick prevents these packets from crossing out of the EC2 instance.

IMDSv2

The IMDSv1 requests are not authenticated and are susceptible to SSRF attacks. The newer version, IMDSv2 adds protections against SSRF. IMDSv2 also requires users to create and use session tokens.

The 2019 CapitalOne breach exploited IMDSv1 weakness. After this incident, AWS took corrective steps and introduced IMDSv2.

While establishing an IMDSv2 session, users can specify the session duration. The longest allowed session duration is of six hours. These session tokens are valid only for that specific EC2 instance.

The IMDSv2 session always starts with an HTTP PUT request. This is an important improvement over IMDSv1 as we will see shortly. The request also has a mandatory HTTP header: X-aws-ec2-metadata-token-ttl-seconds. This header specifies the maximum time (TTL) the session can be valid.


TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
curl http://169.254.169.254/latest/meta-data/profile -H "X-aws-ec2-metadata-token: $TOKEN"

IMDSv2 improves security

Session initiation through PUT requests

The GET requests in IMDSv1 is one major reason for its security weakness. Instead, IMDSv2 chose the PUT request as most WAF and reverse proxies do not support the PUT requests.

Besides, IMDSv2 requires the session to begin with a PUT request only. The session initiation request handles generating the token as seen earlier. Without a token, IMDSv2 access is not granted thus securing the instance metadata.

Blocking X-Forwarded-For header

Some reverse proxies may still allow PUT requests. Many of these proxies use the X-Forwarded-For header to pass the IP of the original caller. IMDSv2 rejects all requests set with this header. This helps in blocking all mischievous requests from the misconfigured open reverse proxy.

Ingenious IP Packet TTL reduction trick

Sometimes the EC2 instances work as open routers, NAT devices etc. This opens a communication path to the outside world. This communication happens over IP packets. IMDSv2 sets the token response’s lower-level IP packet’s TTL value to 1. Each IP packet crosses the hardware or software layers. Every time an IP packet crosses such a layer, the layer subtracts the TTL time by 1. The software or hardware layer discards the IP packet if the TTL is 0. Thus the IMDSv2 instance metadata is safely returned to the caller application on the instance. But well before the metadata IP packet crosses the EC2 instance boundary the packet is dropped. Thus an attacker outside the EC2 instance is unable to get the access credentials.

What IMDS version to use?

It is almost certain that you should choose the IMDSv2 as the default version on your instances.

The IMDSv2 enforces restrictions on HTTP verbs or HTTP headers. In some cases, this would break your existing scripts or applications.  In such cases, you may still want to continue to use IMDSv1.

For these reasons AWS still allows users to select the IMDSv1, even though the IMDSv2 is more secure. AWS has committed to supporting IMDSv1 indefinitely. So you now have a choice to completely disable IMDS or enable either IMDSv1 or IMDSv2.

By default, the IMDS is always ON. If you have not explicitly disabled it your EC2 instance would be running IMDSv1.

Identifying IMDSv1 instances

Using AWS CLI

Use AWS EC2 CLI describe-instances to pull the instance metadata for each instance. For IMDSv1, the instance metadata option HttpTokens is set to optional.


aws ec2 describe-instances 
--region us-east-1 
--instance-id i-0123456789abcdef 
--query "Reservations[0].Instances[0].MetadataOptions" 

Output:
{ 
	"State": "applied", 
	"HttpTokens": "optional", 
  "HttpPutResponseHopLimit": 1, 
	"HttpEndpoint": "enabled" 
 }

Using CloudWatch

You can use CloudWatch metric MetadataNoToken to track the number of calls to IMDSv1. The zero counts for this metric mean all your software is using IMDSv2. For more information, see Instance metrics.

Using CloudYali Resource Attribute Search

For a large cloud footprint that spans across many regions or/and many AWS accounts using AWS CLI can be difficult. CloudYali provides a simple way to identify all IMDSv1 instances in a single window.

CloudYali Resource Attribute Search help find IMDSv1 Instances
IMDSv1 Instance Search with CloudYali Resource Attribute Search

To find all EC2 instances which have IMDSv1 use the below steps in the CloudYali console.

  • Select the AWS accounts and regions into which you want to search. By default, the search would include all AWS accounts and regions.
  • Select the resource type AWS::EC2::Instance from the resource type dropdown.
  • Now select the Resource attributes dropdown. Selecting this dropdown would show you all the attributes for the EC2 instance to search.
  • Select MetadaOptions.HttpTokens attribute and in the next input box type in "optional".
  • Press enter to start the search. The result should list all EC2 instances with IMDSv1.

The below 20-second screen recording shows how CloudYali simplifies IMDSv1 instances search.

Disabling IMDSv1

Security-wise IMDSv1 is not a great choice. But, we live in an imperfect world. Many third-party libraries may have dependencies on IMDSv1. Make sure that your code, the libraries the code uses or the tech stack has no dependency on the IMDSv1. Disable IMDSv1 if no such dependencies are found. If you’re using the latest versions of AWS SDK to interact with AWS APIs, you should be good to move ahead.

We learned from a few of our customers, disabling IMDSv1 caused the software to fail. In some cases, the third-party vendors have not enabled their products for IMDSv1.

In such cases, you should keep IMDSv1 running until you update your code. We recommend that you add an “IMDSv1Exception=true” tag to the instance. The tags help you to document the weak spots in your infrastructure and address them with a proper plan. CloudYali Global Tag Search instantly find such tagged instances across your cloud.

Enabling IMDSv2

While launching your EC2 instances from the console select the option below.

If you’re using IaC such as terraform, use the recommended options.


resource "aws_instance" "imdsv2_enabled_instance" {
  ami = "ami-00fe44dee72dc1de0"
  instance_type = "t2.micro"

metadata_options {
    http_endpoint = "enabled"
    http_tokens = "required"
  }
}

If you already have EC2 instances running in your environment, you may use AWS CLI.


aws ec2 modify-instance-metadata-options

Identifying IMDSv2 instances

Using AWS CLI

The AWS CLI describe-instances provides the metadata options information. The instance metadata HttpTokens is set to required for IMDSv2.


aws ec2 describe-instances --region us-east-1 --instance-id i-0123456789abcdef --query "Reservations[0].Instances[0].MetadataOptions" 
Output:
{ 
	"State": "applied", 
	"HttpTokens": "required", 
	"HttpPutResponseHopLimit": 1, 
	"HttpEndpoint": "enabled" 
}

Using CloudYali Resource Attribute Search

For a large cloud footprint that spans across many regions or/and many AWS accounts using AWS CLI can be difficult. CloudYali provides a simple way to identify all IMDSv2 instances in a single window.

CloudYali Resource Attribute Search help find IMDSv2 Instances
IMDSv2 Instances with CloudYali Attribute Search

To find all EC2 instances which have IMDSv2 use the below steps in the CloudYali console.

  • Select the AWS accounts and regions into which you want to search. By default, the search would include all AWS accounts and regions.
  • Select the resource type AWS::EC2::Instance from the resource type dropdown.
  • Now select the Resource attributes dropdown. Selecting this dropdown would show you all the attributes for the EC2 instance to search.
  • Select MetadaOptions.HttpTokens attribute and in the next input box type in "required".
  • Press enter to start the search. The result should list all EC2 instances with IMDSv2.

The below 20-second screen recording shows exactly how CloudYali simplifies IMDSv2 instances search.

Enforcing IMDSv2

It is possible to enforce EC2 instances to use IMDSv2 by various means. It includes enforcing with SCP, IaC or EC2 launch template. A great blog post by Karim El-Melhaoui details various such possibilities.

Should you disable IMDS completely?

Disable the IMDS if your application on EC2 doesn’t need instance metadata (at all). This would cut the active attack surface. Our advice is to perform due diligence before you take this action.

Disabling the IMDS is a reasonable step, but, you should be careful about disabling IMDS. Many libraries or AWS tools/Services such as CloudWatch or SSM need access to IMDS. The IMDS helps these tools/services to get the credentials. These tools/services use access credentials to access various AWS services. The attached instance role control the permissions. IMDS also provides other information such as Availability Zone or regions. Many applications use this information. Hence you should not disable IMDS completely, before due diligence.

To completely disable IMDS use the below command line.


aws ec2 modify-instance-metadata-options 
–instance-id '' –http-endpoint disabled

Remember:

  • IMDS comes into picture when you start workloads on EC2 instances.
  • EC2 instances can be launched via AWS Console, IaC, as part of Autoscaling Groups (ASG) and ECS.
  • IMDSv1 is running by default on EC2 instances, AWS has no intention to change this behavior in near future.
  • In most cases moving to IMDSv2 is a viable option. IMDSv2 is more secure option.
  • Perform due dilligence before you either disable IMDS or move to IMDSv2.
  • Check if you use any third party software which does not have IMDSv2 support. Refer to IMDSv2 Wall of Shame maintained by Scott Piper.
  • Maintain AWS asset inventory.
  • Run periodic checks on AWS asset inventory for IMDS disabled, IMDSv1 enabled and IMDSv2 enabled EC2 instances.
  • Enforing IMDSv2 needs to be handled for multiple scenarios and in multiple ways.

References:

More Reads

Join Our Newsletter and Get the Latest
Posts to Your Inbox

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.