CloudWatch Dashboard (Over)Sharing

To increase our visibility into the previous steps of the process (the creation and sharing of a dashboard) we need to set up a lab and add some dynamic analysis into the mix. In a sandbox account under our complete control, we created a barebones CloudWatch dashboard and proceeded to hit the "Share" button in the Console. Once again, a hard-to-miss callout was presented:

The callout informed us of the set of resources that would be created as a result of this process:

• A Cognito Identity Pool
• A Cognito User Pool
• A Cognito App Client
• An IAM role

This simple action and result was quite a breakthrough, as it provided the following 3 observations:

1. All these resources are created automatically behind the scenes, by Amazon CloudWatch. No user involvement or opportunities to insert potentially insecure configuration are available.
2. The IAM policy for the dashboard role, was never mentioned, which suggested that as above, it was a fixed set of permissions - those 4 mentioned by the documentation page , including EC2:DescribeTags.
3. The remaining 3 resources were all Cognito resources, confirming that all authentication used by the CloudWatch application is handled by Cognito, and Cognito alone.

Past this confirmation step and once the dashboard was eventually shared, the result was visible in the Console, along with yet another warning banner greeting us on every page visit, about the implications of sharing publicly:

Having complete visibility now into all solution components allows us to reflect on the implementation of the sharing feature, and particularly to revisit the mysterious context parameter.

Here's a reverse engineered listing of each field and its purpose:

This complete visibility now also allowed us to understand and draw out the complete viewer flow:

1. User visits a Dashboard link carrying the Dashboard Name and Context as URL parameters

2. The Dashboard application's client-side code is retrieved from Amazon CDN

3. The application obtains temporary credentials from Cognito, through the Enhanced authflow

CognitoIdentity:getId( identityPoolId )
CognitoIdentity:getCredentialsForIdentity( identityId )

4. An authenticated request retrieves the Dashboard's Manifest, defining the alarms and metrics resources

CloudWatch:getDashboard( dashboardName )

5. Finally, the data to populate each view is retrieved, through CloudWatch API calls to the respective resource regions

CloudWatch:describeAlarms( alarmNames,alarmTypes )
CloudWatch:getMetricData( defaults,metrics )
...

While these bits of insights are useful, they still didn't explain the issue at hand: Why was the Cognito-retrieved identity not granted the full set of permissions originally intended by the IAM policy?

After some more digging, we eventually bumped into an issue thread on the open-source, AWS

JavaScript SDK project, initiated by a user experiencing the same issue. After a few replies, the SDK maintainer provided a surprising resolution:

This was arguably an interesting observation: It appeared that behind the scenes, the Simplified flow was implemented to deliberately ignore admin-assigned IAM policies, and instead silently grant reduced permissions.

Now knowing where to look, we find out further justification of this design decision, on the linked documentation page:

"For additional security protection, Amazon Cognito applies a scope-down policy to credentials that you assign your unauthenticated users in the enhanced flow, using GetCredentialsForIdentity . The scope-down policy adds an Inline session policy and an AWS managed session policy to the IAM policies that you apply to your unauthenticated role."

This finally explains the original error thrown when trying to retrieve EC2 tags. If we revisit policies and permissions in AWS IAM, the effective set of permissions that are granted to any session are those in the intersection of Identity-based, Resource-based, and Session policies. In the context of dashboard sharing now: The dashboard’s IAM role is granted all 4 permissions (Identity-based policy), and the tags themselves do not restrict listing from that principal (Resource-based policy). However, the viewer’s browser is effectively only granted 3 of these 4 permissions. This is due solely to the Session policy defined by the client:

Hypothetically, if the application was using the Basic (Classic) Flow, no scope-down would be applied, and the intersection would include all 4 permissions defined in the auto-configured dashboard role policy:

It should be noted that using the Basic flow would be possible for the dashboard application. Besides contradicting Amazon's own recommended design patterns, this would require a call to Amazon STS by the client-side code, to retrieve temporary tokens directly via sts:AssumeRoleWithWebIdentity . This would require specifying the desired IAM role, which is however known to the application via the "O" field of the context parameter.

From an exploitation perspective, the question now becomes: Can an unauthenticated dashboard viewer initiate a Basic authflow against the Cognito resources?

Armed with knowledge of the internals, the path needed to carry out this attack was now

straightforward. For an attacker in possession of only a shared dashboard URL, the steps to carry out a Basic authflow against the dashboard's Cognito identity pool and access the account's EC2 tags were the following:

1. Attacker extracts the following from the Context

• IAM role ARN
• Identity Pool ID

2. Attacker acquires an Identity from the Cognito Identity pool, as before 

cognitoIdentity:getId( identityPoolId )

3. Attacker requests an OpenID Connect (OIDC) token for this identity

cognitoIdentity:getOpenIdToken( identityId )

4. Attacker trades the OIDC token for temporary credentials of the target IAM role

sts:assumeRoleWithWebIdentity( dashboardRole, oidcToken )

5. Finally, attacker proceeds to leverage the role's permissions by reading EC2 tags

ec2:describeTags()

Time to put this theory to the test. Setting aim at the sandbox Cognito resources within our WithSecure lab account, we proceed to issue the commands:

1. Retrieving a Cognito Identity using the Identity Pool ID within the context parameter (field "I") 

2. Obtaining an OpenID Token for this identity 

3. Exchanging this token for AWS credentials, specifying the IAM role worked out from the "O" field of the context parameter  

4. Loading these credentials for the final PoC command... et voilà!

And that confirms the exploitation scenario end-to-end: We went from dashboard viewers, to listing EC2 tags. All by just knowing a URL, and URLs like this are not hard to find...

But how common is this Dashboard Sharing functionality really? And which sharing method was the most prevalent? Answers were easily given using Google dorks. For example, the following query filtered out dashboards with SSO authentication:

inurl:https://cloudwatch.amazonaws.com/dashboard.html -corporate 

This revealed no more than 20 or so results:

A few more unique dashboards were found in URLscan:

Clicking through a few of them indeed revealed dashboards of all sorts, with real time data flowing throughout the widgets:

Without, of course going all the way to list EC2 tags in those third party accounts, a few observations can be made simply by looking at those results:

• The naming format of all resources mentioned within the context parameter was consistent, including the IAM role ARN (field "O"): "CWDBSharing-PublicReadOnlyAccess-[0-9A-Z](8)"
• Alarmingly, the number of dashboards shared publicly were far more than the sum of those shared with SSO authentication and those requiring username + password... Once again, users are shown to choose the easy way instead of the secure alternative. Not so surprising.

Listing EC2 tags is arguably not the end of the world. Amazon's guidance clearly advises against putting sensitive data in tags. However, in our assessments we have found in there sensitive information of various forms, such as PII, owner contact details and sometimes even credentials.  

Looking for ways this could cause more serious problems, we first explored potential scenarios where the IAM role of the dashboard would be more privileged than these use cases we've already seen. As previously established, these IAM permissions were fixed and configured by Amazon code, so it was unlikely that users would manually modify "canned resources" that "just work". Unless they are after more features!

Further reading in the docs indeed revealed two scenarios where the dashboard's IAM role would need to be manually granted extra permissions by the user for certain functionality to work:

1. Logs Table Widgets - resulting in the addition of a few more, (hopefully) resource-scoped, CloudWatch permissions

logs:FilterLogEvents
logs:StartQuery
logs:StopQuery
logs:GetLogRecord
logs:DescribeLogGroups

2. Custom Widgets – leading to the juicier, lambda:InvokeFunction permissions

Regarding the former scenario, we read in this blog post from 2020, that these logs: permissions used to be included in the dashboard's IAM role policy by default, and were only later made optional. Another note here is that performing logs: actions on behalf of the dashboard role does not require the Basic authflow sequence we demonstrated above. The "scope-down" Session Policy includes these additional permissions, therefore the credentials obtained by the browser itself through the Simplified authflow would also do the trick.

As for the latter scenario, we finally had a truly impactful outcome in the cards: conditional, but unauthenticated Lambda function execution. In other words, viewers of a dashboard with custom widgets can programmatically invoke the associated function.  An attacker could use this to run up the victim’s AWS bill, also known as a Denial of Wallet attack. Through this Lambda backend, a custom widget can also allow execution of technically any AWS API, through the cwdb-action, however once again, this would require further configuration.

Finally, we tried a few different ways to take this attack further, by attempting IAM attacks such as:

• specifying a more privileged policy on the sts:AssumeRoleWithWebIdentity step of the programmatic Basic authflow 
• assuming a different IAM role altogether,
• going cross-account, and trying to assume the dashboard's role from a foreign Cognito Identity in the Simplified authflow's CognitoIdentity:getCredentialsForIdentity - inspired by Nick Frichette's similar research against AWS Amplify

…All without success. This last one in particular, was prevented here by the additional Condition added automatically to the dashboard role's Trust Policy, restricting assumption to a single Cognito Identity universally:

While reading up on typical attacks against Cognito resources in those previous attempts to escalate further, we came to a realisation: this vector could have been prevented. It is possible to block Basic authflow on a Cognito Identity Pool altogether, and indeed our dashboard sharing one had it blocked, as seen in the Console:

Or had it? If we look closely at the image above, we see the value of the relevant field is in fact a hyphen! Not a clear-cut "No" or "Inactive" - more like a "N/A". Could a Boolean flag have three possible values instead of two? Querying the resource programmatically confirms our hypothesis:

Notice how the AllowClassicFlow field is missing from the JSON object…

What we're seeing is a vulnerability known as "Fail Open" behaviour. In effect, a system insecurely skipping its controls once faced with undefined conditions. In our case, if users would omit to explicitly set the AllowClassicFlow flag, it would be left in an uninitialised state, which effectively translated to True , subsequently allowing OpenID tokens to be retrieved via the Basic (classic) authflow. Through communications with AWS, it was revealed that this behaviour was historically chosen for backwards compatibility, when the Enhanced flow was introduced, to avoid breaking customers' exisiting solutions when creating new identity pools.   

Note that the same "undefined" setting was seen in Identity pools created when dashboards were shared with username & password authentication. In fact, closer inspection of the sharing process (with both authentication options - username & password or public sharing) revealed that the API calls to create the Cognito resources were taking place client-side, in the browser. And this included the vulnerable Cognito Identity Pool. Opening the inspector allowed us to observe the CreateIdentityPool operation carried out by the Console's JavaScript SDKs, and confirmed our finding - the AllowClassicFlow field was not set:

At that point, we had finally solved the riddle of the EC2 tag exposure: The combined effect of a misconfigured Cognito Identity pool created by the Console SDK, and the undefined behaviour of a Boolean attribute when creating such resources, resulted in Basic authflow being unintentionally allowed, and thereby providing viewers of a CloudWatch dashboard with access to EC2 tags within the affected account.

For completeness, before raising the findings with Amazon, we cross-checked for this undefined behaviour across the different methods users deploy cloud resources with. The most common ones, at least:

• the Web Console
• the SDKs
• the AWS CLI 
• Terraform

Note that this aimed at understanding how these methods instantiated Cognito Identity Pools, and particularly, whether they set the AllowClassicAuth flag or not. Not how they shared dashboards.

Sharing a dashboard was - and remains - only possible via the Console, despite a Reddit reply by a CloudWatch developer around the feature release date, which suggested that programmatic dashboard sharing would be implemented in the future:

Starting from the Console, we went ahead and created a new identity pool from the web UI, walking through the multi-step process. As seen below in the second to last step, we are asked explicitly whether to enable Basic (Classic) auth or not:

Interestingly, leaving this field empty here, securely defaults to a False setting:

And this indeed results in the expected error upon attempting to retrieve OpenID tokens:

Moving over to the CLI. Behind the scenes, the AWS CLI uses the Python boto3 library which is the official Python SDK, and both these technologies generate class definitions on the fly, according to the latest upstream API version. The version at the time of testing also demonstrated the behaviour observed within the dashboard sharing process: Creating a Cognito Identity Pool from the CLI did not require the --allow-classic-auth-flow flag to be set, and resulted in the field to be uninitialised in the resulting object:

This issue had been raised before - more than a year earlier, in February 2023 – where the last human comment mentions the expected behaviour. After no response or resolution however, the ticket was closed due to inactivity.

And finally, Terraform. As per the documentation, the allow_classic_flow argument although optional, safely defaults to False when defining an aws_cognito_identity_pool resource: 

In summary, the default behaviour for each client is provided below:

So far we have focused only on the least secure but most common option of public sharing. What about the other forms available? Are dashboards shared with username and password, or SSO authentication, similarly allowing EC2 tag exposure (and potentially logs actions, or Lambda function invocations)?

Username & Password Sharing

Looking at username and password sharing first, we see a similar Warning dialog when sharing a dashboard to an email address:

The generated dashboard URL and encoded context parameter looks no different, except the “M” field:

Trying to reproduce the attack by initiating a Basic authflow via the AWS CLI, using just the Identity Pool ID expectedly fails, given the Cognito Identity Pool no longer supports Unauthenticated access:

$ aws cognito-identity get-id --identity-pool-id "us-east-1:b7aa...-4f04a4b6a6ff" --region us-east-1
An error occurred (NotAuthorizedException) when calling the GetId operation:
Unauthenticated access is not supported for this identity pool.

But even if we skip that step by somehow obtaining an Identity ID, then we still can't retrieve a token via Basic authflow, that would allow us to set a permissive session policy and list tags:

$ aws cognito-identity get-open-id-token --identity-id "us-east-1:3b5...-b707-06ce515fb598" --region us-east-1
An error occurred (InvalidParameterException) when calling the GetOpenIdToken operation: Basic (classic) flow is not supported with RoleMappings, please use enhanced flow.

Awesome, unauthenticated third parties can no longer list EC2 tags. But what about intended viewers?

While Basic authflow is disabled, it is no longer needed, as the Console's frontend code this time explicitly specifies the service-role's IAM Policy in the CognitoIdentity:GetCredentialsForIdentity API call, avoiding the "scope-down" policy of the Simplified authflow. That means that the temporary credentials retrieved by the browser are enough to list the EC2 tags.

SSO Sharing

And what about SSO sharing? In our brief hunt, we found this method to be the next most common after that of public sharing. This scenario is applicable to multi-account AWS Organizations, and leverages the IAM Identity Center service to expose the shared dashboard as an “Application” to assigned users or groups:

Following the guides we eventually managed to set up a lab environment to trial this out. Starting from the dashboard URL as always, we notice little difference to all previous sharing methods:

Once again, initiating a Basic authflow against the Identity Pool is not possible and neither is obtaining an Identity from an unauthenticated perspective via get-id. However, using the temporary credentials retrieved by the browser as above, we can still list EC2 tags and conditionally invoke Lambda functions or retrieve CloudWatch Logs. Investigating why this is possible here, points us to the id_token retrieved in a previous step, by the SAML provider the administrator configured for SSO. 

The role ARN specified in this token is passed in the CognitoIdentity:GetCredentialsForIdentity API call within the Logins parameter, and is effectively the provider’s “suggestion” to Cognito of the preferred IAM role that the requesting user must be granted. This communication between the different components is depicted across steps 1 and 2 in the flow diagram attached to the guide linked above.

And how was this role set in first place? Documentation states that "Amazon Cognito identifies the group with the lowest precedence value in a user’s set of groups, and writes the IAM role ARN associated with that group to the preferred_role claim". And indeed, this @gmail.com user that the dashboard was shared to, was automatically added to a group, with that particular role configured: 

It is worth noting that all of this configuration was done automatically by the Console, when the dashboard was shared. The API calls made by the JavaScript code proceeded to create, or modify the existing Cognito resources, and set this role mapping - all without any administrator interaction. 

So summarising the exposure for all sharing methods:

It is worth noting that we tested for other typical Cognito attacks applicable for this type of sharing namely Self-Registration, and User Attribute modifications. However none resulted in meaningful exploitation scenarios.

Unsharing Dashboards

Before Amazon fixed the issue by blocking Basic authflow (detailed in the next section), we took a moment to investigate whether “unsharing” a dashboard fully mitigated the exposure. 

Our testing indicated that unsharing a dashboard will remove some, but not necessarily all of the relevant Cognito resources. The lifecycle of each resource type differs according to the state of the environment. For example, every time a new dashboard is shared, a new Identity Pool is created. However, a single User Pool persists any dashboard shared in any way. The progression of all related resources between the different states is depicted in a Gantt chart below, also covering the dashboard IAM role – the only non-Cognito resource involved:

The issues in the AWS Console and the Cognito service, along with their resulting impact on user resources, were confidentially disclosed to AWS in late July 2024, through their vulnerability reporting program. The security team was swift in their responses and in early September, we were notified that a fix had been rolled out on August 28, 2024. This fix disabled basic flow and ensured that calls to ec2:DescribeTags were no longer permitted in a shared dashboard. This resulted in EC2 instance names not being shown in dashboard widgets.

Our retesting confirmed that the Console API calls creating the new Cognito Identity Pools upon sharing of a dashboard, were now indeed setting the AllowClassicFlow flag explicitly to False. 

This was the case for all sharing methods, including Username & Password and SSO.

The detailed timeline of the disclosure is provided below:

Overall, we saw that sharing CloudWatch dashboards provides viewers with some permissions over the source account. In the most common sharing method observed on the internet - that of no authentication - a now-fixed bug allowed viewers to enumerate the EC2 tags within the source account. Should additional functionality be enabled, and customers have manually modified IAM policies to grant the additional permissions, then viewers could also invoke Lambda functions and retrieve CloudWatch logs. Although unauthenticated viewers can no longer perform these actions without further knowledge of identifiers within the account, intended viewers are still granted the respective permissions within the account. Carrying out these actions is therefore trivial, simply by extracting the credentials obtained on their behalf by the web browser. 

To protect their environments from external threats, customers using shared dashboards should review whether this functionality - and any additional features such as custom widgets - are required for their operations, given the risk posed. 

Administrators should also review which entities are granted the following permission "bundle", as it can allow them to share Dashboards and introduce the risks described in this post. 

• iam:CreateRole
• iam:CreatePolicy
• iam:AttachRolePolicy
• iam:PassRole
• cognito-idp:*
• cognito-identity:*

Finally, when modifying manually the IAM policy created by the Console to use these additional dashboard features, care should be taken to limit the resources in scope for these additional permissions – avoiding wildcard operators ("*").