AI Docs

Configuring AWS Data Sources

6 min read

image-20251112-142732.png

This guide outlines the process of configuring AWS data exports and connecting them to CerteroX Cloud Management, including specific instructions for root and linked accounts, and migrating from CUR to CUR 2.0.

Configuring Data Export

Before connecting your AWS account to CerteroX Cloud Management, you need to configure data exports in AWS.

Create Data Export

This step is primarily for Root accounts. If your Data Export is already configured, you can skip to …..

Navigate to AWS Billing & Cost Management → Data Exports.

  1. Create a new data export. You have two options:

  • Standard Data Export (Recommended for CUR 2.0):

  • Select "Standard data export" as the export type.

    1. Enter an "Export name".

    2. Under "Data table content settings":

      • Select "CUR 2.0".

      • Select the "Include resource IDs" checkbox.

      • Choose your desired time granularity.

    3. Under "Data export delivery options":

      • Select "Overwrite existing data export file".

      • Choose the compression type.

    4. Under "Data export storage setting":

      • Create a new S3 bucket or use an existing one.

      • Enter an "S3 path prefix".

    5. Confirm export creation. AWS will prepare the data export within 24 hours.

  • Legacy CUR Export (Older version):

    1. Select "Legacy CUR export (CUR)" as the export type.

    2. Enter an "Export name".

    3. Select "Include resource IDs" and "Refresh automatically" checkboxes.

    4. Under "Data export delivery options":

      • Choose your desired time granularity.

      • Select "Overwrite existing report".

      • Choose the compression type.

    5. Under "Data export storage setting":

      • Create a new S3 bucket or use an existing one.

      • Enter an "S3 path prefix".

    6. Confirm export creation. AWS will prepare the data export within 24 hours.

Connect to CerteroX Cloud Management

Configure Policies & User

  1. Configure Data Exports. If this hasn’t already been configured, refer to the

    Configuring AWS Data Sources | Create-Data-Export
    section.

  2. Update bucket policy:

    • Navigate to the Permissions tab of your AWS S3 bucket → select Bucket Policy.

    • Click + Add new statement → insert the following JSON code snippet:

      {
      "Version":"2012-10-17",
      "Statement":[
      {
      "Sid":"EnableAWSDataExportsToWriteToS3AndCheckPolicy",
      "Effect":"Allow",
      "Principal":{
      "Service":[
      "billingreports.amazonaws.com",
      "bcm-data-exports.amazonaws.com"
      ]
      },
      "Action":[
      "s3:PutObject",
      "s3:GetBucketPolicy"
      ],
      "Resource":[
      "arn:aws:s3:::<bucket_name>/*",
      "arn:aws:s3:::<bucket_name>"
      ],
      "Condition":{
      "StringLike":{
      "aws:SourceAccount":"<AWS account ID>",
      "aws:SourceArn":[
      "arn:aws:cur:us-east-1:<AWS account ID>:definition/*",
      "arn:aws:bcm-data-exports:us-east-1:<AWS account ID>:export/*"
      ]
      }
      }
      }
      ]
      }

    • Replace <bucket_name> with the name of your S3 bucket.

    • Replace <AWS account ID> with your 12-digit AWS Account ID (without hyphens).

    • Save.

  3. Configure Policies for Discover Resources and ReadOnly access.

    • ReadOnly Access:

      • Follow steps 1 to 5 of the instructions.

      • Insert the JSON code in the Type or paste a JSON policy document step.

      • {
        "Version":"2012-10-17",
        "Statement":[
        {
        "Sid":"ReportDefinition",
        "Effect":"Allow",
        "Action":[
        "cur:DescribeReportDefinitions"
        ],
        "Resource":"*"
        },
        {
        "Sid":"GetObject",
        "Effect":"Allow",
        "Action":[
        "s3:GetObject"
        ],
        "Resource":"arn:aws:s3:::<bucket_name>/*"
        },
        {
        "Sid":"BucketOperations",
        "Effect":"Allow",
        "Action":[
        "s3:ListBucket",
        "s3:GetBucketLocation"
        ],
        "Resource":"arn:aws:s3:::<bucket_name>"
        }
        ]
        }

      • Replace the <bucket_name> with the name of the bucket from the previous step.

    • Discover Resources:

      • Include the following policy to allow CerteroX Cloud Management to parse EC2 resource data. Follow steps 1 to 5 of the instructions.

      • Insert the JSON code on the Type or paste a JSON policy document step.

      • {
        "Version": "2012-10-17",
        "Statement": [
        {
        "Sid": "CerteroXOperations",
        "Effect": "Allow",
        "Action": [
        "s3:GetBucketPublicAccessBlock",
        "s3:GetBucketPolicyStatus",
        "s3:GetBucketTagging",
        "iam:GetAccessKeyLastUsed",
        "cloudwatch:GetMetricStatistics",
        "s3:GetBucketAcl",
        "ec2:Describe*",
        "s3:ListAllMyBuckets",
        "iam:ListUsers",
        "s3:GetBucketLocation",
        "iam:GetLoginProfile",
        "cur:DescribeReportDefinitions",
        "iam:ListAccessKeys",
        "elasticloadbalancing:Describe*"
        ],
        "Resource": "*"
        }
        ]
        }

  4. Create User and Grant policies.

    • Go to Identity and Access Management (IAM) → Users and create a new user.

    • In Step 2. Set Permissions, select Attach Policies directly & and attach the policies created earlier.

    • Confirm the creation of the user.

  5. Create Access Key.

    • Go to Identity and Access Management (IAM) → Users → Select the created user → create an access key.

    • Download the .csv file with the Access Key and Secret Access Key.

Connect to CerteroX - Data Export Already Configured

Once the user is configured, add the data source to CerteroX Cloud Management.

  1. Go to CerteroX Cloud Management → Settings → Data Sources.

  2. Click Actions → Add

  3. Select AWS as the Data Source and Root as Connection type.

    image-20250723-123253.png

  4. Fill in the fields:

    • using the access key generated in Step 5 of

      Configuring AWS Data Sources | Configure-Policies-&-User

    • Select Export type to match the export created in

      Configuring AWS Data Sources | Create-Data-Export

    • Switch off Automatically detect existing Data Exports

    • Select Connect only to data in bucket.

    • Enter the Data Export parameters

      • Export Name: AWS Billing and Cost Management → Data Exports table → Export Name column.

      • Export Amazon S3 Bucket Name: AWS Billing and Cost Management → Data Exports table → S3 bucket column.

      • Export path prefix: AWS Billing and Cost Management → Data Exports table → Click on Export name → Edit → Data export storage settings → S3 destination → last folder name. E.g. “certeroxbucket/costreport” enter costreport into the field.

  5. Press Connect.

Connect to CerteroX - Data export not configured

Once the user is configured, add the data source to CerteroX Cloud Management.

  1. Go to CerteroX Cloud Management → Settings → Data Sources.

  2. Click Actions → Add

  3. Select AWS as the Data Source and Root as Connection type.

    image-20250723-124046.png

  4. Fill in the fields:

    • using the access key generated in Step 5 of

      Configuring AWS Data Sources | Configure-Policies-&-User

    • Select Export type to match the export created in

      Configuring AWS Data Sources | Create-Data-Export

    • Switch off Automatically detect existing Data Exports

    • Select Create new Data Export.

    • Provide Data Export parameters

      • Export Name: Enter a new name for the data export

      • Export Amazon S3 Bucket Name: AWS Billing and Cost Management → Data Exports table → S3 bucket column.

      • Export path prefix: enter a new export path prefix that you want to prepend to the names of your report files.

  5. Press Connect.

Specify the existing bucket in Export S3 Bucket Name field. CerteroX Cloud Management will then create the report and store it in the bucket using the specified prefix.

 

Linked Data Sources

CerteroX Cloud Management supports the AWS Organizations service which allows linking several Data Sources in order to centrally manage data while receiving all billing exports within a single invoice.

Ensure that the Discover Resources policy outlined in

Configuring AWS Data Sources | Configure-Policies-&-User
has been set to allow CerteroX to parse EC2 resource data.

  1. Go to CerteroX Cloud Management → Settings → Data Sources.

  2. Click Actions → Add

  3. Select AWS as the Data Source and Linked as Connection type.

    image-20250723-124723.png

  4. Enter the access key generated in Step 5 of

    Configuring AWS Data Sources | Configure-Policies-&-User

  5. Press Connect.

This option removes the need to manually input bucket information, as the data will be received through the root account.
If only an AWS Linked account is specified, without connecting a Root account, CerteroX will not be able to import any billing data.

Migrating from CUR to CUR2.0

Configure AWS

In the AWS console:

  1. Navigate to Billing & Cost Management → Data Exports. Click Create.

  2. Select "Standard data export" as the export type.

    1. Enter an "Export name".

    2. Under "Data table content settings":

      • Select "CUR 2.0".

      • Select the "Include resource IDs" checkbox.

      • Choose your desired time granularity.

    3. Under "Data export delivery options":

      • Select "Overwrite existing data export file".

      • Choose the compression type.

    4. Under "Data export storage setting":

      • Create a new S3 bucket or use an existing one.

      • Enter an "S3 path prefix".

    5. Confirm export creation. AWS will prepare the data export within 24 hours.

Update CerteroX Connection

  1. Go to CerteroX Cloud Management → Settings → Data Sources.

  2. Click on the AWS data source to be updated.

image-20250723-125321.png

  1. Click Actions → Update Credentials.

image-20250723-125434.png

  1. Switch on Update Data Export parameters & select Standard data export (CUR 2.0) as the export type.

  2. Update the Export name, Export path prefix as in the updated bucket.

  3. Press Save.