AI Docs

Configuring AWS Data Sources

3 min read

This guide outlines the process of configuring AWS data exports and connecting them to CerteroX Cloud Management, including specific instructions for root and linked accounts, and migrating from CUR to CUR 2.0.

Configuring Data Export

Before connecting your AWS account to CerteroX Cloud Management, you need to configure data exports in AWS.

Create Data Export

This step is primarily for Root accounts. If your Data Export is already configured, you can skip to the Connect to CerteroX section.

Navigate to AWS Billing & Cost Management → Data Exports.

  1. Create a new data export. You have two options:

  • Standard Data Export (Recommended for CUR 2.0):

  • Select "Standard data export" as the export type.

    1. Enter an "Export name".

    2. Under "Data table content settings":

    3. Under "Data export delivery options":

    4. Under "Data export storage setting":

    5. Confirm export creation. AWS will prepare the data export within 24 hours.

  • Legacy CUR Export (Older version):

    1. Select "Legacy CUR export (CUR)" as the export type.

    2. Enter an "Export name".

    3. Select "Include resource IDs" and "Refresh automatically" checkboxes.

    4. Under "Data export delivery options":

    5. Under "Data export storage setting":

    6. Confirm export creation. AWS will prepare the data export within 24 hours.

Connect to CerteroX Cloud Management

Configure Policies & User

  1. Configure Data Exports. If this hasn't already been configured, refer to the Create Data Export section.

  2. Update bucket policy:

    • Navigate to the Permissions tab of your AWS S3 bucket → select Bucket Policy.

    • Click + Add new statement → insert the following JSON code snippet:

      {
      "Version":"2012-10-17",
      "Statement":[
      {
      "Sid":"EnableAWSDataExportsToWriteToS3AndCheckPolicy",
      "Effect":"Allow",
      "Principal":{
      "Service":[
      "billingreports.amazonaws.com",
      "bcm-data-exports.amazonaws.com"
      ]
      },
      "Action":[
      "s3:PutObject",
      "s3:GetBucketPolicy"
      ],
      "Resource":[
      "arn:aws:s3:::<bucket_name>/*",
      "arn:aws:s3:::<bucket_name>"
      ],
      "Condition":{
      "StringLike":{
      "aws:SourceAccount":"<AWS account ID>",
      "aws:SourceArn":[
      "arn:aws:cur:us-east-1:<AWS account ID>:definition/*",
      "arn:aws:bcm-data-exports:us-east-1:<AWS account ID>:export/*"
      ]
      }
      }
      }
      ]
      }

    • Replace <bucket_name> with the name of your S3 bucket.

    • Replace <AWS account ID> with your 12-digit AWS Account ID (without hyphens).

    • Save.

  3. Configure Policies for Discover Resources and ReadOnly access.

    • ReadOnly Access:

      • {
        "Version":"2012-10-17",
        "Statement":[
        {
        "Sid":"ReportDefinition",
        "Effect":"Allow",
        "Action":[
        "cur:DescribeReportDefinitions",
        "bcm-data-exports:ListExports",
        "bcm-data-exports:GetExport",
        "bcm-data-exports:CreateExport"
        ],
        "Resource":"*"
        },
        {
        "Sid":"GetObject",
        "Effect":"Allow",
        "Action":[
        "s3:GetObject"
        ],
        "Resource":"arn:aws:s3:::<bucket_name>/*"
        },
        {
        "Sid":"BucketOperations",
        "Effect":"Allow",
        "Action":[
        "s3:ListBucket",
        "s3:GetBucketLocation"
        ],
        "Resource":"arn:aws:s3:::<bucket_name>"
        }
        ]
        }

    • Discover Resources:

      • {
        "Version": "2012-10-17",
        "Statement": [
        {
        "Sid": "CerteroXOperations",
        "Effect": "Allow",
        "Action": [
        "s3:GetBucketPublicAccessBlock",
        "s3:GetBucketPolicyStatus",
        "s3:GetBucketTagging",
        "iam:GetAccessKeyLastUsed",
        "cloudwatch:GetMetricStatistics",
        "s3:GetBucketAcl",
        "ec2:Describe*",
        "s3:ListAllMyBuckets",
        "iam:ListUsers",
        "s3:GetBucketLocation",
        "iam:GetLoginProfile",
        "cur:DescribeReportDefinitions",
        "bcm-data-exports:ListExports",
        "bcm-data-exports:GetExport",
        "bcm-data-exports:CreateExport",
        "iam:ListAccessKeys",
        "elasticloadbalancing:Describe*"
        ],
        "Resource": "*"
        }
        ]
        }

  4. Create User and Grant policies.

    • Go to Identity and Access Management (IAM) → Users and create a new user.

    • In Step 2. Set Permissions, select Attach Policies directly & and attach the policies created earlier.

    • Confirm the creation of the user.

  5. Create Access Key.

    • Go to Identity and Access Management (IAM) → Users → Select the created user → create an access key.

    • Download the .csv file with the Access Key and Secret Access Key.

Connect to CerteroX - Data Export Already Configured

Once the user is configured, add the data source to CerteroX Cloud Management.

  1. Go to CerteroX Cloud Management → Settings → Data Sources.

  2. Click Actions → Add

  3. Select AWS as the Data Source and Root as Connection type.

  4. Fill in the fields:

    • using the access key generated in Step 5 of Configure Policies & User.

    • Select Export type to match the export created in Create Data Export.

    • Switch off Automatically detect existing Data Exports

    • Select Connect only to data in bucket.

    • Enter the Data Export parameters

  5. Press Connect.

Connect to CerteroX - Data export not configured

Once the user is configured, add the data source to CerteroX Cloud Management.

  1. Go to CerteroX Cloud Management → Settings → Data Sources.

  2. Click Actions → Add

  3. Select AWS as the Data Source and Root as Connection type.

  4. Fill in the fields:

    • using the access key generated in Step 5 of Configure Policies & User.

    • Select Export type to match the export created in Create Data Export.

    • Switch off Automatically detect existing Data Exports

    • Select Create new Data Export.

    • Provide Data Export parameters

  5. Press Connect.

Specify the existing bucket in Export S3 Bucket Name field. CerteroX Cloud Management will then create the report and store it in the bucket using the specified prefix.

Linked Data Sources

CerteroX Cloud Management supports the AWS Organizations service which allows linking several Data Sources in order to centrally manage data while receiving all billing exports within a single invoice.

Ensure that the Discover Resources policy outlined in the Configure Policies & User section has been set to allow CerteroX to parse EC2 resource data.

  1. Go to CerteroX Cloud Management → Settings → Data Sources.

  2. Click Actions → Add

  3. Select AWS as the Data Source and Linked as Connection type.

  4. Enter the access key generated in Step 5 of Configure Policies & User.

  5. Press Connect.

This option removes the need to manually input bucket information, as the data will be received through the root account.
If only an AWS Linked account is specified, without connecting a Root account, CerteroX will not be able to import any billing data.

Migrating from CUR to CUR 2.0

Configure AWS

In the AWS console:

  1. Navigate to Billing & Cost Management → Data Exports. Click Create.

  2. Select "Standard data export" as the export type.

    1. Enter an "Export name".

    2. Under "Data table content settings":

    3. Under "Data export delivery options":

    4. Under "Data export storage setting":

    5. Confirm export creation. AWS will prepare the data export within 24 hours.

Update CerteroX Connection

  1. Go to CerteroX Cloud Management → Settings → Data Sources.

  2. Click on the AWS data source to be updated.

  3. Click Actions → Update Credentials.

  4. Switch on Update Data Export parameters & select Standard data export (CUR 2.0) as the export type.

  5. Update the Export name, Export path prefix as in the updated bucket.

  6. Press Save.