Manage AWS costs by using CCM on Harness Self-Managed Enterprise Edition
This topic walks you through the steps required to set up CCM for AWS in a self-managed platform.
Figure: AWS CCM Self-Managed Enterprise Edition architecture diagram

You need to perform the following tasks to set up CCM for AWS:
- Add a new user for programmatic access.
- Create Amazon S3 Bucket.
- Edit the CF template.
- Upload the CF template to S3 bucket.
- Deploy workloads via Helm charts.
Add a new user for programmatic access
- Sign in to your AWS console.
- Create a new user. For example,
harness-ccm-service-user
. For more information, go to Adding a user. - Set up programmatic user access keys by using the policy given below.
Policy to set up programmatic user access
Make a note of your AWS Access key and Secret key.
Create Amazon S3 Bucket
- Create an S3 bucket with the following naming convention. For more information, go to Creating a bucket. This bucket will be used to retrieve the CUR report from your master AWS accounts.
harness-ccm-service-data-bucket-<accountid>
- Create another bucket with the following naming convention.
harness-ccm-service-template-bucket-<accountid>
- Apply the following bucket policy.
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "AllowPublicRead",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::harness-ccm-service-template-bucket-<accountid>/*"
}
]
}
Edit the CF template
In the given YAML template, modify the specified field.
Replace <accountid>
with the AWS account ID in which the S3 buckets were created.
PrincipalBilling:
Type: String
Default: 'arn:aws:iam::<accountid>:root'
YAML template
Upload the CF template to S3 bucket
Upload the YAML template to S3 bucket harness-ccm-service-template-bucket-<accountid>
.
- Go to Amamzon S3 > ccm-service-template-bucket.
- Create a folder named
v1
. - Upload the file.

Make a note of the following:
-
The names of the two S3 buckets.
-
The complete path of the CF template in the bucket. For example, https://ccm-service-template-bucket.s3.amazonaws.com/v1/HarnessCCMServiceAWSTemplate.yaml.
Deploy workloads via Helm charts
-
Clone the chart repository.
git clone git@github.com:harness/helm-charts.git
cd main/src/harness -
Upgrade charts if you're already using Harness Self-managed Enterprise Edition services. Perform the following steps to update the override files:
-
Retrieve the current override values provided during the installation or upgrade of the Helm charts.
helm get values <chart-name> -n <namespace> > override.yaml
-
After obtaining the override file, you can make necessary modifications based on the type of environment.
-
Override file changes for a connected environment
Override file changes for an air-gapped environment
- After making the necessary updates to the override file, you can proceed with the Helm chart upgrade.
helm upgrade <chart-name> <chart-directory> -n <namespace> -f override.yaml
Handling Kubernetes secrets
When installing or upgrading the Helm charts, Kubernetes secrets with default values are created within the cluster. These generated secrets should be updated with the values mentioned above. Before updating the secrets, you need to convert the secret into base64 encoded format. For example, if your AWS_DESTINATION_BUCKET value is "harness-ccm-service-data-bucket-12345678", it would be stored as aGFybmVzcy1jY20tc2VydmljZS1kYXRhLWJ1Y2tldC0xMjM0NTY3OA==
after encoding. After changing secrets, we will provide directives to kubectl delete
the corresponding pods in order for your release to inherit new changes.
The following are the secrets specific to CCM services:
-
batch-processing
kubectl edit secret batch-processing -n <namespace>
S3_SYNC_CONFIG_ACCESSKEY: <S3_SYNC_CONFIG_ACCESSKEY> [AWS Setup - Add a new user - Use saved aws access key]
S3_SYNC_CONFIG_SECRETKEY: <S3_SYNC_CONFIG_ACCESSKEY> [AWS Setup - Add a new user- Use saved aws secret key] -
cloud-info-secret-mount [config-file]
kubectl edit secret cloud-info-secret-mount -n <namespace>
config-file: <config-file> [Sample can be found below]
-
nextgen-ce
kubectl edit secret nextgen-ce -n <namespace>
AWS_ACCESS_KEY: <AWS_ACCESS_KEY> [AWS Setup - Add a new user - Use saved aws access key]
AWS_SECRET_KEY: <AWS_SECRET_KEY> [AWS Setup - Add a new user- Use saved aws secret key]
AWS_ACCOUNT_ID: <AWS_ACCOUNT_ID> [AWS Setup - Id from here]
AWS_DESTINATION_BUCKET: <AWS_DESTINATION_BUCKET> [AWS Setup - bucket name from here 'harness-ccm-service-data-bucket-<accountid>']
AWS_TEMPLATE_LINK: <AWS_TEMPLATE_LINK> [AWS Setup - Create S3 buckets step - Use complete path of CF template]
CE_AWS_TEMPLATE_URL: <CE_AWS_TEMPLATE_URL> [AWS Setup - Create S3 buckets step - Use c
Config file
The following are some secrets from platform-service that you need to update:
-
smtp-secret - Required to support budget alerts email.
kubectl edit secret smtp-secret -n <namespace>
SMTP_HOST: <SMTP_HOST>
SMTP_PASSWORD: <SMTP_PASSWORD>
SMTP_USERNAME: <SMTP_USERNAME>
To increase TimescaleDB to 100Gi, run: kubectl edit pvc wal-volume-harness-timescaledb-0 -n <namespace>
. Features like Recommendations and Anomalies within CCM services use it.
Next steps
Troubleshooting
If in case the K8s secrets expire, the secrets will have to be set again. First you would have to update the secrets in respective secret.yaml
and then delete the pod. We recommend to kubectl delete
the following pods:
batch-processing
ce-nextgen
cloud-info
and then follow the same steps to set the keys. After the new keys are set, verify the changes by looking at the configs
for the pods.
Please refer to the commands below:
- batch-processing:
kubectl exec -it -n <namespace> batch-processing cat batch-processing-config.yml | grep -E 'awsAccessKey|awsSecretKey
awsAccessKey: Updated Access Key
awsSecretKey: Updated Secret Key
- cloud-info:
kubectl exec -it -n <namespace> cloud-info cat config/config.toml | grep -E 'accessKey|secretKey'
accessKey = "Updated Access Key"
secretKey = "Updated Secret Key"
- nextgen-ce:
kubectl exec -it -n <namespace> nextgen-ce cat config.yml | grep -E 'accessKey|secretKey|harnessAwsAccountId|destinationBucket:|awsConnectorTemplate'
accessKey: Updated Access Key
secretKey: Updated Secret Key
destinationBucket: Updated Destination Bucket
harnessAwsAccountId: Updated AWS Account Id
awsConnectorTemplate: Updated AWS Template URL