As most of you probably know, the AWS Well-Architected Framework (WAF) is a set of best practices and guidelines for designing and operating reliable, secure, efficient, and cost-effective systems in the cloud. It provides a comprehensive set of questions and principles to help customers evaluate their architectures and identify areas for improvement. To review how your application is doing compared to these best practices and guidelines AWS has introduced the AWS Well-Architected Tool.

One of the features of Well-Architected Tool is the ability to create custom lenses. Lenses are extensions to the Well-Architected Framework that allow you to define additional best practices and guidelines specific to your organization or industry. At Nordcloud we have recently started to create a custom lens for the AWS Well-Architected Framework and we needed a way continuously update the lens through an automated process. On the AWS blog you can an excellent post that describes how such a lifecycle could work. Unfortunately, there is no code example for the suggested implementation so in this post I’ll show you how I have built this.

Overview

Overview

As you can see in the diagram the solution I’m going to show today is fairly straightforward. What was new for me is using a AWS Lambda function as part of AWS CodePipeline. While the blog post linked above suggests invoking a Lambda function directly based on an AWS Codecommit event, I thought using CodePipeline would make it a bit easier to extend later on, with for example linting and schema validation. In addition, CodePipeline makes it easy to see if the action was successful or not. To deploy the solution I’ve created a small CDK project, which is deployed using CDK pipelines. If you want to get a quick intro on CDK pipelines, have a look at the CDK Workshop, which will walk you through deploying your CDK app using CDK pipelines.

The CDK Project

I started by importing all the required CDK components for this project.

from constructs import Construct
from aws_cdk import (
    Stack,
    aws_lambda as aws_lambda,
    aws_codecommit as codecommit,
    aws_codepipeline as codepipeline,
    aws_codepipeline_actions as codepipeline_actions,
    aws_iam as iam
)

Next, I have defined a CodeCommit repository. This repository will hold the custom lens source file. 

repo = codecommit.Repository(self, 'LensRepo',
    repository_name='well-architected-lens',
)

The Lambda function to be used in the Codepipeline pipelines is defined as follows. It gets assigned the required IAM permissions to retrieve files from the CodeCommit repository and publish the custom lens to the Well-Architected Tool.

deployment_action_lambda = aws_lambda.Function(
    self,
    "LensDeploymentAction",
    runtime=aws_lambda.Runtime.PYTHON_3_9,
    code=aws_lambda.Code.from_asset("assets/lens-deployment-codepipeline-action"),
    handler="app.handler",
)

deployment_action_lambda.role.add_to_principal_policy(
    iam.PolicyStatement(
        effect=iam.Effect.ALLOW,
        actions=[
            "codecommit:GetFile",
        ],
        resources=[
            repo.repository_arn
        ],
    )
)

deployment_action_lambda.role.add_to_principal_policy(
    iam.PolicyStatement(
        effect=iam.Effect.ALLOW,
        actions=[
            "wellarchitected:ImportLens",
            "wellarchitected:DeleteLens",
            "wellarchitected:CreateLensShare",
            "wellarchitected:CreateLensVersion",
            "wellarchitected:DeleteLensShare",
            "wellarchitected:GetLens",
            "wellarchitected:GetLensVersion",
            "wellarchitected:TagResource"
        ],
        resources=[
            "*"
        ],
    )
)

deployment_action_lambda.role.add_to_principal_policy(
    iam.PolicyStatement(
        effect=iam.Effect.ALLOW,
        actions=[
            "wellarchitected:ListLenses"
        ],
        resources=[
            "arn:aws:wellarchitected:{}:{}:/lenses".format(self.region, self.account)
        ],
    )
)

Lastly, I defined the CodePipeline, including the custom action to Invoke the Lambda.

source = codepipeline.Artifact()

source_action =  codepipeline_actions.CodeCommitSourceAction(
    repository=repo,
    branch='main',
    action_name='CodeCommit',
    output=source
)

pipeline = codepipeline.Pipeline(
    self, "LensDeploymentPipeline",
    pipeline_name='well-architected-lens-deployment',
    stages=[
        codepipeline.StageProps(
            stage_name='Source',
            actions=[
                source_action
            ]
        ),
        codepipeline.StageProps(
            stage_name='DeployLens',
            actions=[
                codepipeline_actions.LambdaInvokeAction(
                    lambda_=deployment_action_lambda,
                    action_name='PublishLens',
                    user_parameters={
                        'CommitId': source_action.variables.commit_id,
                        'Repo': source_action.variables.repository_name,
                        'Branch': source_action.variables.branch_name
                    }
                )
            ]
        )
    ]
)

As you see in the above example I pass data about the source to the Lambda function. This is because this information is not available by default in event data that CodePipeline sends to the Lambda function. Below you see the an example of such event. The UserParameters field now contains a JSON string with the information I need.

{
	'CodePipeline.job': {
		'id': 'c598f0e4-84e3-400d-84a4-326eb5bed68c',
		'accountId': '1234567890',
		'data': {
			'actionConfiguration': {
				'configuration': {
					'FunctionName': 'Deploy-LensDeployment-LensDeploymentAction62CFD909-CKyuo20qYcMC',
					'UserParameters': '{"CommitId":"bac545d3ee76c5323ca66b4dedb2fff73a2f8308", "Repo":"well-architected-lens","Branch":"main"}'
				}
			},
			'inputArtifacts': [],
			'outputArtifacts': [],
			'artifactCredentials': {
				'accessKeyId': 'xxxx',
				'secretAccessKey': 'xxxx',
				'sessionToken': 'xxxx',
				'expirationTime': 1677772078000
			},
			'encryptionKey': {
				'id': 'arn:aws:kms:eu-west-1:1234567890:key/0313e855-f89e-49aa-bd4d-03fa1e09ea38',
				'type': 'KMS'
			}
		}
	}
}

The Lambda function

Now let’s have a look at the Lambda function and how we can process the event.

First I have defined two simple functions that use the AWS python SDK (boto3) to send an event result to Codepipeline to signal if the action was successful or not.

def put_job_success(job_id):
    codepipeline.put_job_success_result(
        jobId=job_id
    )

def put_job_failure(job_id, message, execution_id):
    codepipeline.put_job_failure_result(
        jobId=job_id,
        failureDetails={
            'message': message,
            'type': 'JobFailed',
            'externalExecutionId': execution_id
        }
    )

In the Lambda handler I can now start to process the event. First, I use the information passed by AWS Codepipeline to fetch the lens file from the AWS CodeCommit repository.

user_parameters = json.loads(event["CodePipeline.job"]['data']['actionConfiguration']['configuration']['UserParameters'])
job_id = event['CodePipeline.job']['id']
commit_id = user_parameters['CommitId']
branch = user_parameters['Branch']
repo = user_parameters['Repo']
file_name = 'lens.json'
execution_id = context.aws_request_id
aws_account_id = context.invoked_function_arn.split(":")[4]
aws_region = os.getenv('AWS_REGION')

...

try: 
    response = codecommit.get_file(
        repositoryName=repo,
        commitSpecifier=commit_id,
        filePath=file_name
    )
    lens = json.loads(response['fileContent'].decode("utf-8"))
except botocore.exceptions.ClientError as error:
    error_message = "{}: {}".format(error.response['Error']['Code'], error.response['Error']['Message'])
    put_job_failure(job_id, error_message, execution_id)
    raise
except ValueError as error:
    error_message = "Error getting file {} from {}: Invalid JSON".format(file_name, repo)
    put_job_failure(job_id, error_message, execution_id)
    raise

In case any error occurs I pass the error to earlier defined functions to notify Codepipeline of the failure. If, for example, the file would not exist, you will see the following error in CodePipeline:

Failure

With the file contents available I can now import the lens in the Well-Architected Tool. To update an existing lens the ImportLens method expects a value in the LensAlias field. I used the name in the lens file to determine the AWS ARN. I wasn’t to find the exact format and normalization that is used from the official Lens format specification. We also used the UserParameters information to tag the new custom lens, so we track who has last updated it.

lens_name_normalized = lens['name'].lower().replace(' ', '-')
lens_name = lens['name']
lens_alias = "arn:aws:wellarchitected:{}:{}:lens/{}".format(aws_region, aws_account_id, lens_name_normalized)
lens_version = lens.get('_version', '1.0')

# list existing lenses
lenses = wellarchitected.list_lenses(
    LensType='CUSTOM_SELF',
    LensStatus='ALL',
    LensName=lens_name
)

if len(lenses['LensSummaries']) > 0: # update existing lens
    args = {
        'LensAlias': lenses['LensSummaries'][0]['LensArn'],
        'JSONString': json.dumps(lens) 
    }
else: # import new lens
    args = {
        'JSONString': json.dumps(lens) 
    }

try:
    response = wellarchitected.import_lens(**args)
    lens_arn = response['LensArn']
    wellarchitected.tag_resource(
        WorkloadArn=lens_arn,
        Tags={
            'LensAlias': lens_alias,
            'Repository': repo,
            'Branch': branch,
            'CommitId': commit_id
        }
    )
except botocore.exceptions.ClientError as error:
    error_message = "{}: {}".format(error.response['Error']['Code'], error.response['Error']['Message'])
    logger.error(error_message)
    put_job_failure(job_id, error_message, execution_id)

The last thing to do is to create a new version for the lens, as each imported lens will initially have the DRAFT state. Lenses in DRAFT state cannot be used for reviews. For now, I publish all new updates as MajorVersion, this will ensure workloads using the Lens in our organization will see a notification that there is a new version of the lens available. Note the _version field is not part of the official Lens format specification, I’ve just added it to makes things easier. In case anything fails I delete the draft version, so we don’t have to clean those up manually.

try:
    wellarchitected.create_lens_version(
        LensAlias=lens_arn,
        LensVersion=lens_version,
        IsMajorVersion=True
    )
except botocore.exceptions.ClientError as error:
    error_message = "{}: {}".format(error.response['Error']['Code'], error.response['Error']['Message'])
    logger.error(error_message)
    put_job_failure(job_id, error_message, execution_id)
    wellarchitected.delete_lens(
        LensAlias=lens_arn,
        LensStatus='DRAFT'
    )
    raise

put_job_success(job_id)

Wrap up

That’s it, you know have a simple pipeline to update your well-architected framework custom lens in the AWS Well-Architected tool. As always you can find the full project example on my GitHub.

Success

Photo by Nigel Tadyanehondo on Unsplash

comments powered by Disqus