Domain 4: Monitoring and Logging on Album69

Amazon CloudWatch Synthetics

Sun, 04 Oct 2026 00:00:00 +0000

1. Overview

What it is: Proactive monitoring using canaries — configurable scripts that run on a schedule and simulate user interactions with your endpoints. Synthetics finds problems before real users do.

What problem it solves: Reactive monitoring (alarms on real traffic metrics) only fires after users are already affected. Synthetics runs synthetic transactions continuously so you detect outages, broken APIs, and slow pages even during low-traffic periods.

2. Canaries

A canary is a script (Node.js or Python) that runs on a schedule in a managed Lambda-like environment. It makes HTTP requests, checks responses, and reports pass/fail to CloudWatch.

AWS CloudTrail

Sun, 04 Oct 2026 00:00:00 +0000

1. Overview

AWS CloudTrail is a service that enables governance, compliance, operational auditing, and risk auditing of your AWS account by logging, continuously monitoring, and retaining account activity related to actions across your AWS infrastructure.

Key Characteristics

Audit logging - Records API calls and account activity
Compliance - Meets regulatory and compliance requirements
Security analysis - Detect unusual activity patterns
Operational troubleshooting - Debug issues with API calls
Multi-region - Can log activity across all regions
Real-time - Near real-time event delivery
Integrated - Works with CloudWatch, S3, EventBridge

What Problem Does It Solve?

Provides visibility into user and resource activity
Enables security analysis and compliance reporting
Helps with operational troubleshooting
Tracks changes to AWS resources
Enables forensic analysis after security incidents
Supports compliance audits and governance

2. Core Concepts

Trail

Configuration that enables logging of AWS API calls
Defines where logs are stored (S3 bucket)
Can be single-region or multi-region
Can include global services (IAM, CloudFront, Route 53)

Event

Record of an activity in AWS account
Contains details about API call, user, time, source IP
Types: Management events, Data events, Insight events
JSON format with standardized fields

Management Events

Control plane operations (create, delete, modify resources)
Enabled by default in trails
Examples: CreateInstance, DeleteBucket, PutBucketPolicy
Read events vs Write events

Data Events

Data plane operations (object-level operations)
Not enabled by default (additional cost)
Examples: GetObject, PutObject, DeleteObject
High volume events

Insight Events

Identify unusual activity patterns
Machine learning-based analysis
Detect spikes in API call volume
Additional cost feature

Event History

90-day searchable history
Available by default (no trail needed)
Management events only
Free of charge

3. Trail Configuration

Basic Trail Configuration

{
 "TrailName": "MyTrail",
 "S3BucketName": "my-cloudtrail-bucket",
 "S3KeyPrefix": "cloudtrail-logs/",
 "IncludeGlobalServiceEvents": true,
 "IsMultiRegionTrail": true,
 "EnableLogFileValidation": true,
 "EventSelectors": [
 {
 "ReadWriteType": "All",
 "IncludeManagementEvents": true,
 "DataResources": [
 {
 "Type": "AWS::S3::Object",
 "Values": ["arn:aws:s3:::my-bucket/*"]
 }
 ]
 }
 ]
}

CloudFormation Template

AWSTemplateFormatVersion: '2010-09-09'
Resources:
 CloudTrail:
 Type: AWS::CloudTrail::Trail
 Properties:
 TrailName: MyCloudTrail
 S3BucketName: !Ref CloudTrailBucket
 S3KeyPrefix: cloudtrail-logs/
 IncludeGlobalServiceEvents: true
 IsMultiRegionTrail: true
 EnableLogFileValidation: true
 EventSelectors:
 - ReadWriteType: All
 IncludeManagementEvents: true
 DataResources:
 - Type: AWS::S3::Object
 Values: 
 - "arn:aws:s3:::my-sensitive-bucket/*"
 
 CloudTrailBucket:
 Type: AWS::S3::Bucket
 Properties:
 BucketName: my-cloudtrail-logs-bucket
 PublicAccessBlockConfiguration:
 BlockPublicAcls: true
 BlockPublicPolicy: true
 IgnorePublicAcls: true
 RestrictPublicBuckets: true

4. Event Structure

Standard Event Format

{
 "eventVersion": "1.08",
 "userIdentity": {
 "type": "IAMUser",
 "principalId": "AIDACKCEVSQ6C2EXAMPLE",
 "arn": "arn:aws:iam::123456789012:user/johndoe",
 "accountId": "123456789012",
 "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
 "userName": "johndoe"
 },
 "eventTime": "2023-11-15T14:30:00Z",
 "eventSource": "ec2.amazonaws.com",
 "eventName": "RunInstances",
 "awsRegion": "us-east-1",
 "sourceIPAddress": "203.0.113.12",
 "userAgent": "aws-cli/2.0.0 Python/3.8.0",
 "requestParameters": {
 "instanceType": "t3.micro",
 "minCount": 1,
 "maxCount": 1
 },
 "responseElements": {
 "instancesSet": {
 "items": [
 {
 "instanceId": "i-1234567890abcdef0",
 "imageId": "ami-12345678",
 "state": {
 "code": 0,
 "name": "pending"
 }
 }
 ]
 }
 },
 "requestID": "12345678-1234-1234-1234-123456789012",
 "eventID": "87654321-4321-4321-4321-210987654321",
 "eventType": "AwsApiCall",
 "recipientAccountId": "123456789012",
 "serviceEventDetails": {
 "responseElements": null
 }
}

Key Event Fields

eventVersion - CloudTrail record format version
userIdentity - Information about user/role that made request
eventTime - Date and time of request (UTC)
eventSource - AWS service that request was made to
eventName - Requested action
awsRegion - AWS region where request was made
sourceIPAddress - IP address of requester
userAgent - Agent through which request was made
requestParameters - Parameters sent with request
responseElements - Response elements for actions that make changes
errorCode - AWS service error if request returned error
errorMessage - Description of error

5. IAM Roles & Permissions

CloudTrail Service Role

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Effect": "Allow",
 "Action": [
 "logs:CreateLogGroup",
 "logs:CreateLogStream",
 "logs:PutLogEvents",
 "logs:DescribeLogGroups",
 "logs:DescribeLogStreams"
 ],
 "Resource": "arn:aws:logs:*:*:*"
 }
 ]
}

S3 Bucket Policy for CloudTrail

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Sid": "AWSCloudTrailAclCheck",
 "Effect": "Allow",
 "Principal": {
 "Service": "cloudtrail.amazonaws.com"
 },
 "Action": "s3:GetBucketAcl",
 "Resource": "arn:aws:s3:::my-cloudtrail-bucket"
 },
 {
 "Sid": "AWSCloudTrailWrite",
 "Effect": "Allow",
 "Principal": {
 "Service": "cloudtrail.amazonaws.com"
 },
 "Action": "s3:PutObject",
 "Resource": "arn:aws:s3:::my-cloudtrail-bucket/*",
 "Condition": {
 "StringEquals": {
 "s3:x-amz-acl": "bucket-owner-full-control"
 }
 }
 }
 ]
}

Cross-Account CloudTrail Access

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Effect": "Allow",
 "Principal": {
 "AWS": "arn:aws:iam::111111111111:root"
 },
 "Action": [
 "s3:GetBucketAcl",
 "s3:ListBucket"
 ],
 "Resource": "arn:aws:s3:::central-cloudtrail-bucket"
 },
 {
 "Effect": "Allow",
 "Principal": {
 "AWS": "arn:aws:iam::111111111111:root"
 },
 "Action": "s3:GetObject",
 "Resource": "arn:aws:s3:::central-cloudtrail-bucket/*"
 }
 ]
}

User Permissions for CloudTrail Analysis

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Effect": "Allow",
 "Action": [
 "cloudtrail:LookupEvents",
 "cloudtrail:GetTrailStatus",
 "cloudtrail:DescribeTrails"
 ],
 "Resource": "*"
 },
 {
 "Effect": "Allow",
 "Action": [
 "s3:GetObject",
 "s3:ListBucket"
 ],
 "Resource": [
 "arn:aws:s3:::my-cloudtrail-bucket",
 "arn:aws:s3:::my-cloudtrail-bucket/*"
 ]
 }
 ]
}

6. Integration with AWS Services

CloudWatch Logs Integration

{
 "CloudWatchLogsLogGroupArn": "arn:aws:logs:us-east-1:123456789012:log-group:CloudTrail/MyTrail:*",
 "CloudWatchLogsRoleArn": "arn:aws:iam::123456789012:role/CloudTrail_CloudWatchLogs_Role"
}

EventBridge Integration

CloudTrail events automatically sent to EventBridge
Create rules to trigger on specific API calls
Real-time response to security events

EventBridge Rule Example

{
 "Rules": [
 {
 "Name": "DetectRootLogin",
 "EventPattern": {
 "source": ["aws.signin"],
 "detail-type": ["AWS Console Sign In via CloudTrail"],
 "detail": {
 "userIdentity": {
 "type": ["Root"]
 }
 }
 },
 "Targets": [
 {
 "Arn": "arn:aws:sns:us-east-1:123456789012:security-alerts",
 "Id": "1"
 }
 ]
 }
 ]
}

AWS Config Integration

CloudTrail provides “who” and “when”
Config provides “what changed”
Combined view of configuration changes

Amazon Athena Integration

Query CloudTrail logs using SQL
Analyze patterns and trends
Cost-effective for large-scale analysis

Athena Table Creation

CREATE EXTERNAL TABLE cloudtrail_logs (
 eventversion STRING,
 useridentity STRUCT<
 type:STRING,
 principalid:STRING,
 arn:STRING,
 accountid:STRING,
 invokedby:STRING,
 accesskeyid:STRING,
 userName:STRING,
 sessioncontext:STRUCT<
 attributes:STRUCT<
 mfaauthenticated:STRING,
 creationdate:STRING>,
 sessionissuer:STRUCT<
 type:STRING,
 principalId:STRING,
 arn:STRING,
 accountId:STRING,
 userName:STRING>>>,
 eventtime STRING,
 eventsource STRING,
 eventname STRING,
 awsregion STRING,
 sourceipaddress STRING,
 useragent STRING,
 errorcode STRING,
 errormessage STRING,
 requestparameters STRING,
 responseelements STRING,
 additionaleventdata STRING,
 requestid STRING,
 eventid STRING,
 resources ARRAY<STRUCT<
 ARN:STRING,
 accountId:STRING,
 type:STRING>>,
 eventtype STRING,
 apiversion STRING,
 readonly STRING,
 recipientaccountid STRING,
 serviceeventdetails STRING,
 sharedeventid STRING,
 vpcendpointid STRING
)
PARTITIONED BY (
 region string,
 year string,
 month string,
 day string
)
STORED AS INPUTFORMAT 'com.amazon.emr.cloudtrail.CloudTrailInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 's3://my-cloudtrail-bucket/AWSLogs/123456789012/CloudTrail/'

7. Security Best Practices

Encryption

S3 bucket encryption - Enable default encryption
SSE-KMS - Use customer managed keys for additional control
Log file validation - Enable to detect tampering

S3 Bucket Security

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Sid": "DenyInsecureConnections",
 "Effect": "Deny",
 "Principal": "*",
 "Action": "s3:*",
 "Resource": [
 "arn:aws:s3:::my-cloudtrail-bucket",
 "arn:aws:s3:::my-cloudtrail-bucket/*"
 ],
 "Condition": {
 "Bool": {
 "aws:SecureTransport": "false"
 }
 }
 },
 {
 "Sid": "DenyUnencryptedObjectUploads",
 "Effect": "Deny",
 "Principal": "*",
 "Action": "s3:PutObject",
 "Resource": "arn:aws:s3:::my-cloudtrail-bucket/*",
 "Condition": {
 "StringNotEquals": {
 "s3:x-amz-server-side-encryption": "AES256"
 }
 }
 }
 ]
}

Access Control

Use least privilege for CloudTrail access
Separate read and write permissions
Use resource-based policies for cross-account access
Enable MFA for sensitive operations

Monitoring

Monitor CloudTrail configuration changes
Alert on trail deletion or modification
Monitor for unusual API activity patterns
Set up CloudWatch alarms for critical events

8. Monitoring & Troubleshooting

CloudWatch Metrics

ErrorCount - Number of errors in log delivery
TotalSizeBytes - Size of log files delivered

CloudWatch Alarms

{
 "AlarmName": "CloudTrail-LogDeliveryErrors",
 "MetricName": "ErrorCount",
 "Namespace": "AWS/CloudTrail",
 "Statistic": "Sum",
 "Period": 300,
 "EvaluationPeriods": 1,
 "Threshold": 1,
 "ComparisonOperator": "GreaterThanOrEqualToThreshold",
 "AlarmActions": [
 "arn:aws:sns:us-east-1:123456789012:cloudtrail-alerts"
 ]
}

Common Issues

Trail Not Logging

Check trail status: aws cloudtrail get-trail-status
Verify S3 bucket permissions
Check IAM role permissions
Verify trail is enabled

Missing Events

Check if event type is enabled (management vs data events)
Verify region configuration (single vs multi-region)
Check event selectors configuration
Verify global services events setting

Log Delivery Delays

Normal delay: 5-15 minutes
Check CloudWatch metrics for errors
Verify S3 bucket accessibility
Check for S3 bucket policy restrictions

Troubleshooting Commands

# Check trail status
aws cloudtrail get-trail-status --name MyTrail

# Describe trail configuration
aws cloudtrail describe-trails --trail-name-list MyTrail

# Look up recent events
aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventName,AttributeValue=RunInstances

# Validate log file integrity
aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:us-east-1:123456789012:trail/MyTrail --start-time 2023-11-15T00:00:00Z

9. Common Exam Scenarios

Scenario 1: Detect root account usage

Solution:

AWS CloudWatch Logs

Sun, 04 Oct 2026 00:00:00 +0000

1. Overview

AWS CloudWatch Logs enables you to monitor, store, and access your log files from Amazon EC2 instances, AWS CloudTrail, Route 53, and other sources. It provides centralized log management with real-time monitoring and analysis capabilities.

Key Characteristics

Centralized logging - Collect logs from multiple sources in one location
Real-time monitoring - Stream logs in real-time for immediate analysis
Log retention - Configurable retention periods from 1 day to never expire
Log insights - Query and analyze log data using CloudWatch Logs Insights
Metric filters - Extract metrics from log data for monitoring and alerting
Subscription filters - Stream log data to other services for processing
Cross-account access - Share logs across AWS accounts
Encryption - Server-side encryption for log data at rest

What Problem Does It Solve?

Centralizes log management across distributed systems
Provides real-time log monitoring and alerting
Enables log-based troubleshooting and debugging
Facilitates compliance and audit requirements
Supports automated log analysis and processing
Reduces operational overhead of log management

2. Core Components

Log Groups and Log Streams

# Create log group
aws logs create-log-group --log-group-name /aws/lambda/my-function

# Set retention policy
aws logs put-retention-policy \
 --log-group-name /aws/lambda/my-function \
 --retention-in-days 30

# Create log stream
aws logs create-log-stream \
 --log-group-name /aws/lambda/my-function \
 --log-stream-name 2023/12/01/stream1

# Put log events
aws logs put-log-events \
 --log-group-name /aws/lambda/my-function \
 --log-stream-name 2023/12/01/stream1 \
 --log-events timestamp=1640995200000,message="Application started"

CloudFormation Configuration

Resources:
 ApplicationLogGroup:
 Type: AWS::Logs::LogGroup
 Properties:
 LogGroupName: /aws/application/myapp
 RetentionInDays: 14
 KmsKeyId: !Ref LogsKMSKey

 LambdaLogGroup:
 Type: AWS::Logs::LogGroup
 Properties:
 LogGroupName: !Sub "/aws/lambda/${LambdaFunction}"
 RetentionInDays: 30

 # Custom log group with subscription filter
 CustomLogGroup:
 Type: AWS::Logs::LogGroup
 Properties:
 LogGroupName: /custom/application/logs
 RetentionInDays: 7

 LogSubscriptionFilter:
 Type: AWS::Logs::SubscriptionFilter
 Properties:
 LogGroupName: !Ref CustomLogGroup
 FilterPattern: "[timestamp, request_id, level=\"ERROR\", ...]"
 DestinationArn: !GetAtt ProcessingLambda.Arn

 # Metric filter for error counting
 ErrorMetricFilter:
 Type: AWS::Logs::MetricFilter
 Properties:
 LogGroupName: !Ref ApplicationLogGroup
 FilterPattern: "ERROR"
 MetricTransformations:
 - MetricNamespace: MyApp/Errors
 MetricName: ErrorCount
 MetricValue: "1"
 DefaultValue: 0

3. Log Agents and Collection

CloudWatch Agent Configuration

{
 "agent": {
 "metrics_collection_interval": 60,
 "run_as_user": "cwagent"
 },
 "logs": {
 "logs_collected": {
 "files": {
 "collect_list": [
 {
 "file_path": "/var/log/messages",
 "log_group_name": "/aws/ec2/system",
 "log_stream_name": "{instance_id}/messages",
 "timezone": "UTC",
 "timestamp_format": "%b %d %H:%M:%S"
 },
 {
 "file_path": "/var/log/httpd/access_log",
 "log_group_name": "/aws/ec2/httpd",
 "log_stream_name": "{instance_id}/access",
 "timezone": "UTC",
 "timestamp_format": "[%d/%b/%Y:%H:%M:%S %z]"
 },
 {
 "file_path": "/opt/myapp/logs/application.log",
 "log_group_name": "/aws/application/myapp",
 "log_stream_name": "{instance_id}",
 "timezone": "UTC",
 "multi_line_start_pattern": "^\\d{4}-\\d{2}-\\d{2}",
 "timestamp_format": "%Y-%m-%d %H:%M:%S"
 }
 ]
 },
 "windows_events": {
 "collect_list": [
 {
 "event_name": "System",
 "event_levels": ["ERROR", "WARNING"],
 "log_group_name": "/aws/ec2/windows/system",
 "log_stream_name": "{instance_id}"
 }
 ]
 }
 }
 },
 "metrics": {
 "namespace": "CWAgent",
 "metrics_collected": {
 "cpu": {
 "measurement": ["cpu_usage_idle", "cpu_usage_iowait"],
 "metrics_collection_interval": 60
 },
 "disk": {
 "measurement": ["used_percent"],
 "metrics_collection_interval": 60,
 "resources": ["*"]
 },
 "mem": {
 "measurement": ["mem_used_percent"],
 "metrics_collection_interval": 60
 }
 }
 }
}

Container Logging

# ECS Task Definition with CloudWatch Logs
Resources:
 TaskDefinition:
 Type: AWS::ECS::TaskDefinition
 Properties:
 Family: myapp-task
 NetworkMode: awsvpc
 RequiresCompatibilities:
 - FARGATE
 Cpu: 256
 Memory: 512
 ExecutionRoleArn: !GetAtt TaskExecutionRole.Arn
 ContainerDefinitions:
 - Name: myapp
 Image: myapp:latest
 LogConfiguration:
 LogDriver: awslogs
 Options:
 awslogs-group: !Ref ApplicationLogGroup
 awslogs-region: !Ref AWS::Region
 awslogs-stream-prefix: ecs
 Environment:
 - Name: LOG_LEVEL
 Value: INFO

# Kubernetes with Fluent Bit
apiVersion: v1
kind: ConfigMap
metadata:
 name: fluent-bit-config
data:
 fluent-bit.conf: |
 [SERVICE]
 Flush 1
 Log_Level info
 Daemon off
 Parsers_File parsers.conf
 HTTP_Server On
 HTTP_Listen 0.0.0.0
 HTTP_Port 2020

 [INPUT]
 Name tail
 Tag kube.*
 Path /var/log/containers/*.log
 Parser docker
 DB /var/log/flb_kube.db
 Mem_Buf_Limit 50MB
 Skip_Long_Lines On
 Refresh_Interval 10

 [OUTPUT]
 Name cloudwatch_logs
 Match kube.*
 region us-east-1
 log_group_name /aws/eks/cluster/logs
 log_stream_prefix ${HOSTNAME}-
 auto_create_group true

4. Log Insights and Querying

CloudWatch Logs Insights Queries

-- Find errors in the last hour
fields @timestamp, @message
| filter @message like /ERROR/
| sort @timestamp desc
| limit 100

-- Analyze Lambda function performance
fields @timestamp, @duration, @billedDuration, @memorySize, @maxMemoryUsed
| filter @type = "REPORT"
| stats avg(@duration), max(@duration), min(@duration) by bin(5m)

-- Parse and analyze application logs
fields @timestamp, @message
| parse @message /(?<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) \[(?<level>\w+)\] (?<message>.*)/
| filter level = "ERROR"
| stats count() by bin(1h)

-- Monitor API Gateway access patterns
fields @timestamp, @message
| filter @message like /\[.*\] ".*" \d{3}/
| parse @message /\[(?<timestamp>.*)\] "(?<method>\w+) (?<path>.*) HTTP\/\d\.\d" (?<status>\d{3})/
| stats count() by status, method
| sort count desc

-- Analyze ECS container logs
fields @timestamp, @message, @logStream
| filter @message like /ERROR/
| parse @logStream /ecs\/(?<service>.*?)\/(?<task>.*)/
| stats count() by service
| sort count desc

-- Custom application metrics from logs
fields @timestamp, @message
| parse @message /response_time=(?<response_time>\d+)ms/
| filter ispresent(response_time)
| stats avg(response_time), max(response_time), min(response_time) by bin(5m)

Programmatic Log Analysis

import boto3
import json
from datetime import datetime, timedelta

class CloudWatchLogsAnalyzer:
 def __init__(self):
 self.logs_client = boto3.client('logs')
 
 def run_insights_query(self, log_group, query, start_time, end_time):
 """Run CloudWatch Logs Insights query"""
 
 response = self.logs_client.start_query(
 logGroupName=log_group,
 startTime=int(start_time.timestamp()),
 endTime=int(end_time.timestamp()),
 queryString=query
 )
 
 query_id = response['queryId']
 
 # Wait for query to complete
 while True:
 result = self.logs_client.get_query_results(queryId=query_id)
 
 if result['status'] == 'Complete':
 return result['results']
 elif result['status'] == 'Failed':
 raise Exception(f"Query failed: {result.get('statistics', {}).get('recordsMatched', 0)}")
 
 time.sleep(1)
 
 def analyze_error_patterns(self, log_group, hours_back=24):
 """Analyze error patterns in logs"""
 
 end_time = datetime.utcnow()
 start_time = end_time - timedelta(hours=hours_back)
 
 query = """
 fields @timestamp, @message
 | filter @message like /ERROR/
 | parse @message /ERROR: (?<error_type>.*?) - (?<error_message>.*)/
 | stats count() by error_type
 | sort count desc
 """
 
 results = self.run_insights_query(log_group, query, start_time, end_time)
 
 error_summary = {}
 for result in results:
 error_type = next((field['value'] for field in result if field['field'] == 'error_type'), 'Unknown')
 count = int(next((field['value'] for field in result if field['field'] == 'count'), 0))
 error_summary[error_type] = count
 
 return error_summary
 
 def get_log_events(self, log_group, log_stream=None, start_time=None, end_time=None, filter_pattern=None):
 """Get log events with optional filtering"""
 
 params = {
 'logGroupName': log_group,
 'startFromHead': True
 }
 
 if log_stream:
 params['logStreamNames'] = [log_stream]
 
 if start_time:
 params['startTime'] = int(start_time.timestamp() * 1000)
 
 if end_time:
 params['endTime'] = int(end_time.timestamp() * 1000)
 
 if filter_pattern:
 params['filterPattern'] = filter_pattern
 
 events = []
 paginator = self.logs_client.get_paginator('filter_log_events')
 
 for page in paginator.paginate(**params):
 events.extend(page['events'])
 
 return events
 
 def create_metric_filter(self, log_group, filter_name, filter_pattern, metric_name, metric_namespace):
 """Create metric filter from log data"""
 
 self.logs_client.put_metric_filter(
 logGroupName=log_group,
 filterName=filter_name,
 filterPattern=filter_pattern,
 metricTransformations=[
 {
 'metricName': metric_name,
 'metricNamespace': metric_namespace,
 'metricValue': '1',
 'defaultValue': 0
 }
 ]
 )
 
 def export_logs_to_s3(self, log_group, s3_bucket, s3_prefix, start_time, end_time):
 """Export logs to S3 for long-term storage"""
 
 response = self.logs_client.create_export_task(
 logGroupName=log_group,
 fromTime=int(start_time.timestamp() * 1000),
 to=int(end_time.timestamp() * 1000),
 destination=s3_bucket,
 destinationPrefix=s3_prefix
 )
 
 return response['taskId']

5. Subscription Filters and Real-time Processing

Lambda Processing

# Lambda function for log processing
import json
import gzip
import base64
import boto3

def lambda_handler(event, context):
 """Process CloudWatch Logs via subscription filter"""
 
 # Decode and decompress log data
 compressed_payload = base64.b64decode(event['awslogs']['data'])
 uncompressed_payload = gzip.decompress(compressed_payload)
 log_data = json.loads(uncompressed_payload)
 
 processed_events = []
 
 for log_event in log_data['logEvents']:
 message = log_event['message']
 timestamp = log_event['timestamp']
 
 # Parse log message
 if 'ERROR' in message:
 # Extract error information
 error_info = parse_error_message(message)
 
 # Send to SNS for alerting
 send_error_alert(error_info, timestamp)
 
 # Store in DynamoDB for tracking
 store_error_event(error_info, timestamp)
 
 # Process metrics
 if 'response_time=' in message:
 response_time = extract_response_time(message)
 send_custom_metric('ResponseTime', response_time, timestamp)
 
 processed_events.append({
 'timestamp': timestamp,
 'message': message,
 'processed': True
 })
 
 return {
 'statusCode': 200,
 'body': json.dumps(f'Processed {len(processed_events)} log events')
 }

def parse_error_message(message):
 """Parse error message to extract structured information"""
 
 # Example parsing logic
 import re
 
 pattern = r'ERROR: (?P<error_type>.*?) - (?P<error_message>.*?) \[(?P<component>.*?)\]'
 match = re.search(pattern, message)
 
 if match:
 return {
 'error_type': match.group('error_type'),
 'error_message': match.group('error_message'),
 'component': match.group('component')
 }
 
 return {'error_type': 'Unknown', 'error_message': message, 'component': 'Unknown'}

def send_error_alert(error_info, timestamp):
 """Send error alert via SNS"""
 
 sns = boto3.client('sns')
 
 message = f"""
 Error Alert:
 Type: {error_info['error_type']}
 Message: {error_info['error_message']}
 Component: {error_info['component']}
 Timestamp: {timestamp}
 """
 
 sns.publish(
 TopicArn='arn:aws:sns:us-east-1:123456789012:error-alerts',
 Message=message,
 Subject=f"Application Error: {error_info['error_type']}"
 )

def send_custom_metric(metric_name, value, timestamp):
 """Send custom metric to CloudWatch"""
 
 cloudwatch = boto3.client('cloudwatch')
 
 cloudwatch.put_metric_data(
 Namespace='MyApp/Performance',
 MetricData=[
 {
 'MetricName': metric_name,
 'Value': float(value),
 'Timestamp': datetime.fromtimestamp(timestamp / 1000),
 'Unit': 'Milliseconds'
 }
 ]
 )

Kinesis Data Streams Integration

# CloudFormation for Kinesis integration
Resources:
 LogProcessingStream:
 Type: AWS::Kinesis::Stream
 Properties:
 Name: log-processing-stream
 ShardCount: 2
 RetentionPeriodHours: 24

 LogSubscriptionFilter:
 Type: AWS::Logs::SubscriptionFilter
 Properties:
 LogGroupName: !Ref ApplicationLogGroup
 FilterPattern: "[timestamp, request_id, level, message]"
 DestinationArn: !GetAtt LogProcessingStream.Arn
 RoleArn: !GetAtt LogsRole.Arn

 # Kinesis Analytics for real-time processing
 LogAnalyticsApplication:
 Type: AWS::KinesisAnalytics::Application
 Properties:
 ApplicationName: log-analytics
 ApplicationDescription: Real-time log analysis
 ApplicationCode: |
 CREATE OR REPLACE STREAM "DESTINATION_SQL_STREAM" (
 timestamp TIMESTAMP,
 level VARCHAR(10),
 message VARCHAR(1000),
 error_count INTEGER
 );
 
 CREATE OR REPLACE PUMP "STREAM_PUMP" AS INSERT INTO "DESTINATION_SQL_STREAM"
 SELECT STREAM
 ROWTIME_TO_TIMESTAMP(ROWTIME) as timestamp,
 level,
 message,
 CASE WHEN level = 'ERROR' THEN 1 ELSE 0 END as error_count
 FROM "SOURCE_SQL_STREAM_001"
 WHERE level IN ('ERROR', 'WARN', 'INFO');
 Inputs:
 - NamePrefix: SOURCE_SQL_STREAM
 KinesisStreamsInput:
 ResourceARN: !GetAtt LogProcessingStream.Arn
 RoleARN: !GetAtt AnalyticsRole.Arn
 InputSchema:
 RecordColumns:
 - Name: timestamp
 SqlType: TIMESTAMP
 Mapping: $.timestamp
 - Name: level
 SqlType: VARCHAR(10)
 Mapping: $.level
 - Name: message
 SqlType: VARCHAR(1000)
 Mapping: $.message

Cross-Account Access Setup

# Destination account (log aggregation account)
Resources:
 LogDestination:
 Type: AWS::Logs::Destination
 Properties:
 DestinationName: central-log-destination
 RoleArn: !GetAtt CrossAccountLogsRole.Arn
 TargetArn: !GetAtt CentralLogGroup.Arn
 DestinationPolicy: !Sub |
 {
 "Version": "2012-10-17",
 "Statement": [
 {
 "Sid": "AllowSourceAccounts",
 "Effect": "Allow",
 "Principal": {
 "AWS": [
 "arn:aws:iam::111111111111:root",
 "arn:aws:iam::222222222222:root"
 ]
 },
 "Action": "logs:PutSubscriptionFilter",
 "Resource": "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:destination:central-log-destination"
 }
 ]
 }

 CrossAccountLogsRole:
 Type: AWS::IAM::Role
 Properties:
 AssumeRolePolicyDocument:
 Version: '2012-10-17'
 Statement:
 - Effect: Allow
 Principal:
 Service: logs.amazonaws.com
 Action: sts:AssumeRole
 Policies:
 - PolicyName: LogsDeliveryPolicy
 PolicyDocument:
 Version: '2012-10-17'
 Statement:
 - Effect: Allow
 Action:
 - logs:PutLogEvents
 Resource: !GetAtt CentralLogGroup.Arn

 CentralLogGroup:
 Type: AWS::Logs::LogGroup
 Properties:
 LogGroupName: /central/aggregated-logs
 RetentionInDays: 90

Source Account Configuration

# Python script to setup cross-account log streaming
import boto3

def setup_cross_account_logging(source_log_group, destination_arn, filter_pattern=""):
 """Setup cross-account log streaming"""
 
 logs_client = boto3.client('logs')
 
 # Create subscription filter pointing to destination account
 logs_client.put_subscription_filter(
 logGroupName=source_log_group,
 filterName=f"{source_log_group}-cross-account-filter",
 filterPattern=filter_pattern,
 destinationArn=destination_arn
 )
 
 print(f"Cross-account logging setup for {source_log_group}")

# Usage
setup_cross_account_logging(
 source_log_group="/aws/lambda/my-function",
 destination_arn="arn:aws:logs:us-east-1:999999999999:destination:central-log-destination",
 filter_pattern="[timestamp, request_id, level=\"ERROR\", ...]"
)

7. Security and Encryption

Encryption Configuration

Resources:
 LogsKMSKey:
 Type: AWS::KMS::Key
 Properties:
 Description: KMS key for CloudWatch Logs encryption
 KeyPolicy:
 Version: '2012-10-17'
 Statement:
 - Sid: Enable IAM User Permissions
 Effect: Allow
 Principal:
 AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root"
 Action: "kms:*"
 Resource: "*"
 - Sid: Allow CloudWatch Logs
 Effect: Allow
 Principal:
 Service: !Sub "logs.${AWS::Region}.amazonaws.com"
 Action:
 - kms:Encrypt
 - kms:Decrypt
 - kms:ReEncrypt*
 - kms:GenerateDataKey*
 - kms:DescribeKey
 Resource: "*"
 Condition:
 ArnEquals:
 "kms:EncryptionContext:aws:logs:arn": !Sub "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/secure/application/*"

 LogsKMSKeyAlias:
 Type: AWS::KMS::Alias
 Properties:
 AliasName: alias/cloudwatch-logs
 TargetKeyId: !Ref LogsKMSKey

 SecureLogGroup:
 Type: AWS::Logs::LogGroup
 Properties:
 LogGroupName: /secure/application/logs
 RetentionInDays: 30
 KmsKeyId: !GetAtt LogsKMSKey.Arn

IAM Policies for Log Access

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Sid": "CloudWatchLogsFullAccess",
 "Effect": "Allow",
 "Action": [
 "logs:CreateLogGroup",
 "logs:CreateLogStream",
 "logs:PutLogEvents",
 "logs:DescribeLogGroups",
 "logs:DescribeLogStreams"
 ],
 "Resource": [
 "arn:aws:logs:*:*:log-group:/aws/lambda/*",
 "arn:aws:logs:*:*:log-group:/application/*"
 ]
 },
 {
 "Sid": "CloudWatchLogsInsights",
 "Effect": "Allow",
 "Action": [
 "logs:StartQuery",
 "logs:StopQuery",
 "logs:GetQueryResults",
 "logs:DescribeQueries"
 ],
 "Resource": "*"
 },
 {
 "Sid": "MetricFiltersAccess",
 "Effect": "Allow",
 "Action": [
 "logs:PutMetricFilter",
 "logs:DeleteMetricFilter",
 "logs:DescribeMetricFilters"
 ],
 "Resource": "arn:aws:logs:*:*:log-group:*"
 },
 {
 "Sid": "SubscriptionFiltersAccess",
 "Effect": "Allow",
 "Action": [
 "logs:PutSubscriptionFilter",
 "logs:DeleteSubscriptionFilter",
 "logs:DescribeSubscriptionFilters"
 ],
 "Resource": "arn:aws:logs:*:*:log-group:*",
 "Condition": {
 "StringEquals": {
 "logs:destination-arn": [
 "arn:aws:lambda:*:*:function:log-processor",
 "arn:aws:kinesis:*:*:stream/log-stream"
 ]
 }
 }
 }
 ]
}

8. Integration with CI/CD

CodePipeline Integration

# CloudFormation for CI/CD log monitoring
Resources:
 PipelineLogGroup:
 Type: AWS::Logs::LogGroup
 Properties:
 LogGroupName: /aws/codepipeline/deployment-pipeline
 RetentionInDays: 14

 BuildLogGroup:
 Type: AWS::Logs::LogGroup
 Properties:
 LogGroupName: /aws/codebuild/build-project
 RetentionInDays: 7

 # Metric filter for deployment failures
 DeploymentFailureMetric:
 Type: AWS::Logs::MetricFilter
 Properties:
 LogGroupName: !Ref PipelineLogGroup
 FilterPattern: "[timestamp, request_id, level=\"FAILED\", stage, ...]"
 MetricTransformations:
 - MetricNamespace: CI/CD/Pipeline
 MetricName: DeploymentFailures
 MetricValue: "1"
 DefaultValue: 0

 # Alarm for deployment failures
 DeploymentFailureAlarm:
 Type: AWS::CloudWatch::Alarm
 Properties:
 AlarmName: DeploymentFailures
 AlarmDescription: Alert on deployment failures
 MetricName: DeploymentFailures
 Namespace: CI/CD/Pipeline
 Statistic: Sum
 Period: 300
 EvaluationPeriods: 1
 Threshold: 1
 ComparisonOperator: GreaterThanOrEqualToThreshold
 AlarmActions:
 - !Ref DeploymentAlertsTopic

Application Deployment Monitoring

# Lambda function for deployment log analysis
import boto3
import json
from datetime import datetime, timedelta

def lambda_handler(event, context):
 """Analyze deployment logs for issues"""
 
 logs_client = boto3.client('logs')
 cloudwatch = boto3.client('cloudwatch')
 
 # Analyze recent deployment logs
 end_time = datetime.utcnow()
 start_time = end_time - timedelta(hours=1)
 
 # Query for deployment events
 query = """
 fields @timestamp, @message
 | filter @message like /DEPLOYMENT/
 | parse @message /DEPLOYMENT (?<status>\w+): (?<service>.*?) - (?<details>.*)/
 | stats count() by status, service
 """
 
 try:
 response = logs_client.start_query(
 logGroupName='/aws/codepipeline/deployment-pipeline',
 startTime=int(start_time.timestamp()),
 endTime=int(end_time.timestamp()),
 queryString=query
 )
 
 query_id = response['queryId']
 
 # Wait for results
 while True:
 result = logs_client.get_query_results(queryId=query_id)
 
 if result['status'] == 'Complete':
 break
 elif result['status'] == 'Failed':
 raise Exception("Query failed")
 
 time.sleep(1)
 
 # Process results and send metrics
 for row in result['results']:
 status = next((field['value'] for field in row if field['field'] == 'status'), 'Unknown')
 service = next((field['value'] for field in row if field['field'] == 'service'), 'Unknown')
 count = int(next((field['value'] for field in row if field['field'] == 'count'), 0))
 
 # Send custom metrics
 cloudwatch.put_metric_data(
 Namespace='CI/CD/Deployments',
 MetricData=[
 {
 'MetricName': f'Deployment{status}',
 'Dimensions': [
 {
 'Name': 'Service',
 'Value': service
 }
 ],
 'Value': count,
 'Unit': 'Count',
 'Timestamp': end_time
 }
 ]
 )
 
 return {
 'statusCode': 200,
 'body': json.dumps('Deployment analysis completed')
 }
 
 except Exception as e:
 print(f"Error analyzing deployment logs: {str(e)}")
 return {
 'statusCode': 500,
 'body': json.dumps(f'Error: {str(e)}')
 }

9. Common Exam Scenarios

Scenario 1: Centralized Logging Architecture

# Complete centralized logging setup
import boto3
import json

class CentralizedLoggingSetup:
 def __init__(self):
 self.logs_client = boto3.client('logs')
 self.iam_client = boto3.client('iam')
 self.lambda_client = boto3.client('lambda')
 
 def setup_centralized_logging(self, applications):
 """Setup centralized logging for multiple applications"""
 
 # Create central log groups
 central_groups = {}
 
 for app in applications:
 log_group_name = f"/central/{app['name']}/logs"
 
 try:
 self.logs_client.create_log_group(
 logGroupName=log_group_name,
 kmsKeyId=app.get('kms_key_id'),
 tags={
 'Application': app['name'],
 'Environment': app['environment'],
 'LogType': 'Centralized'
 }
 )
 
 # Set retention policy
 self.logs_client.put_retention_policy(
 logGroupName=log_group_name,
 retentionInDays=app.get('retention_days', 30)
 )
 
 central_groups[app['name']] = log_group_name
 
 except self.logs_client.exceptions.ResourceAlreadyExistsException:
 central_groups[app['name']] = log_group_name
 
 # Setup log aggregation Lambda
 aggregation_function = self.create_log_aggregation_function()
 
 # Create subscription filters
 for app in applications:
 for source_log_group in app['source_log_groups']:
 self.create_subscription_filter(
 source_log_group,
 aggregation_function['FunctionArn'],
 app.get('filter_pattern', '')
 )
 
 return {
 'central_log_groups': central_groups,
 'aggregation_function': aggregation_function['FunctionArn']
 }
 
 def create_log_aggregation_function(self):
 """Create Lambda function for log aggregation"""
 
 function_code = '''
import json
import gzip
import base64
import boto3
import re
from datetime import datetime

def lambda_handler(event, context):
 # Decode log data
 compressed_payload = base64.b64decode(event['awslogs']['data'])
 uncompressed_payload = gzip.decompress(compressed_payload)
 log_data = json.loads(uncompressed_payload)
 
 logs_client = boto3.client('logs')
 
 # Process each log event
 for log_event in log_data['logEvents']:
 # Parse and enrich log message
 enriched_message = enrich_log_message(
 log_event['message'],
 log_data['logGroup'],
 log_data['logStream']
 )
 
 # Send to central log group
 central_log_group = determine_central_log_group(log_data['logGroup'])
 
 logs_client.put_log_events(
 logGroupName=central_log_group,
 logStreamName=f"aggregated-{datetime.now().strftime('%Y/%m/%d')}",
 logEvents=[
 {
 'timestamp': log_event['timestamp'],
 'message': enriched_message
 }
 ]
 )
 
 return {'statusCode': 200}

def enrich_log_message(message, log_group, log_stream):
 # Add metadata to log message
 enriched = {
 'original_message': message,
 'source_log_group': log_group,
 'source_log_stream': log_stream,
 'processed_timestamp': datetime.utcnow().isoformat(),
 'metadata': extract_metadata(message)
 }
 
 return json.dumps(enriched)

def extract_metadata(message):
 # Extract structured data from log message
 metadata = {}
 
 # Extract log level
 level_match = re.search(r'\\b(DEBUG|INFO|WARN|ERROR|FATAL)\\b', message)
 if level_match:
 metadata['level'] = level_match.group(1)
 
 # Extract request ID
 request_id_match = re.search(r'request[_-]?id[=:]\\s*([a-zA-Z0-9-]+)', message, re.IGNORECASE)
 if request_id_match:
 metadata['request_id'] = request_id_match.group(1)
 
 return metadata

def determine_central_log_group(source_log_group):
 # Map source log groups to central log groups
 mapping = {
 '/aws/lambda/': '/central/lambda/logs',
 '/aws/apigateway/': '/central/apigateway/logs',
 '/aws/ecs/': '/central/ecs/logs'
 }
 
 for prefix, central_group in mapping.items():
 if source_log_group.startswith(prefix):
 return central_group
 
 return '/central/unknown/logs'
 '''
 
 # Create IAM role for Lambda
 role_response = self.iam_client.create_role(
 RoleName='LogAggregationLambdaRole',
 AssumeRolePolicyDocument=json.dumps({
 "Version": "2012-10-17",
 "Statement": [
 {
 "Effect": "Allow",
 "Principal": {"Service": "lambda.amazonaws.com"},
 "Action": "sts:AssumeRole"
 }
 ]
 })
 )
 
 # Attach policies
 self.iam_client.attach_role_policy(
 RoleName='LogAggregationLambdaRole',
 PolicyArn='arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'
 )
 
 # Create custom policy for logs access
 self.iam_client.put_role_policy(
 RoleName='LogAggregationLambdaRole',
 PolicyName='LogsAccess',
 PolicyDocument=json.dumps({
 "Version": "2012-10-17",
 "Statement": [
 {
 "Effect": "Allow",
 "Action": [
 "logs:CreateLogStream",
 "logs:PutLogEvents",
 "logs:DescribeLogGroups",
 "logs:DescribeLogStreams"
 ],
 "Resource": "arn:aws:logs:*:*:log-group:/central/*"
 }
 ]
 })
 )
 
 # Create Lambda function
 function_response = self.lambda_client.create_function(
 FunctionName='log-aggregation-function',
 Runtime='python3.9',
 Role=role_response['Role']['Arn'],
 Handler='index.lambda_handler',
 Code={'ZipFile': function_code.encode()},
 Description='Centralized log aggregation function',
 Timeout=60,
 MemorySize=256
 )
 
 return function_response

Scenario 2: Real-time Log Monitoring and Alerting

# Complete real-time monitoring setup
Resources:
 # Application log group
 ApplicationLogGroup:
 Type: AWS::Logs::LogGroup
 Properties:
 LogGroupName: /aws/application/myapp
 RetentionInDays: 14

 # Error rate metric filter
 ErrorRateMetricFilter:
 Type: AWS::Logs::MetricFilter
 Properties:
 LogGroupName: !Ref ApplicationLogGroup
 FilterPattern: "ERROR"
 MetricTransformations:
 - MetricNamespace: MyApp/Errors
 MetricName: ErrorRate
 MetricValue: "1"
 DefaultValue: 0

 # Response time metric filter
 ResponseTimeMetricFilter:
 Type: AWS::Logs::MetricFilter
 Properties:
 LogGroupName: !Ref ApplicationLogGroup
 FilterPattern: "[timestamp, request_id, level, message, response_time_ms]"
 MetricTransformations:
 - MetricNamespace: MyApp/Performance
 MetricName: ResponseTime
 MetricValue: "$response_time_ms"
 DefaultValue: 0

 # High error rate alarm
 HighErrorRateAlarm:
 Type: AWS::CloudWatch::Alarm
 Properties:
 AlarmName: HighErrorRate
 AlarmDescription: Alert when error rate is high
 MetricName: ErrorRate
 Namespace: MyApp/Errors
 Statistic: Sum
 Period: 300
 EvaluationPeriods: 2
 Threshold: 10
 ComparisonOperator: GreaterThanThreshold
 AlarmActions:
 - !Ref ErrorAlertsTopic
 TreatMissingData: notBreaching

 # Slow response time alarm
 SlowResponseAlarm:
 Type: AWS::CloudWatch::Alarm
 Properties:
 AlarmName: SlowResponseTime
 AlarmDescription: Alert when response time is slow
 MetricName: ResponseTime
 Namespace: MyApp/Performance
 Statistic: Average
 Period: 300
 EvaluationPeriods: 3
 Threshold: 2000
 ComparisonOperator: GreaterThanThreshold
 AlarmActions:
 - !Ref PerformanceAlertsTopic

 # Real-time log processing
 LogProcessingFunction:
 Type: AWS::Lambda::Function
 Properties:
 FunctionName: real-time-log-processor
 Runtime: python3.9
 Handler: index.lambda_handler
 Role: !GetAtt LogProcessingRole.Arn
 Code:
 ZipFile: |
 import json
 import gzip
 import base64
 import boto3
 import re
 from datetime import datetime
 
 def lambda_handler(event, context):
 # Process real-time log events
 compressed_payload = base64.b64decode(event['awslogs']['data'])
 uncompressed_payload = gzip.decompress(compressed_payload)
 log_data = json.loads(uncompressed_payload)
 
 sns = boto3.client('sns')
 cloudwatch = boto3.client('cloudwatch')
 
 critical_errors = []
 performance_issues = []
 
 for log_event in log_data['logEvents']:
 message = log_event['message']
 timestamp = log_event['timestamp']
 
 # Detect critical errors
 if re.search(r'CRITICAL|FATAL|OutOfMemoryError', message, re.IGNORECASE):
 critical_errors.append({
 'timestamp': timestamp,
 'message': message
 })
 
 # Detect performance issues
 response_time_match = re.search(r'response_time=(\d+)ms', message)
 if response_time_match:
 response_time = int(response_time_match.group(1))
 if response_time > 5000: # 5 seconds
 performance_issues.append({
 'timestamp': timestamp,
 'response_time': response_time,
 'message': message
 })
 
 # Send immediate alerts for critical errors
 if critical_errors:
 sns.publish(
 TopicArn='arn:aws:sns:us-east-1:123456789012:critical-alerts',
 Message=json.dumps(critical_errors, indent=2),
 Subject=f'CRITICAL: {len(critical_errors)} critical errors detected'
 )
 
 # Send performance alerts
 if performance_issues:
 sns.publish(
 TopicArn='arn:aws:sns:us-east-1:123456789012:performance-alerts',
 Message=json.dumps(performance_issues, indent=2),
 Subject=f'Performance Issue: {len(performance_issues)} slow responses detected'
 )
 
 return {
 'statusCode': 200,
 'body': json.dumps({
 'critical_errors': len(critical_errors),
 'performance_issues': len(performance_issues)
 })
 }

 # Subscription filter for real-time processing
 RealTimeSubscriptionFilter:
 Type: AWS::Logs::SubscriptionFilter
 Properties:
 LogGroupName: !Ref ApplicationLogGroup
 FilterPattern: ""
 DestinationArn: !GetAtt LogProcessingFunction.Arn

 # Permission for CloudWatch Logs to invoke Lambda
 LogsLambdaPermission:
 Type: AWS::Lambda::Permission
 Properties:
 FunctionName: !Ref LogProcessingFunction
 Action: lambda:InvokeFunction
 Principal: logs.amazonaws.com
 SourceArn: !Sub "${ApplicationLogGroup}:*"

10. Exam Tips

Understand log hierarchy - Log groups contain log streams, streams contain events
Master retention policies - Know default retention (never expire) and cost implications
Know metric filters - Extract metrics from log data for monitoring
Practice Logs Insights - Query syntax and common patterns
Understand subscription filters - Real-time log processing and destinations
Learn cross-account sharing - Log destinations and IAM policies
Know encryption options - KMS integration and key policies
Practice troubleshooting - Common issues with log agents and permissions
Understand cost optimization - Retention policies, log compression, S3 export
Master integration patterns - Lambda, Kinesis, ElasticSearch, S3 integration

AWS GuardDuty

Sun, 04 Oct 2026 00:00:00 +0000

1. Overview

AWS GuardDuty is a threat detection service that continuously monitors for malicious activity and unauthorized behavior to protect your AWS accounts, workloads, and data stored in Amazon S3.

Key Characteristics

Threat intelligence - Uses machine learning, anomaly detection, and integrated threat intelligence
Continuous monitoring - 24/7 monitoring of AWS accounts and workloads
Multi-data source - Analyzes VPC Flow Logs, DNS logs, CloudTrail logs, and S3 data events
Automated response - Integration with Lambda, SNS, and other services for automated remediation
Multi-account support - Centralized security monitoring across AWS Organizations
Malware detection - Scans EBS volumes for malware (container image scanning is Amazon Inspector, not GuardDuty)
Runtime monitoring - Monitors EKS clusters and EC2 instances for suspicious activity

What Problem Does It Solve?

Detects sophisticated threats and attacks in real-time
Provides centralized security monitoring across multiple accounts
Reduces time to detect and respond to security incidents
Automates threat detection without requiring security expertise
Protects against data exfiltration and cryptocurrency mining
Monitors for insider threats and compromised credentials

2. Core Components

Detector Configuration

# CloudFormation for GuardDuty setup
Resources:
 GuardDutyDetector:
 Type: AWS::GuardDuty::Detector
 Properties:
 Enable: true
 FindingPublishingFrequency: FIFTEEN_MINUTES
 DataSources:
 S3Logs:
 Enable: true
 KubernetesConfiguration:
 AuditLogs:
 Enable: true
 MalwareProtection:
 ScanEc2InstanceWithFindings:
 EbsVolumes: true

 # Custom threat intelligence set
 ThreatIntelSet:
 Type: AWS::GuardDuty::ThreatIntelSet
 Properties:
 DetectorId: !Ref GuardDutyDetector
 Format: TXT
 Location: !Sub "https://${ThreatIntelBucket}.s3.amazonaws.com/threat-intel.txt"
 Name: CustomThreatIntelligence
 Activate: true

 # IP whitelist
 IPSet:
 Type: AWS::GuardDuty::IPSet
 Properties:
 DetectorId: !Ref GuardDutyDetector
 Format: TXT
 Location: !Sub "https://${ThreatIntelBucket}.s3.amazonaws.com/trusted-ips.txt"
 Name: TrustedIPAddresses
 Activate: true

 ThreatIntelBucket:
 Type: AWS::S3::Bucket
 Properties:
 BucketName: !Sub "${AWS::StackName}-threat-intel"
 BucketEncryption:
 ServerSideEncryptionConfiguration:
 - ServerSideEncryptionByDefault:
 SSEAlgorithm: AES256

Multi-Account Setup

# Master account configuration
Resources:
 GuardDutyMaster:
 Type: AWS::GuardDuty::Master
 Properties:
 DetectorId: !Ref GuardDutyDetector
 MasterId: !Ref AWS::AccountId
 InvitationId: !Ref InvitationId

 # Member account invitation
 GuardDutyMember:
 Type: AWS::GuardDuty::Member
 Properties:
 DetectorId: !Ref GuardDutyDetector
 MemberId: "123456789012"
 Email: "security@company.com"
 Message: "Please accept GuardDuty invitation"
 Status: Invited
 DisableEmailNotification: false

 # Organization configuration (for AWS Organizations)
 OrganizationConfiguration:
 Type: AWS::GuardDuty::OrganizationConfiguration
 Properties:
 DetectorId: !Ref GuardDutyDetector
 AutoEnable: true
 DataSources:
 S3Logs:
 AutoEnable: true
 KubernetesConfiguration:
 AuditLogs:
 AutoEnable: true
 MalwareProtection:
 ScanEc2InstanceWithFindings:
 EbsVolumes:
 AutoEnable: true

3. Finding Types and Analysis

Common Finding Types

# GuardDuty finding categories and examples
GUARDDUTY_FINDING_TYPES = {
 "Backdoor": [
 "Backdoor:EC2/C&CActivity.B!DNS",
 "Backdoor:EC2/DenialOfService.Dns",
 "Backdoor:EC2/Spambot"
 ],
 "Behavior": [
 "Behavior:EC2/NetworkPortUnusual",
 "Behavior:EC2/TrafficVolumeUnusual"
 ],
 "CryptoCurrency": [
 "CryptoCurrency:EC2/BitcoinTool.B!DNS",
 "CryptoCurrency:EC2/BitcoinTool.B"
 ],
 "Malware": [
 "Malware:EC2/SuspiciousFile",
 "Malware:Runtime/SuspiciousFile"
 ],
 "Persistence": [
 "Persistence:IAMUser/NetworkPermissions",
 "Persistence:IAMUser/ResourcePermissions"
 ],
 "Policy": [
 "Policy:IAMUser/RootCredentialUsage",
 "Policy:S3/BucketBlockPublicAccessDisabled"
 ],
 "PrivilegeEscalation": [
 "PrivilegeEscalation:IAMUser/AdministrativePermissions",
 "PrivilegeEscalation:Kubernetes/PrivilegedContainer"
 ],
 "Recon": [
 "Recon:EC2/PortProbeUnprotectedPort",
 "Recon:IAMUser/NetworkPermissions"
 ],
 "ResourceConsumption": [
 "ResourceConsumption:IAMUser/ComputeResources",
 "ResourceConsumption:IAMUser/ComputeResources"
 ],
 "Stealth": [
 "Stealth:IAMUser/CloudTrailLoggingDisabled",
 "Stealth:S3/ServerAccessLoggingDisabled"
 ],
 "Trojan": [
 "Trojan:EC2/BlackholeTraffic",
 "Trojan:EC2/DropPoint"
 ],
 "UnauthorizedAccess": [
 "UnauthorizedAccess:EC2/SSHBruteForce",
 "UnauthorizedAccess:IAMUser/ConsoleLoginSuccess.B"
 ]
}

Finding Analysis and Processing

import boto3
import json
from datetime import datetime, timedelta

class GuardDutyAnalyzer:
 def __init__(self):
 self.guardduty_client = boto3.client('guardduty')
 self.sns_client = boto3.client('sns')
 self.lambda_client = boto3.client('lambda')
 self.ec2_client = boto3.client('ec2')
 
 def get_detector_id(self):
 """Get the GuardDuty detector ID"""
 detectors = self.guardduty_client.list_detectors()
 if detectors['DetectorIds']:
 return detectors['DetectorIds'][0]
 return None
 
 def analyze_findings(self, hours_back=24):
 """Analyze GuardDuty findings from the last N hours"""
 
 detector_id = self.get_detector_id()
 if not detector_id:
 return {'error': 'No GuardDuty detector found'}
 
 # Calculate time range
 end_time = datetime.utcnow()
 start_time = end_time - timedelta(hours=hours_back)
 
 # Get findings
 findings_response = self.guardduty_client.list_findings(
 DetectorId=detector_id,
 FindingCriteria={
 'Criterion': {
 'updatedAt': {
 'Gte': int(start_time.timestamp() * 1000),
 'Lte': int(end_time.timestamp() * 1000)
 }
 }
 }
 )
 
 if not findings_response['FindingIds']:
 return {'message': 'No findings in the specified time range'}
 
 # Get detailed finding information
 findings_details = self.guardduty_client.get_findings(
 DetectorId=detector_id,
 FindingIds=findings_response['FindingIds']
 )
 
 # Analyze findings
 analysis = {
 'total_findings': len(findings_details['Findings']),
 'severity_breakdown': {'Low': 0, 'Medium': 0, 'High': 0, 'Critical': 0},
 'type_breakdown': {},
 'affected_resources': set(),
 'critical_findings': [],
 'recommendations': []
 }
 
 for finding in findings_details['Findings']:
 # Count by severity
 severity = self.get_severity_label(finding['Severity'])
 analysis['severity_breakdown'][severity] += 1
 
 # Count by type
 finding_type = finding['Type']
 analysis['type_breakdown'][finding_type] = analysis['type_breakdown'].get(finding_type, 0) + 1
 
 # Track affected resources
 if 'Resource' in finding:
 resource_type = finding['Resource']['ResourceType']
 analysis['affected_resources'].add(resource_type)
 
 # Identify critical findings
 if finding['Severity'] >= 9.0: # Critical severity
 analysis['critical_findings'].append({
 'id': finding['Id'],
 'type': finding['Type'],
 'severity': finding['Severity'],
 'title': finding['Title'],
 'description': finding['Description'],
 'resource': finding.get('Resource', {})
 })
 
 # Convert set to list for JSON serialization
 analysis['affected_resources'] = list(analysis['affected_resources'])
 
 # Generate recommendations
 analysis['recommendations'] = self.generate_recommendations(analysis)
 
 return analysis
 
 def get_severity_label(self, severity_score):
 """Convert severity score to label"""
 if severity_score >= 9.0:
 return 'Critical'
 elif severity_score >= 7.0:
 return 'High'
 elif severity_score >= 4.0:
 return 'Medium'
 else:
 return 'Low'
 
 def generate_recommendations(self, analysis):
 """Generate security recommendations based on findings"""
 
 recommendations = []
 
 # Check for high-severity findings
 if analysis['critical_findings']:
 recommendations.append({
 'priority': 'CRITICAL',
 'action': 'Immediate investigation required for high-severity findings',
 'details': f"Found {len(analysis['critical_findings'])} critical security findings"
 })
 
 # Check for specific threat types
 for finding_type, count in analysis['type_breakdown'].items():
 if 'CryptoCurrency' in finding_type:
 recommendations.append({
 'priority': 'HIGH',
 'action': 'Investigate cryptocurrency mining activity',
 'details': f"Detected {count} cryptocurrency-related findings"
 })
 
 elif 'Backdoor' in finding_type:
 recommendations.append({
 'priority': 'CRITICAL',
 'action': 'Investigate potential backdoor or C&C communication',
 'details': f"Detected {count} backdoor-related findings"
 })
 
 elif 'UnauthorizedAccess' in finding_type:
 recommendations.append({
 'priority': 'HIGH',
 'action': 'Review access controls and authentication mechanisms',
 'details': f"Detected {count} unauthorized access attempts"
 })
 
 return recommendations
 
 def create_automated_response(self, finding):
 """Create automated response for specific finding types"""
 
 finding_type = finding['Type']
 resource = finding.get('Resource', {})
 
 response_actions = []
 
 # Automated responses based on finding type
 if 'CryptoCurrency' in finding_type and resource.get('ResourceType') == 'Instance':
 # Isolate instance for cryptocurrency mining
 instance_id = resource['InstanceDetails']['InstanceId']
 response_actions.append(self.isolate_instance(instance_id))
 
 elif 'UnauthorizedAccess:EC2/SSHBruteForce' in finding_type:
 # Block source IP in security group
 source_ip = finding['Service']['RemoteIpDetails']['IpAddressV4']
 response_actions.append(self.block_ip_address(source_ip, resource))
 
 elif 'Stealth:IAMUser/CloudTrailLoggingDisabled' in finding_type:
 # Re-enable CloudTrail logging
 response_actions.append(self.enable_cloudtrail_logging())
 
 elif 'Policy:S3/BucketBlockPublicAccessDisabled' in finding_type:
 # Enable S3 bucket public access block
 bucket_name = resource['S3BucketDetails'][0]['Name']
 response_actions.append(self.enable_s3_public_access_block(bucket_name))
 
 return response_actions
 
 def isolate_instance(self, instance_id):
 """Isolate EC2 instance by modifying security groups"""
 
 try:
 # Create isolation security group
 isolation_sg = self.ec2_client.create_security_group(
 GroupName=f'isolation-{instance_id}',
 Description='Isolation security group for compromised instance'
 )
 
 # Modify instance security groups
 self.ec2_client.modify_instance_attribute(
 InstanceId=instance_id,
 Groups=[isolation_sg['GroupId']]
 )
 
 return {
 'action': 'Instance Isolation',
 'status': 'SUCCESS',
 'details': f'Instance {instance_id} isolated with security group {isolation_sg["GroupId"]}'
 }
 
 except Exception as e:
 return {
 'action': 'Instance Isolation',
 'status': 'FAILED',
 'error': str(e)
 }
 
 def block_ip_address(self, ip_address, resource):
 """Block attacker IP: revoke any allow rules for it in the instance's
 security groups, then add an explicit DENY in the subnet NACL.
 (Security groups are allow-only - they have no deny rules.)"""

 try:
 instance_id = resource['InstanceDetails']['InstanceId']

 # Get instance security groups and subnet
 instance_response = self.ec2_client.describe_instances(InstanceIds=[instance_id])
 instance = instance_response['Reservations'][0]['Instances'][0]
 security_groups = instance['SecurityGroups']
 subnet_id = instance['SubnetId']

 revoked = []
 # Revoke any ingress rule that currently allows the attacker IP
 for sg in security_groups:
 sg_details = self.ec2_client.describe_security_groups(
 GroupIds=[sg['GroupId']]
 )['SecurityGroups'][0]
 for perm in sg_details.get('IpPermissions', []):
 matching_ranges = [
 r for r in perm.get('IpRanges', [])
 if r.get('CidrIp') == f'{ip_address}/32'
 ]
 if matching_ranges:
 revoke_perm = dict(perm)
 revoke_perm['IpRanges'] = matching_ranges
 self.ec2_client.revoke_security_group_ingress(
 GroupId=sg['GroupId'],
 IpPermissions=[revoke_perm]
 )
 revoked.append(sg['GroupId'])

 # Explicit DENY via the subnet network ACL (NACLs support deny rules;
 # lowest rule number wins, so a low number like 90 takes precedence)
 nacls = self.ec2_client.describe_network_acls(
 Filters=[{'Name': 'association.subnet-id', 'Values': [subnet_id]}]
 )['NetworkAcls']
 for nacl in nacls:
 self.ec2_client.create_network_acl_entry(
 NetworkAclId=nacl['NetworkAclId'],
 RuleNumber=90,
 Protocol='-1',
 RuleAction='deny',
 Egress=False,
 CidrBlock=f'{ip_address}/32'
 )

 return {
 'action': 'IP Address Block',
 'status': 'SUCCESS',
 'details': f'Revoked attacker IP {ip_address} from {len(revoked)} security group(s); NACL deny added'
 }

 except Exception as e:
 return {
 'action': 'IP Address Block',
 'status': 'FAILED',
 'error': str(e)
 }

4. Automated Response and Remediation

Lambda-based Automated Response

# Lambda function for GuardDuty automated response
import json
import boto3
from datetime import datetime

def lambda_handler(event, context):
 """
 Automated response to GuardDuty findings
 Triggered by CloudWatch Events when GuardDuty creates findings
 """
 
 # Parse GuardDuty finding from CloudWatch Event
 finding = event['detail']
 
 finding_type = finding['type']
 severity = finding['severity']
 resource = finding.get('resource', {})
 
 print(f"Processing GuardDuty finding: {finding_type} (Severity: {severity})")
 
 # Initialize AWS clients
 ec2 = boto3.client('ec2')
 iam = boto3.client('iam')
 sns = boto3.client('sns')
 ssm = boto3.client('ssm')
 
 response_actions = []
 
 try:
 # High-severity findings require immediate action
 if severity >= 7.0:
 response_actions.extend(handle_critical_finding(finding, ec2, iam, ssm))
 
 # Medium-severity findings require investigation
 elif severity >= 4.0:
 response_actions.extend(handle_medium_finding(finding, ec2, sns))
 
 # Low-severity findings are logged for analysis
 else:
 response_actions.append(log_finding_for_analysis(finding))
 
 # Send notification
 send_notification(finding, response_actions, sns)
 
 return {
 'statusCode': 200,
 'body': json.dumps({
 'finding_id': finding['id'],
 'actions_taken': len(response_actions),
 'response_actions': response_actions
 })
 }
 
 except Exception as e:
 print(f"Error processing GuardDuty finding: {str(e)}")
 
 # Send error notification
 sns.publish(
 TopicArn='arn:aws:sns:us-east-1:123456789012:security-alerts',
 Message=f"Error processing GuardDuty finding {finding['id']}: {str(e)}",
 Subject='GuardDuty Automated Response Error'
 )
 
 return {
 'statusCode': 500,
 'body': json.dumps({'error': str(e)})
 }

def handle_critical_finding(finding, ec2, iam, ssm):
 """Handle critical severity findings with immediate automated response"""
 
 actions = []
 finding_type = finding['type']
 resource = finding.get('resource', {})
 
 # Cryptocurrency mining detection
 if 'CryptoCurrency' in finding_type:
 if resource.get('resourceType') == 'Instance':
 instance_id = resource['instanceDetails']['instanceId']
 
 # Stop the instance immediately
 try:
 ec2.stop_instances(InstanceIds=[instance_id])
 actions.append({
 'action': 'Stop Instance',
 'resource': instance_id,
 'status': 'SUCCESS',
 'reason': 'Cryptocurrency mining detected'
 })
 except Exception as e:
 actions.append({
 'action': 'Stop Instance',
 'resource': instance_id,
 'status': 'FAILED',
 'error': str(e)
 })
 
 # Backdoor or C&C communication
 elif 'Backdoor' in finding_type:
 if resource.get('resourceType') == 'Instance':
 instance_id = resource['instanceDetails']['instanceId']
 
 # Isolate instance by creating restrictive security group
 try:
 isolation_sg = create_isolation_security_group(ec2, instance_id)
 
 ec2.modify_instance_attribute(
 InstanceId=instance_id,
 Groups=[isolation_sg]
 )
 
 actions.append({
 'action': 'Isolate Instance',
 'resource': instance_id,
 'security_group': isolation_sg,
 'status': 'SUCCESS',
 'reason': 'Backdoor communication detected'
 })
 except Exception as e:
 actions.append({
 'action': 'Isolate Instance',
 'resource': instance_id,
 'status': 'FAILED',
 'error': str(e)
 })
 
 # Compromised IAM credentials
 elif 'UnauthorizedAccess:IAMUser' in finding_type:
 user_name = resource.get('accessKeyDetails', {}).get('userName')
 
 if user_name:
 try:
 # Disable user's access keys
 access_keys = iam.list_access_keys(UserName=user_name)
 
 for key in access_keys['AccessKeyMetadata']:
 iam.update_access_key(
 UserName=user_name,
 AccessKeyId=key['AccessKeyId'],
 Status='Inactive'
 )
 
 actions.append({
 'action': 'Disable Access Keys',
 'resource': user_name,
 'status': 'SUCCESS',
 'reason': 'Unauthorized access detected'
 })
 except Exception as e:
 actions.append({
 'action': 'Disable Access Keys',
 'resource': user_name,
 'status': 'FAILED',
 'error': str(e)
 })
 
 # CloudTrail logging disabled
 elif 'Stealth:IAMUser/CloudTrailLoggingDisabled' in finding_type:
 try:
 # Re-enable CloudTrail logging via Systems Manager
 ssm.send_command(
 DocumentName='AWS-ConfigureCloudTrail',
 Parameters={
 'trailName': ['SecurityAuditTrail'],
 's3BucketName': ['security-audit-logs'],
 'includeGlobalServiceEvents': ['true'],
 'isMultiRegionTrail': ['true']
 },
 Targets=[
 {
 'Key': 'tag:Role',
 'Values': ['SecurityManagement']
 }
 ]
 )
 
 actions.append({
 'action': 'Re-enable CloudTrail',
 'status': 'SUCCESS',
 'reason': 'CloudTrail logging was disabled'
 })
 except Exception as e:
 actions.append({
 'action': 'Re-enable CloudTrail',
 'status': 'FAILED',
 'error': str(e)
 })
 
 return actions

def handle_medium_finding(finding, ec2, sns):
 """Handle medium severity findings with investigation and monitoring"""
 
 actions = []
 finding_type = finding['type']
 
 # SSH brute force attacks
 if 'UnauthorizedAccess:EC2/SSHBruteForce' in finding_type:
 source_ip = finding['service']['remoteIpDetails']['ipAddressV4']
 
 # Add IP to threat intelligence for monitoring
 actions.append({
 'action': 'Monitor IP Address',
 'ip_address': source_ip,
 'status': 'SUCCESS',
 'reason': 'SSH brute force detected'
 })
 
 # Send detailed alert
 sns.publish(
 TopicArn='arn:aws:sns:us-east-1:123456789012:security-investigations',
 Message=f"SSH brute force attack detected from {source_ip}. Please investigate and consider blocking this IP.",
 Subject='Security Investigation Required - SSH Brute Force'
 )
 
 # Unusual network behavior
 elif 'Behavior:EC2' in finding_type:
 instance_id = finding['resource']['instanceDetails']['instanceId']
 
 # Enable detailed monitoring
 try:
 ec2.monitor_instances(InstanceIds=[instance_id])
 
 actions.append({
 'action': 'Enable Detailed Monitoring',
 'resource': instance_id,
 'status': 'SUCCESS',
 'reason': 'Unusual network behavior detected'
 })
 except Exception as e:
 actions.append({
 'action': 'Enable Detailed Monitoring',
 'resource': instance_id,
 'status': 'FAILED',
 'error': str(e)
 })
 
 return actions

def create_isolation_security_group(ec2, instance_id):
 """Create isolation security group for compromised instances"""
 
 # Get VPC ID from instance
 instance_response = ec2.describe_instances(InstanceIds=[instance_id])
 vpc_id = instance_response['Reservations'][0]['Instances'][0]['VpcId']
 
 # Create isolation security group
 sg_response = ec2.create_security_group(
 GroupName=f'isolation-{instance_id}-{int(datetime.now().timestamp())}',
 Description=f'Isolation security group for compromised instance {instance_id}',
 VpcId=vpc_id
 )
 
 # Add minimal egress rules (only for investigation)
 ec2.authorize_security_group_egress(
 GroupId=sg_response['GroupId'],
 IpPermissions=[
 {
 'IpProtocol': 'tcp',
 'FromPort': 443,
 'ToPort': 443,
 'IpRanges': [{'CidrIp': '0.0.0.0/0', 'Description': 'HTTPS for investigation tools'}]
 }
 ]
 )
 
 return sg_response['GroupId']

def send_notification(finding, actions, sns):
 """Send notification about GuardDuty finding and response actions"""
 
 message = f"""
 GuardDuty Security Finding
 =========================
 
 Finding ID: {finding['id']}
 Type: {finding['type']}
 Severity: {finding['severity']}
 Title: {finding['title']}
 Description: {finding['description']}
 
 Automated Response Actions:
 """
 
 for action in actions:
 message += f"\n- {action['action']}: {action['status']}"
 if action['status'] == 'FAILED':
 message += f" (Error: {action.get('error', 'Unknown')})"
 
 message += f"\n\nTimestamp: {datetime.utcnow().isoformat()}"
 
 # Determine topic based on severity
 if finding['severity'] >= 7.0:
 topic_arn = 'arn:aws:sns:us-east-1:123456789012:critical-security-alerts'
 subject = f"CRITICAL: GuardDuty Finding - {finding['type']}"
 else:
 topic_arn = 'arn:aws:sns:us-east-1:123456789012:security-alerts'
 subject = f"GuardDuty Finding - {finding['type']}"
 
 sns.publish(
 TopicArn=topic_arn,
 Message=message,
 Subject=subject
 )

CloudWatch Events Integration

# CloudFormation for GuardDuty event processing
Resources:
 GuardDutyEventRule:
 Type: AWS::Events::Rule
 Properties:
 Name: GuardDutyFindingProcessor
 Description: Process GuardDuty findings for automated response
 EventPattern:
 source:
 - aws.guardduty
 detail-type:
 - GuardDuty Finding
 detail:
 severity:
 - numeric:
 - ">="
 - 4.0
 State: ENABLED
 Targets:
 - Arn: !GetAtt GuardDutyResponseFunction.Arn
 Id: GuardDutyResponseTarget
 - Arn: !Ref SecurityAlertsTopic
 Id: SecurityAlertsTarget
 InputTransformer:
 InputPathsMap:
 severity: "$.detail.severity"
 type: "$.detail.type"
 title: "$.detail.title"
 InputTemplate: |
 {
 "severity": "<severity>",
 "finding_type": "<type>",
 "title": "<title>",
 "message": "GuardDuty finding requires attention"
 }

 # High-severity findings trigger immediate response
 CriticalFindingRule:
 Type: AWS::Events::Rule
 Properties:
 Name: GuardDutyCriticalFindings
 Description: Process critical GuardDuty findings
 EventPattern:
 source:
 - aws.guardduty
 detail-type:
 - GuardDuty Finding
 detail:
 severity:
 - numeric:
 - ">="
 - 7.0
 State: ENABLED
 Targets:
 - Arn: !GetAtt CriticalResponseFunction.Arn
 Id: CriticalResponseTarget
 - Arn: !Ref CriticalAlertsTopic
 Id: CriticalAlertsTarget

 # Lambda permission for EventBridge
 GuardDutyLambdaPermission:
 Type: AWS::Lambda::Permission
 Properties:
 FunctionName: !Ref GuardDutyResponseFunction
 Action: lambda:InvokeFunction
 Principal: events.amazonaws.com
 SourceArn: !GetAtt GuardDutyEventRule.Arn

5. Integration with Security Hub

Security Hub Integration

Resources:
 SecurityHubIntegration:
 Type: AWS::SecurityHub::Hub
 Properties:
 Tags:
 - Key: Purpose
 Value: CentralizedSecurityMonitoring

 # Enable GuardDuty integration with Security Hub
 GuardDutyProductSubscription:
 Type: AWS::SecurityHub::ProductSubscription
 Properties:
 ProductArn: !Sub "arn:aws:securityhub:${AWS::Region}::product/aws/guardduty"
 DependsOn: SecurityHubIntegration

 # Custom insight for GuardDuty findings
 GuardDutyInsight:
 Type: AWS::SecurityHub::Insight
 Properties:
 Name: GuardDuty High Severity Findings
 Description: High severity findings from GuardDuty
 GroupByAttribute: ProductFields/aws/guardduty/service/resourceRole
 Filters:
 ProductName:
 - Value: GuardDuty
 Comparison: EQUALS
 SeverityLabel:
 - Value: HIGH
 Comparison: EQUALS
 - Value: CRITICAL
 Comparison: EQUALS
 RecordState:
 - Value: ACTIVE
 Comparison: EQUALS

6. Malware Protection

EBS Volume Scanning

Resources:
 MalwareProtectionConfiguration:
 Type: AWS::GuardDuty::Detector
 Properties:
 Enable: true
 DataSources:
 MalwareProtection:
 ScanEc2InstanceWithFindings:
 EbsVolumes: true

 # Lambda function to handle malware findings
 MalwareResponseFunction:
 Type: AWS::Lambda::Function
 Properties:
 FunctionName: guardduty-malware-response
 Runtime: python3.9
 Handler: index.lambda_handler
 Role: !GetAtt MalwareResponseRole.Arn
 Code:
 ZipFile: |
 import boto3
 import json
 
 def lambda_handler(event, context):
 """Handle GuardDuty malware detection findings"""
 
 finding = event['detail']
 
 if 'Malware' in finding['type']:
 ec2 = boto3.client('ec2')
 ssm = boto3.client('ssm')
 
 instance_id = finding['resource']['instanceDetails']['instanceId']
 
 # Immediate isolation
 isolate_instance(ec2, instance_id)
 
 # Create forensic snapshot
 create_forensic_snapshot(ec2, instance_id)
 
 # Run malware removal if safe
 if finding['service']['additionalInfo']['threatListName'] != 'CRITICAL':
 run_malware_removal(ssm, instance_id)
 
 return {'statusCode': 200}
 
 def isolate_instance(ec2, instance_id):
 """Isolate infected instance"""
 
 # Create isolation security group
 vpc_response = ec2.describe_instances(InstanceIds=[instance_id])
 vpc_id = vpc_response['Reservations'][0]['Instances'][0]['VpcId']
 
 sg_response = ec2.create_security_group(
 GroupName=f'malware-isolation-{instance_id}',
 Description='Isolation for malware-infected instance',
 VpcId=vpc_id
 )
 
 # Apply isolation security group
 ec2.modify_instance_attribute(
 InstanceId=instance_id,
 Groups=[sg_response['GroupId']]
 )
 
 def create_forensic_snapshot(ec2, instance_id):
 """Create forensic snapshot of infected instance"""
 
 # Get instance volumes
 instance_response = ec2.describe_instances(InstanceIds=[instance_id])
 volumes = []
 
 for reservation in instance_response['Reservations']:
 for instance in reservation['Instances']:
 for bdm in instance['BlockDeviceMappings']:
 volumes.append(bdm['Ebs']['VolumeId'])
 
 # Create snapshots
 for volume_id in volumes:
 ec2.create_snapshot(
 VolumeId=volume_id,
 Description=f'Forensic snapshot for malware investigation - {instance_id}',
 TagSpecifications=[
 {
 'ResourceType': 'snapshot',
 'Tags': [
 {'Key': 'Purpose', 'Value': 'Forensic'},
 {'Key': 'SourceInstance', 'Value': instance_id},
 {'Key': 'Reason', 'Value': 'Malware Detection'}
 ]
 }
 ]
 )
 
 def run_malware_removal(ssm, instance_id):
 """Run malware removal tools"""
 
 ssm.send_command(
 InstanceIds=[instance_id],
 DocumentName='AWS-RunShellScript',
 Parameters={
 'commands': [
 '#!/bin/bash',
 'yum update -y',
 'yum install -y clamav clamav-update',
 'freshclam',
 'clamscan -r --remove /home /tmp /var/tmp',
 'echo "Malware scan completed"'
 ]
 }
 )

 MalwareEventRule:
 Type: AWS::Events::Rule
 Properties:
 Name: GuardDutyMalwareFindings
 EventPattern:
 source:
 - aws.guardduty
 detail-type:
 - GuardDuty Finding
 detail:
 type:
 - prefix: "Malware:"
 Targets:
 - Arn: !GetAtt MalwareResponseFunction.Arn
 Id: MalwareResponseTarget

7. Common Exam Scenarios

Scenario 1: Multi-Account GuardDuty Setup

# Complete multi-account GuardDuty setup
import boto3
import json

class MultiAccountGuardDutySetup:
 def __init__(self):
 self.guardduty_client = boto3.client('guardduty')
 self.organizations_client = boto3.client('organizations')
 self.sts_client = boto3.client('sts')
 
 def setup_organization_guardduty(self):
 """Setup GuardDuty across AWS Organization"""
 
 # Enable GuardDuty in master account
 master_detector = self.enable_guardduty_master()
 
 # Get organization accounts
 accounts = self.get_organization_accounts()
 
 # Enable GuardDuty in member accounts
 member_results = {}
 
 for account in accounts:
 if account['Status'] == 'ACTIVE' and account['Id'] != self.get_master_account_id():
 result = self.enable_guardduty_member(account['Id'], account['Email'])
 member_results[account['Id']] = result
 
 # Configure organization settings
 org_config = self.configure_organization_settings(master_detector['DetectorId'])
 
 return {
 'master_detector': master_detector,
 'member_results': member_results,
 'organization_config': org_config
 }
 
 def enable_guardduty_master(self):
 """Enable GuardDuty in master account"""
 
 try:
 # Check if detector already exists
 detectors = self.guardduty_client.list_detectors()
 
 if detectors['DetectorIds']:
 detector_id = detectors['DetectorIds'][0]
 
 # Update detector configuration
 self.guardduty_client.update_detector(
 DetectorId=detector_id,
 Enable=True,
 FindingPublishingFrequency='FIFTEEN_MINUTES',
 DataSources={
 'S3Logs': {'Enable': True},
 'KubernetesConfiguration': {
 'AuditLogs': {'Enable': True}
 },
 'MalwareProtection': {
 'ScanEc2InstanceWithFindings': {
 'EbsVolumes': True
 }
 }
 }
 )
 else:
 # Create new detector
 response = self.guardduty_client.create_detector(
 Enable=True,
 FindingPublishingFrequency='FIFTEEN_MINUTES',
 DataSources={
 'S3Logs': {'Enable': True},
 'KubernetesConfiguration': {
 'AuditLogs': {'Enable': True}
 },
 'MalwareProtection': {
 'ScanEc2InstanceWithFindings': {
 'EbsVolumes': True
 }
 }
 }
 )
 detector_id = response['DetectorId']
 
 return {
 'DetectorId': detector_id,
 'Status': 'SUCCESS'
 }
 
 except Exception as e:
 return {
 'Status': 'FAILED',
 'Error': str(e)
 }
 
 def enable_guardduty_member(self, account_id, email):
 """Enable GuardDuty in member account"""
 
 try:
 detector_id = self.guardduty_client.list_detectors()['DetectorIds'][0]
 
 # Create member
 self.guardduty_client.create_members(
 DetectorId=detector_id,
 AccountDetails=[
 {
 'AccountId': account_id,
 'Email': email
 }
 ]
 )
 
 # Invite member
 self.guardduty_client.invite_members(
 DetectorId=detector_id,
 AccountIds=[account_id],
 Message='Please accept GuardDuty invitation for centralized security monitoring'
 )
 
 return {
 'Status': 'INVITED',
 'AccountId': account_id
 }
 
 except Exception as e:
 return {
 'Status': 'FAILED',
 'AccountId': account_id,
 'Error': str(e)
 }
 
 def configure_organization_settings(self, detector_id):
 """Configure organization-wide GuardDuty settings"""
 
 try:
 # Enable organization configuration
 self.guardduty_client.update_organization_configuration(
 DetectorId=detector_id,
 AutoEnable=True,
 DataSources={
 'S3Logs': {'AutoEnable': True},
 'KubernetesConfiguration': {
 'AuditLogs': {'AutoEnable': True}
 },
 'MalwareProtection': {
 'ScanEc2InstanceWithFindings': {
 'EbsVolumes': {'AutoEnable': True}
 }
 }
 }
 )
 
 return {
 'Status': 'SUCCESS',
 'AutoEnable': True
 }
 
 except Exception as e:
 return {
 'Status': 'FAILED',
 'Error': str(e)
 }
 
 def get_organization_accounts(self):
 """Get all accounts in the organization"""
 
 accounts = []
 paginator = self.organizations_client.get_paginator('list_accounts')
 
 for page in paginator.paginate():
 accounts.extend(page['Accounts'])
 
 return accounts
 
 def get_master_account_id(self):
 """Get master account ID"""
 
 return self.sts_client.get_caller_identity()['Account']

Scenario 2: Threat Intelligence Integration

# Threat intelligence integration setup
Resources:
 ThreatIntelligenceBucket:
 Type: AWS::S3::Bucket
 Properties:
 BucketName: !Sub "${AWS::StackName}-threat-intel"
 BucketEncryption:
 ServerSideEncryptionConfiguration:
 - ServerSideEncryptionByDefault:
 SSEAlgorithm: AES256

 # Lambda function to update threat intelligence
 ThreatIntelUpdateFunction:
 Type: AWS::Lambda::Function
 Properties:
 FunctionName: guardduty-threat-intel-updater
 Runtime: python3.9
 Handler: index.lambda_handler
 Role: !GetAtt ThreatIntelRole.Arn
 Code:
 ZipFile: |
 import boto3
 import requests # NOTE: 'requests' is NOT in the Lambda Python runtime - bundle it via a Lambda layer
 import json
 
 def lambda_handler(event, context):
 """Update GuardDuty threat intelligence from external sources"""
 
 s3 = boto3.client('s3')
 guardduty = boto3.client('guardduty')
 
 # Download threat intelligence feeds
 threat_feeds = [
 'https://rules.emergingthreats.net/open/suricata/rules/compromised-ips.txt',
 'https://www.spamhaus.org/drop/drop.txt'
 ]
 
 combined_threats = []
 
 for feed_url in threat_feeds:
 try:
 response = requests.get(feed_url, timeout=30)
 if response.status_code == 200:
 # Parse threat indicators
 threats = parse_threat_feed(response.text, feed_url)
 combined_threats.extend(threats)
 except Exception as e:
 print(f"Error downloading {feed_url}: {str(e)}")
 
 # Upload to S3
 threat_intel_content = '\n'.join(combined_threats)
 
 s3.put_object(
 Bucket=os.environ['THREAT_INTEL_BUCKET'],
 Key='threat-intel.txt',
 Body=threat_intel_content,
 ContentType='text/plain'
 )
 
 # Update GuardDuty threat intelligence set
 detector_id = guardduty.list_detectors()['DetectorIds'][0]
 
 try:
 guardduty.update_threat_intel_set(
 DetectorId=detector_id,
 ThreatIntelSetId=os.environ['THREAT_INTEL_SET_ID'],
 Location=f"https://{os.environ['THREAT_INTEL_BUCKET']}.s3.amazonaws.com/threat-intel.txt",
 Activate=True
 )
 except Exception as e:
 print(f"Error updating threat intelligence set: {str(e)}")
 
 return {
 'statusCode': 200,
 'body': json.dumps(f'Updated {len(combined_threats)} threat indicators')
 }
 
 def parse_threat_feed(content, source):
 """Parse threat intelligence feed content"""
 
 threats = []
 
 for line in content.split('\n'):
 line = line.strip()
 
 # Skip comments and empty lines
 if not line or line.startswith('#') or line.startswith(';'):
 continue
 
 # Extract IP addresses (simple regex)
 import re
 ip_pattern = r'\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b'
 
 matches = re.findall(ip_pattern, line)
 for ip in matches:
 if is_valid_ip(ip):
 threats.append(ip)
 
 return threats
 
 def is_valid_ip(ip):
 """Validate IP address"""
 
 try:
 parts = ip.split('.')
 return len(parts) == 4 and all(0 <= int(part) <= 255 for part in parts)
 except:
 return False
 Environment:
 Variables:
 THREAT_INTEL_BUCKET: !Ref ThreatIntelligenceBucket
 THREAT_INTEL_SET_ID: !Ref ThreatIntelSet

 # Schedule threat intelligence updates
 ThreatIntelUpdateSchedule:
 Type: AWS::Events::Rule
 Properties:
 Description: Update threat intelligence daily
 ScheduleExpression: rate(1 day)
 State: ENABLED
 Targets:
 - Arn: !GetAtt ThreatIntelUpdateFunction.Arn
 Id: ThreatIntelUpdateTarget

8. Exam Tips

Understand finding types - Know major categories and what they indicate
Master automated response - Lambda integration with CloudWatch Events
Know multi-account setup - Organizations integration and member management
Practice threat intelligence - Custom threat intel sets and IP sets
Learn malware protection - EBS volume scanning and response workflows
Understand data sources - VPC Flow Logs, DNS logs, CloudTrail integration
Know Security Hub integration - Centralized security findings management
Practice cost optimization - Finding frequency, data source configuration
Master troubleshooting - Common issues with detectors and member accounts
Understand compliance - SOC, PCI DSS, and other compliance frameworks

AWS Inspector

Sun, 04 Oct 2026 00:00:00 +0000

1. Overview

AWS Inspector is an automated security assessment service that helps improve the security and compliance of applications deployed on AWS. It automatically assesses applications for exposure, vulnerabilities, and deviations from best practices.

Key Characteristics

Automated vulnerability assessment - Continuous scanning of EC2 instances and container images
Network reachability analysis - Identifies network paths that allow access to instances
Software vulnerability assessment - Scans for known vulnerabilities in installed packages
Security best practices - Checks against CIS benchmarks and security standards
Integration with CI/CD - Automated scanning in deployment pipelines
Centralized findings - Integration with Security Hub for unified security management
Continuous monitoring - Ongoing assessment as infrastructure changes

What Problem Does It Solve?

Identifies security vulnerabilities before they can be exploited
Provides continuous security assessment without manual intervention
Ensures compliance with security standards and best practices
Integrates security scanning into DevOps workflows
Reduces time to identify and remediate security issues
Provides detailed remediation guidance for found vulnerabilities

2. Inspector V2 (Current Version)

Assessment Types

# CloudFormation for Inspector V2 setup
Resources:
 InspectorConfiguration:
 Type: AWS::Inspector2::EnabledConfiguration
 Properties:
 AccountId: !Ref AWS::AccountId
 ResourceTypes:
 - EC2
 - ECR
 AutoEnable:
 Ec2: true
 Ecr: true

 # Organization-wide Inspector setup
 InspectorOrganizationConfiguration:
 Type: AWS::Inspector2::OrganizationConfiguration
 Properties:
 AutoEnable:
 Ec2: true
 Ecr: true

Programmatic Configuration

import boto3
import json
from datetime import datetime, timedelta

class InspectorManager:
 def __init__(self):
 self.inspector2_client = boto3.client('inspector2')
 self.ec2_client = boto3.client('ec2')
 self.ecr_client = boto3.client('ecr')
 self.organizations_client = boto3.client('organizations')
 
 def enable_inspector(self, resource_types=['EC2', 'ECR']):
 """Enable Inspector for specified resource types"""
 
 try:
 response = self.inspector2_client.enable(
 resourceTypes=resource_types,
 accountIds=[boto3.Session().get_credentials().access_key[:12]]
 )
 
 return {
 'status': 'SUCCESS',
 'enabled_resources': resource_types,
 'accounts': response.get('accounts', [])
 }
 
 except Exception as e:
 return {
 'status': 'FAILED',
 'error': str(e)
 }
 
 def get_findings_summary(self, days_back=7):
 """Get summary of Inspector findings"""
 
 # Calculate time range
 end_time = datetime.utcnow()
 start_time = end_time - timedelta(days=days_back)
 
 try:
 # Get findings
 findings_response = self.inspector2_client.list_findings(
 filterCriteria={
 'firstObservedAt': [
 {
 'startInclusive': start_time,
 'endInclusive': end_time
 }
 ]
 },
 maxResults=1000
 )
 
 # Analyze findings
 summary = {
 'total_findings': len(findings_response['findings']),
 'severity_breakdown': {
 'CRITICAL': 0,
 'HIGH': 0,
 'MEDIUM': 0,
 'LOW': 0,
 'INFORMATIONAL': 0
 },
 'resource_breakdown': {
 'EC2': 0,
 'ECR': 0
 },
 'top_vulnerabilities': {},
 'affected_resources': set()
 }
 
 for finding in findings_response['findings']:
 # Count by severity
 severity = finding.get('severity', 'INFORMATIONAL')
 summary['severity_breakdown'][severity] += 1
 
 # Count by resource type
 resource_type = finding.get('type', 'UNKNOWN')
 if 'PACKAGE_VULNERABILITY' in resource_type:
 if 'ec2-instance' in finding.get('resources', [{}])[0].get('id', ''):
 summary['resource_breakdown']['EC2'] += 1
 else:
 summary['resource_breakdown']['ECR'] += 1
 
 # Track vulnerabilities
 vuln_id = finding.get('packageVulnerabilityDetails', {}).get('vulnerabilityId', 'Unknown')
 summary['top_vulnerabilities'][vuln_id] = summary['top_vulnerabilities'].get(vuln_id, 0) + 1
 
 # Track affected resources
 for resource in finding.get('resources', []):
 summary['affected_resources'].add(resource.get('id', 'Unknown'))
 
 # Convert set to list for JSON serialization
 summary['affected_resources'] = list(summary['affected_resources'])
 
 # Get top 10 vulnerabilities
 summary['top_vulnerabilities'] = dict(
 sorted(summary['top_vulnerabilities'].items(), 
 key=lambda x: x[1], reverse=True)[:10]
 )
 
 return summary
 
 except Exception as e:
 return {
 'error': str(e),
 'status': 'FAILED'
 }
 
 def get_vulnerability_details(self, vulnerability_id):
 """Get detailed information about a specific vulnerability"""
 
 try:
 findings_response = self.inspector2_client.list_findings(
 filterCriteria={
 'packageVulnerabilityDetails': {
 'vulnerabilityId': [vulnerability_id]
 }
 }
 )
 
 if not findings_response['findings']:
 return {'error': 'Vulnerability not found'}
 
 finding = findings_response['findings'][0]
 vuln_details = finding.get('packageVulnerabilityDetails', {})
 
 return {
 'vulnerability_id': vulnerability_id,
 'severity': finding.get('severity'),
 'title': finding.get('title'),
 'description': finding.get('description'),
 'cvss_score': vuln_details.get('cvss', {}).get('baseScore'),
 'cvss_vector': vuln_details.get('cvss', {}).get('scoringVector'),
 'affected_packages': [
 {
 'name': vuln_details.get('vulnerablePackages', [{}])[0].get('name'),
 'version': vuln_details.get('vulnerablePackages', [{}])[0].get('version'),
 'fixed_version': vuln_details.get('vulnerablePackages', [{}])[0].get('fixedInVersion')
 }
 ],
 'references': vuln_details.get('referenceUrls', []),
 'affected_resources': [resource.get('id') for resource in finding.get('resources', [])]
 }
 
 except Exception as e:
 return {
 'error': str(e),
 'status': 'FAILED'
 }
 
 def create_suppression_rule(self, rule_name, filter_criteria, reason):
 """Create suppression rule for specific findings"""
 
 try:
 response = self.inspector2_client.create_filter(
 name=rule_name,
 description=f"Suppression rule: {reason}",
 filterCriteria=filter_criteria,
 action='SUPPRESS',
 reason=reason
 )
 
 return {
 'status': 'SUCCESS',
 'filter_arn': response['arn']
 }
 
 except Exception as e:
 return {
 'status': 'FAILED',
 'error': str(e)
 }

3. EC2 Instance Assessment

Instance Configuration

# CloudFormation for Inspector-ready EC2 instances
Resources:
 InspectorRole:
 Type: AWS::IAM::Role
 Properties:
 RoleName: InspectorEC2Role
 AssumeRolePolicyDocument:
 Version: '2012-10-17'
 Statement:
 - Effect: Allow
 Principal:
 Service: ec2.amazonaws.com
 Action: sts:AssumeRole
 ManagedPolicyArns:
 - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
 Policies:
 - PolicyName: InspectorAccess
 PolicyDocument:
 Version: '2012-10-17'
 Statement:
 - Effect: Allow
 Action:
 - inspector2:ListFindings
 - inspector2:GetMember
 Resource: '*'

 InspectorInstanceProfile:
 Type: AWS::IAM::InstanceProfile
 Properties:
 Roles:
 - !Ref InspectorRole

 # EC2 instance with Inspector scanning enabled
 WebServerInstance:
 Type: AWS::EC2::Instance
 Properties:
 ImageId: ami-0abcdef1234567890  # Amazon Linux 2
 InstanceType: t3.medium
 IamInstanceProfile: !Ref InspectorInstanceProfile
 SecurityGroupIds:
 - !Ref WebServerSecurityGroup
 SubnetId: !Ref PrivateSubnet
 UserData:
 Fn::Base64: !Sub |
 #!/bin/bash
 yum update -y
 
 # Install SSM Agent (usually pre-installed on Amazon Linux 2)
 yum install -y amazon-ssm-agent
 systemctl enable amazon-ssm-agent
 systemctl start amazon-ssm-agent
 
 # Install Inspector agent (for Inspector Classic - not needed for Inspector V2)
 # wget https://inspector-agent.amazonaws.com/linux/latest/install
 # bash install
 
 # Install application packages
 yum install -y httpd mysql php
 systemctl enable httpd
 systemctl start httpd
 
 # Create sample application
 echo "<?php phpinfo(); ?>" > /var/www/html/info.php
 Tags:
 - Key: Name
 Value: Inspector-WebServer
 - Key: Environment
 Value: Production
 - Key: InspectorScan
 Value: Enabled

 WebServerSecurityGroup:
 Type: AWS::EC2::SecurityGroup
 Properties:
 GroupDescription: Security group for web server
 VpcId: !Ref VPC
 SecurityGroupIngress:
 - IpProtocol: tcp
 FromPort: 80
 ToPort: 80
 SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup
 - IpProtocol: tcp
 FromPort: 443
 ToPort: 443
 SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup
 SecurityGroupEgress:
 - IpProtocol: tcp
 FromPort: 443
 ToPort: 443
 CidrIp: 0.0.0.0/0
 Description: HTTPS outbound
 - IpProtocol: tcp
 FromPort: 80
 ToPort: 80
 CidrIp: 0.0.0.0/0
 Description: HTTP outbound

Network Reachability Assessment

# Network reachability analysis
import boto3
import json

class NetworkReachabilityAnalyzer:
 def __init__(self):
 self.inspector2_client = boto3.client('inspector2')
 self.ec2_client = boto3.client('ec2')
 
 def analyze_network_reachability(self, instance_ids=None):
 """Analyze network reachability for EC2 instances"""
 
 if not instance_ids:
 # Get all instances if none specified
 instances_response = self.ec2_client.describe_instances(
 Filters=[
 {'Name': 'instance-state-name', 'Values': ['running']}
 ]
 )
 
 instance_ids = []
 for reservation in instances_response['Reservations']:
 for instance in reservation['Instances']:
 instance_ids.append(instance['InstanceId'])
 
 reachability_analysis = {
 'analyzed_instances': len(instance_ids),
 'findings': [],
 'summary': {
 'internet_reachable': 0,
 'vpc_reachable': 0,
 'isolated': 0
 }
 }
 
 for instance_id in instance_ids:
 try:
 # Get Inspector findings for network reachability
 findings_response = self.inspector2_client.list_findings(
 filterCriteria={
 'resourceId': [instance_id],
 'findingType': ['NETWORK_REACHABILITY']
 }
 )
 
 instance_analysis = self.analyze_instance_network(instance_id)
 
 if findings_response['findings']:
 for finding in findings_response['findings']:
 network_details = finding.get('networkReachabilityDetails', {})
 
 reachability_finding = {
 'instance_id': instance_id,
 'finding_id': finding['findingArn'],
 'severity': finding.get('severity'),
 'network_path': network_details.get('networkPath', {}),
 'open_port_range': network_details.get('openPortRange', {}),
 'protocol': network_details.get('protocol'),
 'reachable_from': self.determine_reachability_source(network_details)
 }
 
 reachability_analysis['findings'].append(reachability_finding)
 
 # Update summary
 if 'internet' in reachability_finding['reachable_from'].lower():
 reachability_analysis['summary']['internet_reachable'] += 1
 elif 'vpc' in reachability_finding['reachable_from'].lower():
 reachability_analysis['summary']['vpc_reachable'] += 1
 
 else:
 # No network reachability findings - likely isolated
 reachability_analysis['summary']['isolated'] += 1
 
 except Exception as e:
 print(f"Error analyzing instance {instance_id}: {str(e)}")
 
 return reachability_analysis
 
 def analyze_instance_network(self, instance_id):
 """Analyze network configuration of an instance"""
 
 try:
 instance_response = self.ec2_client.describe_instances(InstanceIds=[instance_id])
 instance = instance_response['Reservations'][0]['Instances'][0]
 
 network_analysis = {
 'instance_id': instance_id,
 'vpc_id': instance.get('VpcId'),
 'subnet_id': instance.get('SubnetId'),
 'public_ip': instance.get('PublicIpAddress'),
 'private_ip': instance.get('PrivateIpAddress'),
 'security_groups': [sg['GroupId'] for sg in instance.get('SecurityGroups', [])],
 'network_interfaces': []
 }
 
 # Analyze network interfaces
 for eni in instance.get('NetworkInterfaces', []):
 network_analysis['network_interfaces'].append({
 'interface_id': eni.get('NetworkInterfaceId'),
 'public_ip': eni.get('Association', {}).get('PublicIp'),
 'private_ip': eni.get('PrivateIpAddress'),
 'security_groups': [sg['GroupId'] for sg in eni.get('Groups', [])]
 })
 
 # Check subnet route table
 subnet_response = self.ec2_client.describe_subnets(SubnetIds=[instance['SubnetId']])
 subnet = subnet_response['Subnets'][0]
 
 # Get route table associations
 route_tables_response = self.ec2_client.describe_route_tables(
 Filters=[
 {'Name': 'association.subnet-id', 'Values': [subnet['SubnetId']]}
 ]
 )
 
 if route_tables_response['RouteTables']:
 route_table = route_tables_response['RouteTables'][0]
 network_analysis['route_table_id'] = route_table['RouteTableId']
 network_analysis['routes'] = route_table['Routes']
 
 # Check for internet gateway route
 has_igw_route = any(
 route.get('GatewayId', '').startswith('igw-') and route.get('DestinationCidrBlock') == '0.0.0.0/0'
 for route in route_table['Routes']
 )
 network_analysis['has_internet_route'] = has_igw_route
 
 return network_analysis
 
 except Exception as e:
 return {
 'instance_id': instance_id,
 'error': str(e)
 }
 
 def determine_reachability_source(self, network_details):
 """Determine the source of network reachability"""
 
 network_path = network_details.get('networkPath', {})
 
 # Check for internet gateway in path
 for step in network_path.get('steps', []):
 component = step.get('component', {})
 if component.get('type') == 'AWS_INTERNET_GATEWAY':
 return 'Internet'
 
 # Check for VPC peering or transit gateway
 for step in network_path.get('steps', []):
 component = step.get('component', {})
 if component.get('type') in ['AWS_VPC_PEERING_CONNECTION', 'AWS_TRANSIT_GATEWAY']:
 return 'VPC Peering/Transit Gateway'
 
 return 'VPC Internal'

4. Container Image Assessment

ECR Integration

# CloudFormation for ECR with Inspector scanning
Resources:
 ECRRepository:
 Type: AWS::ECR::Repository
 Properties:
 RepositoryName: my-application
 ImageScanningConfiguration:
 ScanOnPush: true
 ImageTagMutability: MUTABLE
 LifecyclePolicy:
 LifecyclePolicyText: |
 {
 "rules": [
 {
 "rulePriority": 1,
 "description": "Keep last 10 images",
 "selection": {
 "tagStatus": "tagged",
 "tagPrefixList": ["v"],
 "countType": "imageCountMoreThan",
 "countNumber": 10
 },
 "action": {
 "type": "expire"
 }
 }
 ]
 }

 # Lambda function to process ECR scan results
 ECRScanProcessor:
 Type: AWS::Lambda::Function
 Properties:
 FunctionName: ecr-scan-processor
 Runtime: python3.9
 Handler: index.lambda_handler
 Role: !GetAtt ECRScanProcessorRole.Arn
 Code:
 ZipFile: |
 import boto3
 import json
 
 def lambda_handler(event, context):
 """Process ECR image scan results"""
 
 ecr = boto3.client('ecr')
 inspector2 = boto3.client('inspector2')
 sns = boto3.client('sns')
 
 # Parse ECR scan event
 detail = event['detail']
 repository_name = detail['repository-name']
 image_tag = detail['image-tags'][0] if detail['image-tags'] else 'latest'
 scan_status = detail['scan-status']
 
 if scan_status == 'COMPLETE':
 # Get scan results from Inspector
 findings = get_image_findings(inspector2, repository_name, image_tag)
 
 # Process findings
 scan_summary = process_scan_findings(findings)
 
 # Send notification if critical vulnerabilities found
 if scan_summary['critical_count'] > 0:
 send_critical_alert(sns, repository_name, image_tag, scan_summary)
 
 # Block deployment if too many high-severity vulnerabilities
 if scan_summary['high_count'] > 5:
 block_deployment(repository_name, image_tag)
 
 return {
 'statusCode': 200,
 'body': json.dumps('ECR scan processed successfully')
 }
 
 def get_image_findings(inspector2, repository_name, image_tag):
 """Get Inspector findings for container image"""
 
 try:
 response = inspector2.list_findings(
 filterCriteria={
 'ecrImageTags': [image_tag],
 'ecrRepositoryName': [repository_name]
 }
 )
 
 return response['findings']
 
 except Exception as e:
 print(f"Error getting findings: {str(e)}")
 return []
 
 def process_scan_findings(findings):
 """Process and summarize scan findings"""
 
 summary = {
 'total_findings': len(findings),
 'critical_count': 0,
 'high_count': 0,
 'medium_count': 0,
 'low_count': 0,
 'vulnerabilities': []
 }
 
 for finding in findings:
 severity = finding.get('severity', 'INFORMATIONAL')
 
 if severity == 'CRITICAL':
 summary['critical_count'] += 1
 elif severity == 'HIGH':
 summary['high_count'] += 1
 elif severity == 'MEDIUM':
 summary['medium_count'] += 1
 elif severity == 'LOW':
 summary['low_count'] += 1
 
 # Extract vulnerability details
 vuln_details = finding.get('packageVulnerabilityDetails', {})
 if vuln_details:
 summary['vulnerabilities'].append({
 'id': vuln_details.get('vulnerabilityId'),
 'severity': severity,
 'package': vuln_details.get('vulnerablePackages', [{}])[0].get('name'),
 'fixed_version': vuln_details.get('vulnerablePackages', [{}])[0].get('fixedInVersion')
 })
 
 return summary
 
 def send_critical_alert(sns, repository_name, image_tag, summary):
 """Send alert for critical vulnerabilities"""
 
 message = f"""
 Critical Vulnerabilities Detected
 ================================
 
 Repository: {repository_name}
 Image Tag: {image_tag}
 
 Critical Vulnerabilities: {summary['critical_count']}
 High Vulnerabilities: {summary['high_count']}
 
 Top Vulnerabilities:
 """
 
 for vuln in summary['vulnerabilities'][:5]:
 message += f"\n- {vuln['id']} ({vuln['severity']}) in {vuln['package']}"
 
 sns.publish(
 TopicArn='arn:aws:sns:us-east-1:123456789012:security-alerts',
 Message=message,
 Subject=f'Critical Vulnerabilities in {repository_name}:{image_tag}'
 )
 
 def block_deployment(repository_name, image_tag):
 """Block deployment of vulnerable image"""
 
 # Add tag to indicate blocked status
 ecr = boto3.client('ecr')
 
 try:
 ecr.put_image_tag_mutability(
 repositoryName=repository_name,
 imageTagMutability='IMMUTABLE'
 )
 
 print(f"Blocked deployment of {repository_name}:{image_tag}")
 
 except Exception as e:
 print(f"Error blocking deployment: {str(e)}")

 # EventBridge rule for ECR scan completion
 ECRScanEventRule:
 Type: AWS::Events::Rule
 Properties:
 Name: ECRScanCompletion
 Description: Process ECR image scan completion
 EventPattern:
 source:
 - aws.ecr
 detail-type:
 - ECR Image Scan
 detail:
 scan-status:
 - COMPLETE
 State: ENABLED
 Targets:
 - Arn: !GetAtt ECRScanProcessor.Arn
 Id: ECRScanProcessorTarget

CI/CD Integration

# CodePipeline with Inspector scanning
Resources:
 BuildProject:
 Type: AWS::CodeBuild::Project
 Properties:
 Name: secure-container-build
 ServiceRole: !GetAtt CodeBuildRole.Arn
 Artifacts:
 Type: CODEPIPELINE
 Environment:
 Type: LINUX_CONTAINER
 ComputeType: BUILD_GENERAL1_MEDIUM
 Image: aws/codebuild/amazonlinux2-x86_64-standard:3.0
 PrivilegedMode: true
 Source:
 Type: CODEPIPELINE
 BuildSpec: |
 version: 0.2
 phases:
 pre_build:
 commands:
 - echo Logging in to Amazon ECR...
 - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
 build:
 commands:
 - echo Build started on `date`
 - docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
 - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
 post_build:
 commands:
 - echo Build completed on `date`
 - docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
 - echo Waiting for Inspector scan...
 - python3 wait_for_scan.py
 - python3 check_vulnerabilities.py
 artifacts:
 files:
 - '**/*'

 # Security gate Lambda function
 SecurityGateFunction:
 Type: AWS::Lambda::Function
 Properties:
 FunctionName: container-security-gate
 Runtime: python3.9
 Handler: index.lambda_handler
 Role: !GetAtt SecurityGateRole.Arn
 Timeout: 300
 Code:
 ZipFile: |
 import boto3
 import json
 import time
 
 def lambda_handler(event, context):
 """Security gate for container deployment"""
 
 codepipeline = boto3.client('codepipeline')
 inspector2 = boto3.client('inspector2')
 
 # Get job details
 job_id = event['CodePipeline.job']['id']
 job_data = event['CodePipeline.job']['data']
 
 # Extract image details from input artifacts
 repository_name = job_data['actionConfiguration']['configuration']['RepositoryName']
 image_tag = job_data['actionConfiguration']['configuration']['ImageTag']
 
 try:
 # Wait for scan to complete
 scan_complete = wait_for_scan_completion(inspector2, repository_name, image_tag)
 
 if not scan_complete:
 codepipeline.put_job_failure_result(
 jobId=job_id,
 failureDetails={'message': 'Scan did not complete within timeout', 'type': 'JobFailed'}
 )
 return
 
 # Check vulnerability threshold
 vulnerability_check = check_vulnerability_threshold(inspector2, repository_name, image_tag)
 
 if vulnerability_check['passed']:
 codepipeline.put_job_success_result(jobId=job_id)
 else:
 codepipeline.put_job_failure_result(
 jobId=job_id,
 failureDetails={
 'message': f"Security gate failed: {vulnerability_check['reason']}",
 'type': 'JobFailed'
 }
 )
 
 except Exception as e:
 codepipeline.put_job_failure_result(
 jobId=job_id,
 failureDetails={'message': str(e), 'type': 'JobFailed'}
 )
 
 def wait_for_scan_completion(inspector2, repository_name, image_tag, max_wait=300):
 """Wait for Inspector scan to complete"""
 
 start_time = time.time()
 
 while time.time() - start_time < max_wait:
 try:
 findings = inspector2.list_findings(
 filterCriteria={
 'ecrRepositoryName': [repository_name],
 'ecrImageTags': [image_tag]
 }
 )
 
 # If we get findings, scan is complete
 if findings['findings']:
 return True
 
 time.sleep(30) # Wait 30 seconds before checking again
 
 except Exception as e:
 print(f"Error checking scan status: {str(e)}")
 time.sleep(30)
 
 return False
 
 def check_vulnerability_threshold(inspector2, repository_name, image_tag):
 """Check if vulnerabilities exceed threshold"""
 
 try:
 findings = inspector2.list_findings(
 filterCriteria={
 'ecrRepositoryName': [repository_name],
 'ecrImageTags': [image_tag]
 }
 )
 
 critical_count = 0
 high_count = 0
 
 for finding in findings['findings']:
 severity = finding.get('severity', 'INFORMATIONAL')
 if severity == 'CRITICAL':
 critical_count += 1
 elif severity == 'HIGH':
 high_count += 1
 
 # Security policy: No critical vulnerabilities, max 3 high
 if critical_count > 0:
 return {
 'passed': False,
 'reason': f'{critical_count} critical vulnerabilities found'
 }
 
 if high_count > 3:
 return {
 'passed': False,
 'reason': f'{high_count} high vulnerabilities found (max 3 allowed)'
 }
 
 return {
 'passed': True,
 'reason': 'Security requirements met'
 }
 
 except Exception as e:
 return {
 'passed': False,
 'reason': f'Error checking vulnerabilities: {str(e)}'
 }

5. Findings Management and Remediation

Automated Remediation

# Automated remediation for Inspector findings
import boto3
import json
from datetime import datetime

class InspectorRemediationManager:
 def __init__(self):
 self.inspector2_client = boto3.client('inspector2')
 self.ssm_client = boto3.client('ssm')
 self.ec2_client = boto3.client('ec2')
 self.sns_client = boto3.client('sns')
 
 def process_findings_for_remediation(self, severity_threshold='HIGH'):
 """Process Inspector findings and trigger remediation"""
 
 # Get findings above severity threshold
 findings_response = self.inspector2_client.list_findings(
 filterCriteria={
 'severity': [severity_threshold, 'CRITICAL']
 }
 )
 
 remediation_results = []
 
 for finding in findings_response['findings']:
 try:
 remediation_result = self.remediate_finding(finding)
 remediation_results.append(remediation_result)
 except Exception as e:
 remediation_results.append({
 'finding_id': finding.get('findingArn'),
 'status': 'FAILED',
 'error': str(e)
 })
 
 return remediation_results
 
 def remediate_finding(self, finding):
 """Remediate a specific Inspector finding"""
 
 finding_type = finding.get('type')
 severity = finding.get('severity')
 resources = finding.get('resources', [])
 
 remediation_actions = []
 
 # Package vulnerability remediation
 if 'PACKAGE_VULNERABILITY' in finding_type:
 for resource in resources:
 if 'ec2-instance' in resource.get('id', ''):
 # EC2 instance vulnerability
 instance_id = resource['id'].split('/')[-1]
 action = self.remediate_ec2_vulnerability(finding, instance_id)
 remediation_actions.append(action)
 
 elif 'ecr' in resource.get('id', ''):
 # Container image vulnerability
 action = self.remediate_container_vulnerability(finding, resource)
 remediation_actions.append(action)
 
 # Network reachability remediation
 elif 'NETWORK_REACHABILITY' in finding_type:
 for resource in resources:
 if 'ec2-instance' in resource.get('id', ''):
 instance_id = resource['id'].split('/')[-1]
 action = self.remediate_network_exposure(finding, instance_id)
 remediation_actions.append(action)
 
 return {
 'finding_id': finding.get('findingArn'),
 'finding_type': finding_type,
 'severity': severity,
 'remediation_actions': remediation_actions,
 'status': 'COMPLETED'
 }
 
 def remediate_ec2_vulnerability(self, finding, instance_id):
 """Remediate vulnerability on EC2 instance"""
 
 vuln_details = finding.get('packageVulnerabilityDetails', {})
 vulnerable_packages = vuln_details.get('vulnerablePackages', [])
 
 if not vulnerable_packages:
 return {
 'action': 'EC2 Vulnerability Remediation',
 'status': 'SKIPPED',
 'reason': 'No vulnerable packages identified'
 }
 
 package_name = vulnerable_packages[0].get('name')
 fixed_version = vulnerable_packages[0].get('fixedInVersion')
 
 try:
 # Create remediation script
 remediation_script = self.create_package_update_script(package_name, fixed_version)
 
 # Execute via Systems Manager
 response = self.ssm_client.send_command(
 InstanceIds=[instance_id],
 DocumentName='AWS-RunShellScript',
 Parameters={
 'commands': remediation_script
 },
 Comment=f'Inspector vulnerability remediation for {package_name}'
 )
 
 return {
 'action': 'EC2 Vulnerability Remediation',
 'status': 'INITIATED',
 'command_id': response['Command']['CommandId'],
 'package': package_name,
 'fixed_version': fixed_version
 }
 
 except Exception as e:
 return {
 'action': 'EC2 Vulnerability Remediation',
 'status': 'FAILED',
 'error': str(e)
 }
 
 def create_package_update_script(self, package_name, fixed_version):
 """Create script to update vulnerable package"""
 
 script = [
 '#!/bin/bash',
 'set -e',
 '',
 '# Update package repositories',
 'if command -v yum &> /dev/null; then',
 ' yum update -y',
 f' yum update -y {package_name}',
 'elif command -v apt-get &> /dev/null; then',
 ' apt-get update',
 f' apt-get install -y {package_name}',
 'elif command -v apk &> /dev/null; then',
 ' apk update',
 f' apk upgrade {package_name}',
 'fi',
 '',
 '# Verify update',
 f'echo "Package {package_name} updated successfully"',
 '',
 '# Check if reboot is required',
 'if [ -f /var/run/reboot-required ]; then',
 ' echo "REBOOT_REQUIRED: System requires reboot"',
 'fi'
 ]
 
 return script
 
 def remediate_container_vulnerability(self, finding, resource):
 """Remediate vulnerability in container image"""
 
 # Extract repository and tag information
 resource_id = resource.get('id', '')
 repository_name = resource_id.split('/')[-2] if '/' in resource_id else 'unknown'
 
 vuln_details = finding.get('packageVulnerabilityDetails', {})
 vulnerable_packages = vuln_details.get('vulnerablePackages', [])
 
 if not vulnerable_packages:
 return {
 'action': 'Container Vulnerability Remediation',
 'status': 'SKIPPED',
 'reason': 'No vulnerable packages identified'
 }
 
 # Create remediation guidance
 remediation_guidance = self.create_container_remediation_guidance(vulnerable_packages)
 
 # Send notification to development team
 self.send_container_remediation_notification(repository_name, finding, remediation_guidance)
 
 return {
 'action': 'Container Vulnerability Remediation',
 'status': 'NOTIFICATION_SENT',
 'repository': repository_name,
 'guidance': remediation_guidance
 }
 
 def create_container_remediation_guidance(self, vulnerable_packages):
 """Create remediation guidance for container vulnerabilities"""
 
 guidance = {
 'dockerfile_updates': [],
 'base_image_recommendations': [],
 'package_updates': []
 }
 
 for package in vulnerable_packages:
 package_name = package.get('name')
 current_version = package.get('version')
 fixed_version = package.get('fixedInVersion')
 
 if fixed_version:
 guidance['package_updates'].append({
 'package': package_name,
 'current_version': current_version,
 'fixed_version': fixed_version,
 'dockerfile_instruction': f'RUN apt-get update && apt-get install -y {package_name}={fixed_version}'
 })
 
 return guidance
 
 def remediate_network_exposure(self, finding, instance_id):
 """Remediate network exposure issues"""
 
 network_details = finding.get('networkReachabilityDetails', {})
 open_port_range = network_details.get('openPortRange', {})
 protocol = network_details.get('protocol', 'TCP')
 
 try:
 # Get instance security groups
 instance_response = self.ec2_client.describe_instances(InstanceIds=[instance_id])
 security_groups = instance_response['Reservations'][0]['Instances'][0]['SecurityGroups']
 
 remediation_actions = []
 
 for sg in security_groups:
 sg_id = sg['GroupId']
 
 # Get security group rules
 sg_response = self.ec2_client.describe_security_groups(GroupIds=[sg_id])
 sg_rules = sg_response['SecurityGroups'][0]['IpPermissions']
 
 # Check for overly permissive rules
 for rule in sg_rules:
 if self.is_rule_overly_permissive(rule, open_port_range, protocol):
 # Create more restrictive rule
 new_rule = self.create_restrictive_rule(rule, open_port_range)
 
 # Remove old rule and add new one
 self.ec2_client.revoke_security_group_ingress(
 GroupId=sg_id,
 IpPermissions=[rule]
 )
 
 self.ec2_client.authorize_security_group_ingress(
 GroupId=sg_id,
 IpPermissions=[new_rule]
 )
 
 remediation_actions.append({
 'security_group': sg_id,
 'action': 'Rule Updated',
 'old_rule': rule,
 'new_rule': new_rule
 })
 
 return {
 'action': 'Network Exposure Remediation',
 'status': 'COMPLETED',
 'instance_id': instance_id,
 'remediation_actions': remediation_actions
 }
 
 except Exception as e:
 return {
 'action': 'Network Exposure Remediation',
 'status': 'FAILED',
 'error': str(e)
 }
 
 def is_rule_overly_permissive(self, rule, open_port_range, protocol):
 """Check if security group rule is overly permissive"""
 
 # Check for 0.0.0.0/0 source
 for ip_range in rule.get('IpRanges', []):
 if ip_range.get('CidrIp') == '0.0.0.0/0':
 return True
 
 # Check for wide port ranges
 from_port = rule.get('FromPort', 0)
 to_port = rule.get('ToPort', 65535)
 
 if to_port - from_port > 100: # Wide port range
 return True
 
 return False
 
 def send_container_remediation_notification(self, repository_name, finding, guidance):
 """Send notification about container vulnerability remediation"""
 
 message = f"""
 Container Vulnerability Remediation Required
 ==========================================
 
 Repository: {repository_name}
 Finding ID: {finding.get('findingArn')}
 Severity: {finding.get('severity')}
 
 Vulnerable Packages:
 """
 
 for update in guidance['package_updates']:
 message += f"\n- {update['package']}: {update['current_version']} -> {update['fixed_version']}"
 
 message += "\n\nRemediation Steps:\n"
 for update in guidance['package_updates']:
 message += f"\n{update['dockerfile_instruction']}"
 
 self.sns_client.publish(
 TopicArn='arn:aws:sns:us-east-1:123456789012:container-security',
 Message=message,
 Subject=f'Container Vulnerability Remediation: {repository_name}'
 )

6. Common Exam Scenarios

Scenario 1: Automated Security Pipeline

# Complete automated security pipeline with Inspector
Resources:
 SecurityPipeline:
 Type: AWS::CodePipeline::Pipeline
 Properties:
 RoleArn: !GetAtt CodePipelineRole.Arn
 ArtifactStore:
 Type: S3
 Location: !Ref ArtifactsBucket
 Stages:
 - Name: Source
 Actions:
 - Name: SourceAction
 ActionTypeId:
 Category: Source
 Owner: AWS
 Provider: CodeCommit
 Version: '1'
 Configuration:
 RepositoryName: secure-application
 BranchName: main
 OutputArtifacts:
 - Name: SourceOutput

 - Name: Build
 Actions:
 - Name: BuildAction
 ActionTypeId:
 Category: Build
 Owner: AWS
 Provider: CodeBuild
 Version: '1'
 Configuration:
 ProjectName: !Ref SecureBuildProject
 InputArtifacts:
 - Name: SourceOutput
 OutputArtifacts:
 - Name: BuildOutput

 - Name: SecurityScan
 Actions:
 - Name: InspectorScan
 ActionTypeId:
 Category: Invoke
 Owner: AWS
 Provider: Lambda
 Version: '1'
 Configuration:
 FunctionName: !Ref SecurityScanFunction
 InputArtifacts:
 - Name: BuildOutput
 OutputArtifacts:
 - Name: ScanOutput

 - Name: SecurityGate
 Actions:
 - Name: SecurityApproval
 ActionTypeId:
 Category: Invoke
 Owner: AWS
 Provider: Lambda
 Version: '1'
 Configuration:
 FunctionName: !Ref SecurityGateFunction
 InputArtifacts:
 - Name: ScanOutput

 - Name: Deploy
 Actions:
 - Name: DeployAction
 ActionTypeId:
 Category: Deploy
 Owner: AWS
 Provider: ECS
 Version: '1'
 Configuration:
 ClusterName: !Ref ECSCluster
 ServiceName: !Ref ECSService
 FileName: imagedefinitions.json
 InputArtifacts:
 - Name: BuildOutput

7. Exam Tips

Understand Inspector V2 - Know the differences from Inspector Classic
Master finding types - Package vulnerabilities vs network reachability
Know ECR integration - Automatic scanning on image push
Practice remediation - Automated and manual remediation workflows
Learn CI/CD integration - Security gates and pipeline integration
Understand suppression - When and how to suppress findings
Know multi-account setup - Organization-wide Inspector deployment
Practice cost optimization - Resource type selection and scanning frequency
Master troubleshooting - Common issues with scanning and findings
Understand compliance - Integration with Security Hub and compliance frameworks

AWS Security Hub

Sun, 04 Oct 2026 00:00:00 +0000

1. Overview

AWS Security Hub provides a comprehensive view of your security posture across your AWS accounts. It aggregates, organizes, and prioritizes security findings from multiple AWS security services and third-party tools.

Key Characteristics

Centralized security dashboard - Single pane of glass for security findings
Multi-service integration - Aggregates findings from GuardDuty, Inspector, Macie, and more
Security standards compliance - Built-in compliance checks for industry standards
Custom insights - Create custom views and filters for security data
Automated remediation - Integration with Lambda and Systems Manager for response
Cross-account management - Centralized security monitoring across AWS Organizations
Third-party integrations - Support for partner security tools and services

What Problem Does It Solve?

Eliminates security tool silos by centralizing findings
Provides consistent security posture visibility across accounts
Automates compliance checking against security standards
Enables faster incident response and remediation
Reduces operational overhead of managing multiple security tools
Facilitates security reporting and audit requirements

2. Core Components

Security Hub Setup

# CloudFormation for Security Hub configuration
Resources:
 SecurityHub:
 Type: AWS::SecurityHub::Hub
 Properties:
 Tags:
 - Key: Purpose
 Value: CentralizedSecurityMonitoring
 - Key: Environment
 Value: Production

 # Enable security standards
 CISStandard:
 Type: AWS::SecurityHub::Standard
 Properties:
 StandardsArn: !Sub "arn:aws:securityhub:::ruleset/finding-format/aws-foundational-security-standard/v/1.0.0"
 DisabledStandardsControls:
 - StandardsControlArn: !Sub "arn:aws:securityhub:${AWS::Region}:${AWS::AccountId}:control/aws-foundational-security-standard/v/1.0.0/S3.1"
 DisabledReason: "S3 bucket public access is required for static website hosting"

 PCIStandard:
 Type: AWS::SecurityHub::Standard
 Properties:
 StandardsArn: !Sub "arn:aws:securityhub:::ruleset/finding-format/pci-dss/v/3.2.1"

 # Product subscriptions
 GuardDutySubscription:
 Type: AWS::SecurityHub::ProductSubscription
 Properties:
 ProductArn: !Sub "arn:aws:securityhub:${AWS::Region}::product/aws/guardduty"

 InspectorSubscription:
 Type: AWS::SecurityHub::ProductSubscription
 Properties:
 ProductArn: !Sub "arn:aws:securityhub:${AWS::Region}::product/aws/inspector"

 ConfigSubscription:
 Type: AWS::SecurityHub::ProductSubscription
 Properties:
 ProductArn: !Sub "arn:aws:securityhub:${AWS::Region}::product/aws/config"

 MacieSubscription:
 Type: AWS::SecurityHub::ProductSubscription
 Properties:
 ProductArn: !Sub "arn:aws:securityhub:${AWS::Region}::product/aws/macie"

Custom Insights

Resources:
 # High severity findings insight
 HighSeverityInsight:
 Type: AWS::SecurityHub::Insight
 Properties:
 Name: High Severity Security Findings
 Description: All high and critical severity findings across services
 GroupByAttribute: ProductFields/aws/securityhub/ProductName
 Filters:
 SeverityLabel:
 - Value: HIGH
 Comparison: EQUALS
 - Value: CRITICAL
 Comparison: EQUALS
 RecordState:
 - Value: ACTIVE
 Comparison: EQUALS

 # Non-compliant resources insight
 ComplianceInsight:
 Type: AWS::SecurityHub::Insight
 Properties:
 Name: Non-Compliant Resources
 Description: Resources failing compliance checks
 GroupByAttribute: ResourceType
 Filters:
 ComplianceStatus:
 - Value: FAILED
 Comparison: EQUALS
 RecordState:
 - Value: ACTIVE
 Comparison: EQUALS

 # Network security findings
 NetworkSecurityInsight:
 Type: AWS::SecurityHub::Insight
 Properties:
 Name: Network Security Issues
 Description: Network-related security findings
 GroupByAttribute: ResourceId
 Filters:
 Type:
 - Value: "Effects/Data Exfiltration"
 Comparison: PREFIX
 - Value: "Unusual Behaviors/Network"
 Comparison: PREFIX
 RecordState:
 - Value: ACTIVE
 Comparison: EQUALS

 # Unresolved findings by age
 AgingFindingsInsight:
 Type: AWS::SecurityHub::Insight
 Properties:
 Name: Aging Security Findings
 Description: Findings older than 30 days
 GroupByAttribute: SeverityLabel
 Filters:
 CreatedAt:
 - DateRange:
 Unit: DAYS
 Value: 30
 Comparison: GREATER_THAN
 RecordState:
 - Value: ACTIVE
 Comparison: EQUALS

3. Multi-Account Management

Organization Integration

import boto3
import json
from datetime import datetime, timedelta

class SecurityHubOrganizationManager:
 def __init__(self):
 self.securityhub_client = boto3.client('securityhub')
 self.organizations_client = boto3.client('organizations')
 self.sts_client = boto3.client('sts')
 
 def setup_organization_security_hub(self):
 """Setup Security Hub across AWS Organization"""
 
 # Enable Security Hub in master account
 master_setup = self.enable_security_hub_master()
 
 # Get organization accounts
 accounts = self.get_organization_accounts()
 
 # Enable Security Hub in member accounts
 member_results = {}
 
 for account in accounts:
 if account['Status'] == 'ACTIVE' and account['Id'] != self.get_master_account_id():
 result = self.enable_security_hub_member(account['Id'], account['Email'])
 member_results[account['Id']] = result
 
 # Configure organization settings
 org_config = self.configure_organization_settings()
 
 return {
 'master_setup': master_setup,
 'member_results': member_results,
 'organization_config': org_config
 }
 
 def enable_security_hub_master(self):
 """Enable Security Hub in master account"""
 
 try:
 # Enable Security Hub
 self.securityhub_client.enable_security_hub(
 Tags={
 'Purpose': 'OrganizationSecurityHub',
 'ManagedBy': 'SecurityTeam'
 },
 EnableDefaultStandards=True
 )
 
 # Enable product integrations
 product_arns = [
 f"arn:aws:securityhub:{boto3.Session().region_name}::product/aws/guardduty",
 f"arn:aws:securityhub:{boto3.Session().region_name}::product/aws/inspector",
 f"arn:aws:securityhub:{boto3.Session().region_name}::product/aws/config",
 f"arn:aws:securityhub:{boto3.Session().region_name}::product/aws/macie"
 ]
 
 for product_arn in product_arns:
 try:
 self.securityhub_client.enable_import_findings_for_product(
 ProductArn=product_arn
 )
 except Exception as e:
 print(f"Error enabling product {product_arn}: {str(e)}")
 
 return {
 'status': 'SUCCESS',
 'enabled_products': len(product_arns)
 }
 
 except Exception as e:
 return {
 'status': 'FAILED',
 'error': str(e)
 }
 
 def enable_security_hub_member(self, account_id, email):
 """Enable Security Hub in member account and invite"""
 
 try:
 # Create member
 self.securityhub_client.create_members(
 AccountDetails=[
 {
 'AccountId': account_id,
 'Email': email
 }
 ]
 )
 
 # Invite member
 self.securityhub_client.invite_members(
 AccountIds=[account_id],
 Message='Please accept Security Hub invitation for centralized security monitoring'
 )
 
 return {
 'status': 'INVITED',
 'account_id': account_id
 }
 
 except Exception as e:
 return {
 'status': 'FAILED',
 'account_id': account_id,
 'error': str(e)
 }
 
 def configure_organization_settings(self):
 """Configure organization-wide Security Hub settings"""
 
 try:
 # Enable organization configuration
 self.securityhub_client.update_organization_configuration(
 AutoEnable=True,
 AutoEnableStandards='DEFAULT'
 )
 
 return {
 'status': 'SUCCESS',
 'auto_enable': True
 }
 
 except Exception as e:
 return {
 'status': 'FAILED',
 'error': str(e)
 }
 
 def get_organization_accounts(self):
 """Get all accounts in the organization"""
 
 accounts = []
 paginator = self.organizations_client.get_paginator('list_accounts')
 
 for page in paginator.paginate():
 accounts.extend(page['Accounts'])
 
 return accounts
 
 def get_master_account_id(self):
 """Get master account ID"""
 
 return self.sts_client.get_caller_identity()['Account']
 
 def aggregate_findings_across_accounts(self):
 """Aggregate Security Hub findings across all member accounts"""
 
 aggregated_data = {
 'timestamp': datetime.utcnow().isoformat(),
 'total_findings': 0,
 'severity_breakdown': {
 'CRITICAL': 0,
 'HIGH': 0,
 'MEDIUM': 0,
 'LOW': 0,
 'INFORMATIONAL': 0
 },
 'compliance_breakdown': {
 'PASSED': 0,
 'FAILED': 0,
 'WARNING': 0,
 'NOT_AVAILABLE': 0
 },
 'account_summaries': {}
 }
 
 try:
 # Get findings from all accounts
 paginator = self.securityhub_client.get_paginator('get_findings')
 
 for page in paginator.paginate(
 Filters={
 'RecordState': [
 {
 'Value': 'ACTIVE',
 'Comparison': 'EQUALS'
 }
 ]
 }
 ):
 for finding in page['Findings']:
 aggregated_data['total_findings'] += 1
 
 # Count by severity
 severity = finding.get('Severity', {}).get('Label', 'INFORMATIONAL')
 aggregated_data['severity_breakdown'][severity] += 1
 
 # Count by compliance
 compliance = finding.get('Compliance', {}).get('Status', 'NOT_AVAILABLE')
 aggregated_data['compliance_breakdown'][compliance] += 1
 
 # Track by account
 account_id = finding.get('AwsAccountId', 'Unknown')
 if account_id not in aggregated_data['account_summaries']:
 aggregated_data['account_summaries'][account_id] = {
 'total_findings': 0,
 'critical_findings': 0,
 'high_findings': 0
 }
 
 aggregated_data['account_summaries'][account_id]['total_findings'] += 1
 
 if severity == 'CRITICAL':
 aggregated_data['account_summaries'][account_id]['critical_findings'] += 1
 elif severity == 'HIGH':
 aggregated_data['account_summaries'][account_id]['high_findings'] += 1
 
 except Exception as e:
 aggregated_data['error'] = str(e)
 
 return aggregated_data

4. Automated Response and Remediation

Lambda-based Response Function

# Lambda function for Security Hub automated response
import boto3
import json
from datetime import datetime

def lambda_handler(event, context):
 """
 Automated response to Security Hub findings
 Triggered by CloudWatch Events when findings are created or updated
 """
 
 # Parse Security Hub finding from CloudWatch Event
 findings = event['detail']['findings']
 
 response_results = []
 
 for finding in findings:
 try:
 response_result = process_security_finding(finding)
 response_results.append(response_result)
 except Exception as e:
 response_results.append({
 'finding_id': finding.get('Id'),
 'status': 'ERROR',
 'error': str(e)
 })
 
 return {
 'statusCode': 200,
 'body': json.dumps({
 'processed_findings': len(findings),
 'response_results': response_results
 })
 }

def process_security_finding(finding):
 """Process individual Security Hub finding"""
 
 finding_id = finding.get('Id')
 severity = finding.get('Severity', {}).get('Label', 'INFORMATIONAL')
 product_name = finding.get('ProductName', 'Unknown')
 finding_type = finding.get('Types', ['Unknown'])[0]
 
 print(f"Processing finding: {finding_id} ({severity}) from {product_name}")
 
 # Initialize AWS clients
 ec2 = boto3.client('ec2')
 iam = boto3.client('iam')
 s3 = boto3.client('s3')
 ssm = boto3.client('ssm')
 sns = boto3.client('sns')
 
 response_actions = []
 
 # Critical and High severity findings require immediate action
 if severity in ['CRITICAL', 'HIGH']:
 response_actions.extend(handle_high_severity_finding(finding, ec2, iam, s3, ssm))
 
 # Compliance failures
 if finding.get('Compliance', {}).get('Status') == 'FAILED':
 response_actions.extend(handle_compliance_failure(finding, ec2, iam, s3, ssm))
 
 # Product-specific handling
 if product_name == 'GuardDuty':
 response_actions.extend(handle_guardduty_finding(finding, ec2, iam))
 elif product_name == 'Inspector':
 response_actions.extend(handle_inspector_finding(finding, ssm))
 elif product_name == 'Config':
 response_actions.extend(handle_config_finding(finding, ec2, s3))
 
 # Send notifications
 send_finding_notification(finding, response_actions, sns)
 
 return {
 'finding_id': finding_id,
 'severity': severity,
 'product_name': product_name,
 'response_actions': response_actions,
 'status': 'PROCESSED'
 }

def handle_high_severity_finding(finding, ec2, iam, s3, ssm):
 """Handle critical and high severity findings"""
 
 actions = []
 resources = finding.get('Resources', [])
 
 for resource in resources:
 resource_type = resource.get('Type', '')
 resource_id = resource.get('Id', '')
 
 # EC2 instance security issues
 if resource_type == 'AwsEc2Instance':
 instance_id = resource_id.split('/')[-1]
 
 # Isolate instance if it's a security threat
 if any(threat in finding.get('Title', '') for threat in ['Backdoor', 'Malware', 'Cryptocurrency']):
 action = isolate_ec2_instance(ec2, instance_id)
 actions.append(action)
 
 # S3 bucket security issues
 elif resource_type == 'AwsS3Bucket':
 bucket_name = resource_id.split('/')[-1]
 
 # Enable public access block for exposed buckets
 if 'public' in finding.get('Title', '').lower():
 action = enable_s3_public_access_block(s3, bucket_name)
 actions.append(action)
 
 # IAM security issues
 elif resource_type in ['AwsIamUser', 'AwsIamRole']:
 # Disable compromised IAM entities
 if 'compromised' in finding.get('Description', '').lower():
 action = disable_iam_entity(iam, resource_type, resource_id)
 actions.append(action)
 
 return actions

def handle_compliance_failure(finding, ec2, iam, s3, ssm):
 """Handle compliance failures"""
 
 actions = []
 compliance_type = finding.get('Compliance', {}).get('StatusReasons', [{}])[0].get('ReasonCode', '')
 
 # Common compliance remediation patterns
 if 'ENCRYPTION' in compliance_type:
 actions.extend(remediate_encryption_compliance(finding, ec2, s3))
 elif 'ACCESS_CONTROL' in compliance_type:
 actions.extend(remediate_access_control_compliance(finding, iam, s3))
 elif 'LOGGING' in compliance_type:
 actions.extend(remediate_logging_compliance(finding, ssm))
 
 return actions

def handle_guardduty_finding(finding, ec2, iam):
 """Handle GuardDuty-specific findings"""
 
 actions = []
 finding_type = finding.get('Title', '')
 
 # Cryptocurrency mining
 if 'CryptoCurrency' in finding_type:
 resources = finding.get('Resources', [])
 for resource in resources:
 if resource.get('Type') == 'AwsEc2Instance':
 instance_id = resource.get('Id', '').split('/')[-1]
 action = stop_ec2_instance(ec2, instance_id, 'Cryptocurrency mining detected')
 actions.append(action)
 
 # Compromised credentials
 elif 'UnauthorizedAccess:IAMUser' in finding_type:
 # Disable access keys for compromised users
 user_name = extract_user_name_from_finding(finding)
 if user_name:
 action = disable_user_access_keys(iam, user_name)
 actions.append(action)
 
 return actions

def handle_inspector_finding(finding, ssm):
 """Handle Inspector-specific findings"""
 
 actions = []
 
 # Package vulnerabilities
 if 'Package Vulnerability' in finding.get('Title', ''):
 resources = finding.get('Resources', [])
 for resource in resources:
 if resource.get('Type') == 'AwsEc2Instance':
 instance_id = resource.get('Id', '').split('/')[-1]
 action = trigger_patch_installation(ssm, instance_id)
 actions.append(action)
 
 return actions

def handle_config_finding(finding, ec2, s3):
 """Handle Config-specific findings"""
 
 actions = []
 rule_name = finding.get('ProductFields', {}).get('aws/config/rule/name', '')
 
 # Security group rules
 if 'security-group' in rule_name:
 resources = finding.get('Resources', [])
 for resource in resources:
 if resource.get('Type') == 'AwsEc2SecurityGroup':
 sg_id = resource.get('Id', '').split('/')[-1]
 action = remediate_security_group(ec2, sg_id)
 actions.append(action)
 
 return actions

def isolate_ec2_instance(ec2, instance_id):
 """Isolate EC2 instance by creating restrictive security group"""
 
 try:
 # Get instance VPC
 instance_response = ec2.describe_instances(InstanceIds=[instance_id])
 vpc_id = instance_response['Reservations'][0]['Instances'][0]['VpcId']
 
 # Create isolation security group
 sg_response = ec2.create_security_group(
 GroupName=f'isolation-{instance_id}-{int(datetime.now().timestamp())}',
 Description=f'Isolation security group for compromised instance {instance_id}',
 VpcId=vpc_id
 )
 
 # Apply isolation security group
 ec2.modify_instance_attribute(
 InstanceId=instance_id,
 Groups=[sg_response['GroupId']]
 )
 
 return {
 'action': 'Instance Isolation',
 'resource': instance_id,
 'status': 'SUCCESS',
 'security_group': sg_response['GroupId']
 }
 
 except Exception as e:
 return {
 'action': 'Instance Isolation',
 'resource': instance_id,
 'status': 'FAILED',
 'error': str(e)
 }

def enable_s3_public_access_block(s3, bucket_name):
 """Enable S3 public access block"""
 
 try:
 s3.put_public_access_block(
 Bucket=bucket_name,
 PublicAccessBlockConfiguration={
 'BlockPublicAcls': True,
 'IgnorePublicAcls': True,
 'BlockPublicPolicy': True,
 'RestrictPublicBuckets': True
 }
 )
 
 return {
 'action': 'Enable S3 Public Access Block',
 'resource': bucket_name,
 'status': 'SUCCESS'
 }
 
 except Exception as e:
 return {
 'action': 'Enable S3 Public Access Block',
 'resource': bucket_name,
 'status': 'FAILED',
 'error': str(e)
 }

def send_finding_notification(finding, actions, sns):
 """Send notification about Security Hub finding and response"""
 
 severity = finding.get('Severity', {}).get('Label', 'INFORMATIONAL')
 
 message = f"""
 Security Hub Finding Alert
 =========================
 
 Finding ID: {finding.get('Id')}
 Title: {finding.get('Title')}
 Severity: {severity}
 Product: {finding.get('ProductName')}
 
 Description: {finding.get('Description')}
 
 Automated Response Actions:
 """
 
 for action in actions:
 message += f"\n- {action['action']}: {action['status']}"
 if action['status'] == 'FAILED':
 message += f" (Error: {action.get('error', 'Unknown')})"
 
 # Determine topic based on severity
 if severity in ['CRITICAL', 'HIGH']:
 topic_arn = 'arn:aws:sns:us-east-1:123456789012:critical-security-alerts'
 subject = f"CRITICAL: Security Hub Finding - {finding.get('Title')}"
 else:
 topic_arn = 'arn:aws:sns:us-east-1:123456789012:security-alerts'
 subject = f"Security Hub Finding - {finding.get('Title')}"
 
 sns.publish(
 TopicArn=topic_arn,
 Message=message,
 Subject=subject
 )

CloudWatch Events Integration

# CloudFormation for Security Hub event processing
Resources:
 SecurityHubEventRule:
 Type: AWS::Events::Rule
 Properties:
 Name: SecurityHubFindingProcessor
 Description: Process Security Hub findings for automated response
 EventPattern:
 source:
 - aws.securityhub
 detail-type:
 - Security Hub Findings - Imported
 detail:
 findings:
 Severity:
 Label:
 - HIGH
 - CRITICAL
 State: ENABLED
 Targets:
 - Arn: !GetAtt SecurityHubResponseFunction.Arn
 Id: SecurityHubResponseTarget

 # Compliance findings rule
 ComplianceFindingsRule:
 Type: AWS::Events::Rule
 Properties:
 Name: SecurityHubComplianceFindings
 Description: Process compliance-related findings
 EventPattern:
 source:
 - aws.securityhub
 detail-type:
 - Security Hub Findings - Imported
 detail:
 findings:
 Compliance:
 Status:
 - FAILED
 State: ENABLED
 Targets:
 - Arn: !GetAtt ComplianceRemediationFunction.Arn
 Id: ComplianceRemediationTarget

 # Lambda permission for EventBridge
 SecurityHubLambdaPermission:
 Type: AWS::Lambda::Permission
 Properties:
 FunctionName: !Ref SecurityHubResponseFunction
 Action: lambda:InvokeFunction
 Principal: events.amazonaws.com
 SourceArn: !GetAtt SecurityHubEventRule.Arn

5. Custom Security Standards

Custom Standard Creation

# Create custom security standard
import boto3
import json

class CustomSecurityStandard:
 def __init__(self):
 self.securityhub_client = boto3.client('securityhub')
 
 def create_custom_standard(self, standard_name, controls):
 """Create custom security standard with controls"""
 
 try:
 # Create custom standard
 standard_response = self.securityhub_client.create_standard(
 Name=standard_name,
 Description=f"Custom security standard: {standard_name}",
 Controls=controls
 )
 
 return {
 'status': 'SUCCESS',
 'standard_arn': standard_response['StandardArn']
 }
 
 except Exception as e:
 return {
 'status': 'FAILED',
 'error': str(e)
 }
 
 def create_devops_security_standard(self):
 """Create DevOps-specific security standard"""
 
 devops_controls = [
 {
 'StandardsControlArn': 'arn:aws:securityhub:::control/custom/devops-1',
 'Title': 'CI/CD Pipeline Security',
 'Description': 'Ensure CI/CD pipelines follow security best practices',
 'RemediationUrl': 'https://docs.aws.amazon.com/codepipeline/latest/userguide/security.html',
 'SeverityRating': 'HIGH',
 'RelatedRequirements': ['DevOps.1', 'DevOps.2']
 },
 {
 'StandardsControlArn': 'arn:aws:securityhub:::control/custom/devops-2',
 'Title': 'Container Image Security',
 'Description': 'Container images must be scanned for vulnerabilities',
 'RemediationUrl': 'https://docs.aws.amazon.com/inspector/latest/userguide/inspector_container-image-scanning.html',
 'SeverityRating': 'HIGH',
 'RelatedRequirements': ['DevOps.3', 'DevOps.4']
 },
 {
 'StandardsControlArn': 'arn:aws:securityhub:::control/custom/devops-3',
 'Title': 'Infrastructure as Code Security',
 'Description': 'IaC templates must be validated for security issues',
 'RemediationUrl': 'https://docs.aws.amazon.com/cloudformation/latest/userguide/security-best-practices.html',
 'SeverityRating': 'MEDIUM',
 'RelatedRequirements': ['DevOps.5', 'DevOps.6']
 },
 {
 'StandardsControlArn': 'arn:aws:securityhub:::control/custom/devops-4',
 'Title': 'Secrets Management',
 'Description': 'Secrets must not be hardcoded in source code',
 'RemediationUrl': 'https://docs.aws.amazon.com/secretsmanager/latest/userguide/best-practices.html',
 'SeverityRating': 'CRITICAL',
 'RelatedRequirements': ['DevOps.7', 'DevOps.8']
 }
 ]
 
 return self.create_custom_standard('DevOps Security Standard', devops_controls)
 
 def batch_import_custom_findings(self, findings):
 """Import custom findings to Security Hub"""
 
 try:
 response = self.securityhub_client.batch_import_findings(
 Findings=findings
 )
 
 return {
 'status': 'SUCCESS',
 'successful_count': response['SuccessCount'],
 'failed_count': response['FailedCount'],
 'failed_findings': response.get('FailedFindings', [])
 }
 
 except Exception as e:
 return {
 'status': 'FAILED',
 'error': str(e)
 }
 
 def create_pipeline_security_finding(self, pipeline_arn, issue_description, severity='MEDIUM'):
 """Create custom finding for CI/CD pipeline security issue"""
 
 finding = {
 'SchemaVersion': '2018-10-08',
 'Id': f"{pipeline_arn}/pipeline-security-check",
 'ProductArn': f"arn:aws:securityhub:{boto3.Session().region_name}:{boto3.Session().get_credentials().access_key[:12]}:product/custom/devops-security",
 'GeneratorId': 'custom-pipeline-scanner',
 'AwsAccountId': boto3.Session().get_credentials().access_key[:12],
 'Types': ['Software and Configuration Checks/Industry and Regulatory Standards/DevOps Security'],
 'CreatedAt': datetime.utcnow().isoformat() + 'Z',
 'UpdatedAt': datetime.utcnow().isoformat() + 'Z',
 'Severity': {
 'Label': severity
 },
 'Title': 'CI/CD Pipeline Security Issue',
 'Description': issue_description,
 'Resources': [
 {
 'Type': 'AwsCodePipelinePipeline',
 'Id': pipeline_arn,
 'Region': boto3.Session().region_name
 }
 ],
 'Compliance': {
 'Status': 'FAILED'
 },
 'RecordState': 'ACTIVE',
 'WorkflowState': 'NEW'
 }
 
 return self.batch_import_custom_findings([finding])

6. Reporting and Analytics

Security Metrics Dashboard

# Security Hub metrics and reporting
import boto3
import json
from datetime import datetime, timedelta

class SecurityHubAnalytics:
 def __init__(self):
 self.securityhub_client = boto3.client('securityhub')
 self.cloudwatch = boto3.client('cloudwatch')
 
 def generate_security_metrics(self):
 """Generate comprehensive security metrics"""
 
 metrics = {
 'timestamp': datetime.utcnow().isoformat(),
 'findings_summary': self.get_findings_summary(),
 'compliance_summary': self.get_compliance_summary(),
 'trend_analysis': self.get_trend_analysis(),
 'top_issues': self.get_top_security_issues(),
 'remediation_metrics': self.get_remediation_metrics()
 }
 
 # Send metrics to CloudWatch
 self.send_metrics_to_cloudwatch(metrics)
 
 return metrics
 
 def get_findings_summary(self):
 """Get summary of all active findings"""
 
 summary = {
 'total_findings': 0,
 'by_severity': {'CRITICAL': 0, 'HIGH': 0, 'MEDIUM': 0, 'LOW': 0, 'INFORMATIONAL': 0},
 'by_product': {},
 'by_compliance_status': {'PASSED': 0, 'FAILED': 0, 'WARNING': 0, 'NOT_AVAILABLE': 0}
 }
 
 try:
 paginator = self.securityhub_client.get_paginator('get_findings')
 
 for page in paginator.paginate(
 Filters={
 'RecordState': [{'Value': 'ACTIVE', 'Comparison': 'EQUALS'}]
 }
 ):
 for finding in page['Findings']:
 summary['total_findings'] += 1
 
 # Count by severity
 severity = finding.get('Severity', {}).get('Label', 'INFORMATIONAL')
 summary['by_severity'][severity] += 1
 
 # Count by product
 product = finding.get('ProductName', 'Unknown')
 summary['by_product'][product] = summary['by_product'].get(product, 0) + 1
 
 # Count by compliance status
 compliance = finding.get('Compliance', {}).get('Status', 'NOT_AVAILABLE')
 summary['by_compliance_status'][compliance] += 1
 
 except Exception as e:
 summary['error'] = str(e)
 
 return summary
 
 def get_compliance_summary(self):
 """Get compliance summary across all standards"""
 
 compliance_summary = {
 'standards': {},
 'overall_score': 0
 }
 
 try:
 # Get enabled standards
 standards_response = self.securityhub_client.get_enabled_standards()
 
 for standard in standards_response['StandardsSubscriptions']:
 standard_arn = standard['StandardsArn']
 standard_name = standard_arn.split('/')[-1]
 
 # Get controls for this standard
 controls_response = self.securityhub_client.describe_standards_controls(
 StandardsSubscriptionArn=standard['StandardsSubscriptionArn']
 )
 
 standard_compliance = {
 'total_controls': len(controls_response['Controls']),
 'enabled_controls': 0,
 'passed_controls': 0,
 'failed_controls': 0,
 'compliance_score': 0
 }
 
 for control in controls_response['Controls']:
 if control['ControlStatus'] == 'ENABLED':
 standard_compliance['enabled_controls'] += 1
 
 # Check control compliance (simplified - would need actual findings analysis)
 # This is a placeholder for actual compliance checking logic
 if control.get('ControlStatusUpdatedAt'): # Assume passed if recently updated
 standard_compliance['passed_controls'] += 1
 else:
 standard_compliance['failed_controls'] += 1
 
 # Calculate compliance score
 if standard_compliance['enabled_controls'] > 0:
 standard_compliance['compliance_score'] = (
 standard_compliance['passed_controls'] / 
 standard_compliance['enabled_controls']
 ) * 100
 
 compliance_summary['standards'][standard_name] = standard_compliance
 
 # Calculate overall compliance score
 total_controls = sum(s['enabled_controls'] for s in compliance_summary['standards'].values())
 total_passed = sum(s['passed_controls'] for s in compliance_summary['standards'].values())
 
 if total_controls > 0:
 compliance_summary['overall_score'] = (total_passed / total_controls) * 100
 
 except Exception as e:
 compliance_summary['error'] = str(e)
 
 return compliance_summary
 
 def get_trend_analysis(self, days_back=30):
 """Analyze security trends over time"""
 
 end_date = datetime.utcnow()
 start_date = end_date - timedelta(days=days_back)
 
 trend_data = {
 'period': f"{start_date.date()} to {end_date.date()}",
 'daily_findings': {},
 'severity_trends': {},
 'product_trends': {}
 }
 
 try:
 # Get findings created in the time period
 paginator = self.securityhub_client.get_paginator('get_findings')
 
 for page in paginator.paginate(
 Filters={
 'CreatedAt': [
 {
 'Start': start_date.isoformat() + 'Z',
 'End': end_date.isoformat() + 'Z'
 }
 ]
 }
 ):
 for finding in page['Findings']:
 created_date = finding.get('CreatedAt', '').split('T')[0]
 severity = finding.get('Severity', {}).get('Label', 'INFORMATIONAL')
 product = finding.get('ProductName', 'Unknown')
 
 # Daily findings count
 trend_data['daily_findings'][created_date] = trend_data['daily_findings'].get(created_date, 0) + 1
 
 # Severity trends
 if severity not in trend_data['severity_trends']:
 trend_data['severity_trends'][severity] = {}
 trend_data['severity_trends'][severity][created_date] = trend_data['severity_trends'][severity].get(created_date, 0) + 1
 
 # Product trends
 if product not in trend_data['product_trends']:
 trend_data['product_trends'][product] = {}
 trend_data['product_trends'][product][created_date] = trend_data['product_trends'][product].get(created_date, 0) + 1
 
 except Exception as e:
 trend_data['error'] = str(e)
 
 return trend_data
 
 def send_metrics_to_cloudwatch(self, metrics):
 """Send security metrics to CloudWatch"""
 
 metric_data = []
 
 # Findings metrics
 findings_summary = metrics.get('findings_summary', {})
 
 metric_data.append({
 'MetricName': 'TotalFindings',
 'Value': findings_summary.get('total_findings', 0),
 'Unit': 'Count',
 'Timestamp': datetime.utcnow()
 })
 
 # Severity metrics
 for severity, count in findings_summary.get('by_severity', {}).items():
 metric_data.append({
 'MetricName': f'{severity}SeverityFindings',
 'Value': count,
 'Unit': 'Count',
 'Timestamp': datetime.utcnow()
 })
 
 # Compliance metrics
 compliance_summary = metrics.get('compliance_summary', {})
 
 metric_data.append({
 'MetricName': 'OverallComplianceScore',
 'Value': compliance_summary.get('overall_score', 0),
 'Unit': 'Percent',
 'Timestamp': datetime.utcnow()
 })
 
 # Send to CloudWatch
 try:
 self.cloudwatch.put_metric_data(
 Namespace='SecurityHub/Metrics',
 MetricData=metric_data
 )
 except Exception as e:
 print(f"Error sending metrics to CloudWatch: {str(e)}")

7. Common Exam Scenarios

Scenario 1: Complete Security Hub Implementation

# Complete Security Hub setup with all integrations
Resources:
 # Enable Security Hub
 SecurityHub:
 Type: AWS::SecurityHub::Hub
 Properties:
 Tags:
 - Key: Purpose
 Value: CentralizedSecurityMonitoring

 # Enable all security standards
 AWSFoundationalStandard:
 Type: AWS::SecurityHub::Standard
 Properties:
 StandardsArn: !Sub "arn:aws:securityhub:::ruleset/finding-format/aws-foundational-security-standard/v/1.0.0"

 CISStandard:
 Type: AWS::SecurityHub::Standard
 Properties:
 StandardsArn: !Sub "arn:aws:securityhub:::ruleset/finding-format/cis-aws-foundations-benchmark/v/1.2.0"

 PCIStandard:
 Type: AWS::SecurityHub::Standard
 Properties:
 StandardsArn: !Sub "arn:aws:securityhub:::ruleset/finding-format/pci-dss/v/3.2.1"

 # Enable all product integrations
 GuardDutyIntegration:
 Type: AWS::SecurityHub::ProductSubscription
 Properties:
 ProductArn: !Sub "arn:aws:securityhub:${AWS::Region}::product/aws/guardduty"

 InspectorIntegration:
 Type: AWS::SecurityHub::ProductSubscription
 Properties:
 ProductArn: !Sub "arn:aws:securityhub:${AWS::Region}::product/aws/inspector"

 ConfigIntegration:
 Type: AWS::SecurityHub::ProductSubscription
 Properties:
 ProductArn: !Sub "arn:aws:securityhub:${AWS::Region}::product/aws/config"

 MacieIntegration:
 Type: AWS::SecurityHub::ProductSubscription
 Properties:
 ProductArn: !Sub "arn:aws:securityhub:${AWS::Region}::product/aws/macie"

 # Automated response system
 SecurityHubResponseFunction:
 Type: AWS::Lambda::Function
 Properties:
 FunctionName: security-hub-automated-response
 Runtime: python3.9
 Handler: index.lambda_handler
 Role: !GetAtt SecurityHubResponseRole.Arn
 Timeout: 300
 Code:
 ZipFile: |
 # Complete automated response code here
 import boto3
 import json
 
 def lambda_handler(event, context):
 # Process Security Hub findings and trigger appropriate responses
 return process_security_hub_findings(event)

 # EventBridge rules for different finding types
 CriticalFindingsRule:
 Type: AWS::Events::Rule
 Properties:
 EventPattern:
 source: [aws.securityhub]
 detail-type: [Security Hub Findings - Imported]
 detail:
 findings:
 Severity:
 Label: [CRITICAL]
 Targets:
 - Arn: !GetAtt SecurityHubResponseFunction.Arn
 Id: CriticalFindingsTarget

 ComplianceFailuresRule:
 Type: AWS::Events::Rule
 Properties:
 EventPattern:
 source: [aws.securityhub]
 detail-type: [Security Hub Findings - Imported]
 detail:
 findings:
 Compliance:
 Status: [FAILED]
 Targets:
 - Arn: !GetAtt ComplianceRemediationFunction.Arn
 Id: ComplianceFailuresTarget

8. Exam Tips

Understand centralized security - Security Hub as single pane of glass
Master multi-account setup - Organization integration and member management
Know product integrations - GuardDuty, Inspector, Config, Macie integration
Practice automated response - EventBridge and Lambda integration patterns
Learn security standards - AWS Foundational, CIS, PCI DSS standards
Understand custom insights - Creating custom views and filters
Know finding formats - ASFF (AWS Security Finding Format) structure
Practice remediation workflows - Automated and manual remediation patterns
Master troubleshooting - Common issues with integrations and findings
Understand compliance reporting - Standards compliance and audit requirements

AWS Systems Manager

Sun, 04 Oct 2026 00:00:00 +0000

1. Overview

AWS Systems Manager provides operational insights and enables automation of common administrative and operational tasks across AWS and on-premises resources. It’s a unified interface for managing infrastructure at scale.

Key Characteristics

Unified management - Single interface for AWS and on-premises resources
Automation - Runbooks, patch management, and operational tasks
Configuration management - Parameter Store and configuration compliance
Session management - Secure shell access without SSH keys or bastion hosts
Patch management - Automated patching across instances and environments
Operational insights - Centralized operational data and compliance reporting
Inventory management - Track software, configurations, and metadata

What Problem Does It Solve?

Centralizes operational management across hybrid environments
Automates routine administrative tasks and remediation workflows
Provides secure access to instances without managing SSH infrastructure
Enables configuration management and compliance at scale
Facilitates patch management and security updates
Supports operational insights and troubleshooting across fleets

2. Core Components

Systems Manager Agent (SSM Agent)

# Check SSM Agent status
sudo systemctl status amazon-ssm-agent

# Start SSM Agent
sudo systemctl start amazon-ssm-agent

# Enable SSM Agent on boot
sudo systemctl enable amazon-ssm-agent

# View SSM Agent logs
sudo tail -f /var/log/amazon/ssm/amazon-ssm-agent.log

# Update SSM Agent
sudo yum update amazon-ssm-agent

Instance Registration

# CloudFormation for SSM-managed instances
Resources:
 SSMRole:
 Type: AWS::IAM::Role
 Properties:
 RoleName: SSMInstanceRole
 AssumeRolePolicyDocument:
 Version: '2012-10-17'
 Statement:
 - Effect: Allow
 Principal:
 Service: ec2.amazonaws.com
 Action: sts:AssumeRole
 ManagedPolicyArns:
 - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
 - arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy

 SSMInstanceProfile:
 Type: AWS::IAM::InstanceProfile
 Properties:
 Roles:
 - !Ref SSMRole

 ManagedInstance:
 Type: AWS::EC2::Instance
 Properties:
 ImageId: ami-0abcdef1234567890
 InstanceType: t3.micro
 IamInstanceProfile: !Ref SSMInstanceProfile
 SecurityGroupIds:
 - !Ref SSMSecurityGroup
 SubnetId: !Ref PrivateSubnet
 UserData:
 Fn::Base64: !Sub |
 #!/bin/bash
 yum update -y
 yum install -y amazon-ssm-agent
 systemctl enable amazon-ssm-agent
 systemctl start amazon-ssm-agent
 Tags:
 - Key: Name
 Value: SSM-Managed-Instance
 - Key: Environment
 Value: Production
 - Key: Patch Group
 Value: WebServers

 SSMSecurityGroup:
 Type: AWS::EC2::SecurityGroup
 Properties:
 GroupDescription: Security group for SSM managed instances
 VpcId: !Ref VPC
 SecurityGroupEgress:
 - IpProtocol: tcp
 FromPort: 443
 ToPort: 443
 CidrIp: 0.0.0.0/0
 Description: HTTPS for SSM communication

3. Session Manager

Session Manager Configuration

Resources:
 SessionManagerPreferences:
 Type: AWS::SSM::Document
 Properties:
 DocumentType: Session
 DocumentFormat: JSON
 Name: SSM-SessionManagerRunShell
 Content:
 schemaVersion: '1.0'
 description: Document to hold regional settings for Session Manager
 sessionType: Standard_Stream
 inputs:
 s3BucketName: !Ref SessionLogsBucket
 s3KeyPrefix: session-logs/
 s3EncryptionEnabled: true
 cloudWatchLogGroupName: !Ref SessionLogsGroup
 cloudWatchEncryptionEnabled: true
 idleSessionTimeout: '20'
 maxSessionDuration: '60'
 runAsEnabled: false
 runAsDefaultUser: ssm-user
 shellProfile:
 windows: 'date'
 linux: 'pwd;whoami'

 SessionLogsBucket:
 Type: AWS::S3::Bucket
 Properties:
 BucketName: !Sub "${AWS::StackName}-session-logs"
 BucketEncryption:
 ServerSideEncryptionConfiguration:
 - ServerSideEncryptionByDefault:
 SSEAlgorithm: AES256
 PublicAccessBlockConfiguration:
 BlockPublicAcls: true
 BlockPublicPolicy: true
 IgnorePublicAcls: true
 RestrictPublicBuckets: true

 SessionLogsGroup:
 Type: AWS::Logs::LogGroup
 Properties:
 LogGroupName: /aws/ssm/session-logs
 RetentionInDays: 30

Session Manager CLI Usage

# Start interactive session
aws ssm start-session --target i-1234567890abcdef0

# Start session with specific document
aws ssm start-session \
 --target i-1234567890abcdef0 \
 --document-name AWS-StartPortForwardingSession \
 --parameters '{"portNumber":["3306"],"localPortNumber":["3306"]}'

# Start session with port forwarding
aws ssm start-session \
 --target i-1234567890abcdef0 \
 --document-name AWS-StartPortForwardingSessionToRemoteHost \
 --parameters '{"host":["mydb.cluster-xyz.us-east-1.rds.amazonaws.com"],"portNumber":["3306"],"localPortNumber":["3306"]}'

# List active sessions
aws ssm describe-sessions --state Active

# Terminate session
aws ssm terminate-session --session-id session-id

# Get session history
aws ssm get-connection-status --target i-1234567890abcdef0

4. Parameter Store

Parameter Management

# Create string parameter
aws ssm put-parameter \
 --name "/myapp/database/host" \
 --value "db.example.com" \
 --type "String" \
 --description "Database hostname"

# Create secure string parameter
aws ssm put-parameter \
 --name "/myapp/database/password" \
 --value "supersecret" \
 --type "SecureString" \
 --key-id "alias/parameter-store-key" \
 --description "Database password"

# Create string list parameter
aws ssm put-parameter \
 --name "/myapp/allowed-ips" \
 --value "10.0.0.1,10.0.0.2,10.0.0.3" \
 --type "StringList" \
 --description "Allowed IP addresses"

# Get parameter
aws ssm get-parameter --name "/myapp/database/host"

# Get parameter with decryption
aws ssm get-parameter --name "/myapp/database/password" --with-decryption

# Get parameters by path
aws ssm get-parameters-by-path \
 --path "/myapp/database" \
 --recursive \
 --with-decryption

# Get parameter history
aws ssm get-parameter-history --name "/myapp/database/password"

Parameter Store in CloudFormation

Resources:
 DatabaseHost:
 Type: AWS::SSM::Parameter
 Properties:
 Name: /myapp/database/host
 Type: String
 Value: !GetAtt DatabaseInstance.Endpoint.Address
 Description: Database endpoint
 Tags:
 Environment: !Ref Environment
 Application: MyApp

 DatabasePassword:
 Type: AWS::SSM::Parameter
 Properties:
 Name: /myapp/database/password
 Type: SecureString
 Value: !Ref DatabaseMasterPassword
 Description: Database master password
 KeyId: !Ref ParameterStoreKMSKey

 # Reference parameters in other resources
 LambdaFunction:
 Type: AWS::Lambda::Function
 Properties:
 Environment:
 Variables:
 DB_HOST: '{{resolve:ssm:/myapp/database/host}}'
 DB_PASSWORD: '{{resolve:ssm-secure:/myapp/database/password}}'
 Code:
 ZipFile: |
 import os
 import boto3
 
 def lambda_handler(event, context):
 # Parameters are automatically resolved
 db_host = os.environ['DB_HOST']
 db_password = os.environ['DB_PASSWORD']
 
 # Use parameters in application logic
 return {'statusCode': 200}

 # Parameter Store KMS Key
 ParameterStoreKMSKey:
 Type: AWS::KMS::Key
 Properties:
 Description: KMS key for Parameter Store encryption
 KeyPolicy:
 Version: '2012-10-17'
 Statement:
 - Sid: Enable IAM User Permissions
 Effect: Allow
 Principal:
 AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root"
 Action: "kms:*"
 Resource: "*"

5. Patch Manager

Patch Baseline Configuration

Resources:
 CustomPatchBaseline:
 Type: AWS::SSM::PatchBaseline
 Properties:
 Name: CustomLinuxBaseline
 Description: Custom patch baseline for Linux instances
 OperatingSystem: AMAZON_LINUX_2
 PatchGroups:
 - WebServers
 - DatabaseServers
 ApprovalRules:
 PatchRules:
 - PatchFilterGroup:
 PatchFilters:
 - Key: CLASSIFICATION
 Values:
 - Security
 - Bugfix
 - Key: SEVERITY
 Values:
 - Critical
 - Important
 ApproveAfterDays: 0
 ComplianceLevel: CRITICAL
 EnableNonSecurity: false
 - PatchFilterGroup:
 PatchFilters:
 - Key: CLASSIFICATION
 Values:
 - Enhancement
 - Recommended
 ApproveAfterDays: 7
 ComplianceLevel: MEDIUM
 ApprovedPatches:
 - kernel-4.14.123-*
 RejectedPatches:
 - kernel-4.14.100-*
 RejectedPatchesAction: BLOCK

 MaintenanceWindow:
 Type: AWS::SSM::MaintenanceWindow
 Properties:
 Name: PatchingMaintenanceWindow
 Description: Weekly patching window for production servers
 Schedule: cron(0 2 ? * SUN *)
 Duration: 4
 Cutoff: 1
 AllowUnassociatedTargets: false
 Tags:
 - Key: Environment
 Value: Production

 MaintenanceWindowTarget:
 Type: AWS::SSM::MaintenanceWindowTarget
 Properties:
 WindowId: !Ref MaintenanceWindow
 ResourceType: INSTANCE
 Targets:
 - Key: tag:PatchGroup
 Values:
 - WebServers
 Name: WebServersTarget
 Description: Web servers for patching

 PatchingTask:
 Type: AWS::SSM::MaintenanceWindowTask
 Properties:
 WindowId: !Ref MaintenanceWindow
 TaskType: RUN_COMMAND
 TaskArn: AWS-RunPatchBaseline
 Targets:
 - Key: WindowTargetIds
 Values:
 - !Ref MaintenanceWindowTarget
 ServiceRoleArn: !GetAtt MaintenanceWindowRole.Arn
 TaskParameters:
 Operation:
 - Install
 Priority: 1
 MaxConcurrency: "50%"
 MaxErrors: "10%"
 Name: PatchInstallationTask

Patch Compliance Monitoring

import boto3
import json
from datetime import datetime, timedelta

class PatchComplianceMonitor:
 def __init__(self):
 self.ssm_client = boto3.client('ssm')
 self.cloudwatch = boto3.client('cloudwatch')
 self.sns = boto3.client('sns')
 
 def generate_patch_compliance_report(self):
 """Generate comprehensive patch compliance report"""
 
 # Get all managed instances
 instances_response = self.ssm_client.describe_instance_information()
 
 compliance_data = {
 'report_timestamp': datetime.utcnow().isoformat(),
 'total_instances': len(instances_response['InstanceInformationList']),
 'compliant_instances': 0,
 'non_compliant_instances': 0,
 'instances_details': []
 }
 
 for instance in instances_response['InstanceInformationList']:
 instance_id = instance['InstanceId']
 
 try:
 # Get patch compliance for instance
 patch_state = self.ssm_client.describe_instance_patch_states(
 InstanceIds=[instance_id]
 )
 
 if patch_state['InstancePatchStates']:
 state = patch_state['InstancePatchStates'][0]
 
 instance_compliance = {
 'instance_id': instance_id,
 'platform_name': instance.get('PlatformName', 'Unknown'),
 'platform_version': instance.get('PlatformVersion', 'Unknown'),
 'installed_count': state['InstalledCount'],
 'missing_count': state['MissingCount'],
 'failed_count': state['FailedCount'],
 'not_applicable_count': state['NotApplicableCount'],
 'operation': state['Operation'],
 'operation_start_time': state.get('OperationStartTime', '').isoformat() if state.get('OperationStartTime') else None,
 'operation_end_time': state.get('OperationEndTime', '').isoformat() if state.get('OperationEndTime') else None
 }
 
 # Determine compliance status
 if state['MissingCount'] == 0 and state['FailedCount'] == 0:
 instance_compliance['compliance_status'] = 'COMPLIANT'
 compliance_data['compliant_instances'] += 1
 else:
 instance_compliance['compliance_status'] = 'NON_COMPLIANT'
 compliance_data['non_compliant_instances'] += 1
 
 # Get details of missing patches
 instance_compliance['missing_patches'] = self.get_missing_patches(instance_id)
 
 compliance_data['instances_details'].append(instance_compliance)
 
 except Exception as e:
 print(f"Error getting patch state for {instance_id}: {str(e)}")
 
 # Send compliance metrics to CloudWatch
 self.send_compliance_metrics(compliance_data)
 
 # Send alert if compliance is below threshold
 compliance_percentage = (compliance_data['compliant_instances'] / compliance_data['total_instances']) * 100
 if compliance_percentage < 90: # 90% compliance threshold
 self.send_compliance_alert(compliance_data, compliance_percentage)
 
 return compliance_data
 
 def get_missing_patches(self, instance_id):
 """Get details of missing patches for an instance"""
 
 try:
 patches_response = self.ssm_client.describe_instance_patches(
 InstanceId=instance_id,
 Filters=[
 {
 'Key': 'State',
 'Values': ['Missing']
 }
 ]
 )
 
 missing_patches = []
 for patch in patches_response['Patches']:
 missing_patches.append({
 'id': patch['Id'],
 'classification': patch['Classification'],
 'severity': patch['Severity'],
 'title': patch['Title'],
 'release_date': patch['ReleaseDate'].isoformat() if patch.get('ReleaseDate') else None
 })
 
 return missing_patches[:10] # Return first 10 missing patches
 
 except Exception as e:
 print(f"Error getting missing patches for {instance_id}: {str(e)}")
 return []
 
 def send_compliance_metrics(self, compliance_data):
 """Send patch compliance metrics to CloudWatch"""
 
 self.cloudwatch.put_metric_data(
 Namespace='AWS/SSM/PatchCompliance',
 MetricData=[
 {
 'MetricName': 'CompliantInstances',
 'Value': compliance_data['compliant_instances'],
 'Unit': 'Count',
 'Timestamp': datetime.utcnow()
 },
 {
 'MetricName': 'NonCompliantInstances',
 'Value': compliance_data['non_compliant_instances'],
 'Unit': 'Count',
 'Timestamp': datetime.utcnow()
 },
 {
 'MetricName': 'CompliancePercentage',
 'Value': (compliance_data['compliant_instances'] / compliance_data['total_instances']) * 100,
 'Unit': 'Percent',
 'Timestamp': datetime.utcnow()
 }
 ]
 )
 
 def send_compliance_alert(self, compliance_data, compliance_percentage):
 """Send compliance alert when threshold is not met"""
 
 message = f"""
 Patch Compliance Alert
 =====================
 
 Compliance Percentage: {compliance_percentage:.1f}%
 Total Instances: {compliance_data['total_instances']}
 Compliant Instances: {compliance_data['compliant_instances']}
 Non-Compliant Instances: {compliance_data['non_compliant_instances']}
 
 Non-compliant instances require immediate attention.
 Please review the patch compliance report for details.
 """
 
 self.sns.publish(
 TopicArn='arn:aws:sns:us-east-1:123456789012:patch-compliance-alerts',
 Message=message,
 Subject=f'Patch Compliance Alert - {compliance_percentage:.1f}% Compliant'
 )

6. Automation Documents

Custom Automation Document

Resources:
 CustomRemediationDocument:
 Type: AWS::SSM::Document
 Properties:
 DocumentType: Automation
 DocumentFormat: YAML
 Name: CustomSecurityRemediation
 Content:
 schemaVersion: '0.3'
 description: Automated security remediation for non-compliant instances
 assumeRole: '{{ AutomationAssumeRole }}'
 parameters:
 InstanceId:
 type: String
 description: EC2 Instance ID to remediate
 AutomationAssumeRole:
 type: String
 description: IAM role for automation execution
 default: ''
 NotificationTopicArn:
 type: String
 description: SNS topic for notifications
 default: ''
 mainSteps:
 - name: CheckInstanceStatus
 action: 'aws:executeAwsApi'
 inputs:
 Service: ec2
 Api: DescribeInstances
 InstanceIds:
 - '{{ InstanceId }}'
 outputs:
 - Name: InstanceState
 Selector: '$.Reservations[0].Instances[0].State.Name'
 Type: String
 
 - name: StopInstanceIfRunning
 action: 'aws:changeInstanceState'
 inputs:
 InstanceIds:
 - '{{ InstanceId }}'
 DesiredState: stopped
 isEnd: false
 onFailure: 'step:NotifyFailure'
 
 - name: CreateSecuritySnapshot
 action: 'aws:executeAwsApi'
 inputs:
 Service: ec2
 Api: CreateSnapshot
 VolumeId: '{{ GetVolumeId.VolumeId }}'
 Description: 'Security remediation snapshot for {{ InstanceId }}'
 outputs:
 - Name: SnapshotId
 Selector: '$.SnapshotId'
 Type: String
 
 - name: ApplySecurityPatches
 action: 'aws:runCommand'
 inputs:
 DocumentName: AWS-RunPatchBaseline
 InstanceIds:
 - '{{ InstanceId }}'
 Parameters:
 Operation: Install
 outputs:
 - Name: CommandId
 Selector: '$.CommandId'
 Type: String
 
 - name: UpdateSecurityGroups
 action: 'aws:executeAwsApi'
 inputs:
 Service: ec2
 Api: ModifyInstanceAttribute
 InstanceId: '{{ InstanceId }}'
 Groups:
 - sg-secure123456
 
 - name: StartInstance
 action: 'aws:changeInstanceState'
 inputs:
 InstanceIds:
 - '{{ InstanceId }}'
 DesiredState: running
 
 - name: VerifyCompliance
 action: 'aws:executeScript'
 inputs:
 Runtime: python3.8
 Handler: verify_compliance
 Script: |
 import boto3
 import time
 
 def verify_compliance(events, context):
 ssm = boto3.client('ssm')
 instance_id = events['InstanceId']
 
 # Wait for compliance scan
 time.sleep(300) # 5 minutes
 
 # Check compliance status
 response = ssm.describe_instance_patch_states(
 InstanceIds=[instance_id]
 )
 
 if response['InstancePatchStates']:
 state = response['InstancePatchStates'][0]
 if state['MissingCount'] == 0 and state['FailedCount'] == 0:
 return {'ComplianceStatus': 'COMPLIANT'}
 
 return {'ComplianceStatus': 'NON_COMPLIANT'}
 InputPayload:
 InstanceId: '{{ InstanceId }}'
 outputs:
 - Name: ComplianceStatus
 Selector: '$.Payload.ComplianceStatus'
 Type: String
 
 - name: NotifySuccess
 action: 'aws:executeAwsApi'
 inputs:
 Service: sns
 Api: Publish
 TopicArn: '{{ NotificationTopicArn }}'
 Message: 'Security remediation completed successfully for instance {{ InstanceId }}'
 Subject: 'Security Remediation Success'
 isEnd: true
 
 - name: NotifyFailure
 action: 'aws:executeAwsApi'
 inputs:
 Service: sns
 Api: Publish
 TopicArn: '{{ NotificationTopicArn }}'
 Message: 'Security remediation failed for instance {{ InstanceId }}'
 Subject: 'Security Remediation Failure'
 isEnd: true

Run Command Documents

# Execute run command
aws ssm send-command \
 --document-name "AWS-RunShellScript" \
 --parameters 'commands=["echo Hello World","uptime","df -h"]' \
 --targets "Key=tag:Environment,Values=Production" \
 --max-concurrency "10" \
 --max-errors "2"

# Execute PowerShell command on Windows
aws ssm send-command \
 --document-name "AWS-RunPowerShellScript" \
 --parameters 'commands=["Get-Process","Get-Service"]' \
 --targets "Key=instanceids,Values=i-1234567890abcdef0"

# Install application via run command
aws ssm send-command \
 --document-name "AWS-ConfigureAWSPackage" \
 --parameters 'action=Install,name=AmazonCloudWatchAgent' \
 --targets "Key=tag:Role,Values=WebServer"

# Get command execution results
aws ssm get-command-invocation \
 --command-id "command-id-here" \
 --instance-id "i-1234567890abcdef0"

7. State Manager

State Manager Association

Resources:
 CloudWatchAgentAssociation:
 Type: AWS::SSM::Association
 Properties:
 Name: AmazonCloudWatch-ManageAgent
 Targets:
 - Key: tag:Environment
 Values:
 - Production
 - Staging
 Parameters:
 action: configure
 mode: ec2
 optionalConfigurationSource: ssm
 optionalConfigurationLocation: !Ref CloudWatchAgentConfig
 optionalRestart: 'yes'
 ScheduleExpression: rate(30 minutes)
 ComplianceSeverity: MEDIUM
 MaxConcurrency: "50%"
 MaxErrors: "10%"

 SecurityComplianceAssociation:
 Type: AWS::SSM::Association
 Properties:
 Name: !Ref SecurityComplianceDocument
 Targets:
 - Key: tag:SecurityCompliance
 Values:
 - Required
 ScheduleExpression: cron(0 2 ? * SUN *)
 Parameters:
 ComplianceType: Security
 RemediationAction: Automatic
 ComplianceSeverity: HIGH

 # Custom compliance document
 SecurityComplianceDocument:
 Type: AWS::SSM::Document
 Properties:
 DocumentType: Command
 DocumentFormat: YAML
 Name: SecurityComplianceCheck
 Content:
 schemaVersion: '2.2'
 description: Security compliance check and remediation
 parameters:
 ComplianceType:
 type: String
 description: Type of compliance check
 default: Security
 RemediationAction:
 type: String
 description: Remediation action to take
 allowedValues:
 - Automatic
 - Manual
 default: Manual
 mainSteps:
 - action: 'aws:runShellScript'
 name: SecurityComplianceCheck
 inputs:
 runCommand:
 - |
 #!/bin/bash
 
 # Check SSH configuration
 if grep -q "PermitRootLogin yes" /etc/ssh/sshd_config; then
 echo "COMPLIANCE_VIOLATION: Root login enabled"
 if [ "{{ RemediationAction }}" == "Automatic" ]; then
 sed -i 's/PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config
 systemctl reload sshd
 echo "REMEDIATION: Root login disabled"
 fi
 fi
 
 # Check for unencrypted volumes
 unencrypted_volumes=$(lsblk -f | grep -v LUKS | grep -c ext4)
 if [ $unencrypted_volumes -gt 0 ]; then
 echo "COMPLIANCE_VIOLATION: Unencrypted volumes detected"
 fi
 
 # Check firewall status
 if ! systemctl is-active --quiet iptables; then
 echo "COMPLIANCE_VIOLATION: Firewall not active"
 if [ "{{ RemediationAction }}" == "Automatic" ]; then
 systemctl enable iptables
 systemctl start iptables
 echo "REMEDIATION: Firewall enabled"
 fi
 fi
 
 echo "Security compliance check completed"

8. Inventory and Compliance

Inventory Collection

Resources:
 InventoryAssociation:
 Type: AWS::SSM::Association
 Properties:
 Name: AWS-GatherSoftwareInventory
 Targets:
 - Key: InstanceIds
 Values:
 - "*"
 ScheduleExpression: rate(1 day)
 Parameters:
 applications: Enabled
 awsComponents: Enabled
 customInventory: Enabled
 files: |
 [
 {
 "Path": "/etc",
 "Pattern": ["*.conf", "*.cfg"],
 "Recursive": true
 },
 {
 "Path": "/var/log",
 "Pattern": ["*.log"],
 "Recursive": false
 }
 ]
 networkConfig: Enabled
 services: Enabled
 windowsRegistry: Enabled
 windowsRoles: Enabled

 # Custom inventory for application metadata
 CustomInventoryDocument:
 Type: AWS::SSM::Document
 Properties:
 DocumentType: Command
 DocumentFormat: YAML
 Name: CustomApplicationInventory
 Content:
 schemaVersion: '2.2'
 description: Collect custom application inventory
 mainSteps:
 - action: 'aws:runShellScript'
 name: CollectApplicationInventory
 inputs:
 runCommand:
 - |
 #!/bin/bash
 
 # Collect application versions
 app_inventory=$(cat << EOF
 {
 "SchemaVersion": "1.0",
 "TypeName": "Custom:ApplicationInventory",
 "Content": [
 {
 "ApplicationName": "MyWebApp",
 "Version": "$(cat /opt/myapp/VERSION 2>/dev/null || echo 'Unknown')",
 "InstallDate": "$(stat -c %y /opt/myapp 2>/dev/null || echo 'Unknown')",
 "Status": "$(systemctl is-active myapp 2>/dev/null || echo 'Unknown')"
 }
 ]
 }
 EOF
 )
 
 # Send inventory to Systems Manager
 aws ssm put-inventory \
 --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id) \
 --items "$app_inventory"

Compliance Monitoring

import boto3
import json
from datetime import datetime, timedelta

class SSMComplianceMonitor:
 def __init__(self):
 self.ssm_client = boto3.client('ssm')
 self.cloudwatch = boto3.client('cloudwatch')
 
 def get_compliance_summary(self):
 """Get overall compliance summary across all resources"""
 
 compliance_summary = {
 'timestamp': datetime.utcnow().isoformat(),
 'compliance_types': {},
 'overall_compliance': 0
 }
 
 # Get compliance summary by type
 response = self.ssm_client.list_compliance_summaries()
 
 total_compliant = 0
 total_resources = 0
 
 for summary in response['ComplianceSummaryItems']:
 compliance_type = summary['ComplianceType']
 compliant_count = summary['CompliantSummary']['CompliantCount']
 non_compliant_count = summary['CompliantSummary']['NonCompliantCount']
 
 total_compliant += compliant_count
 total_resources += compliant_count + non_compliant_count
 
 compliance_summary['compliance_types'][compliance_type] = {
 'compliant': compliant_count,
 'non_compliant': non_compliant_count,
 'compliance_percentage': (compliant_count / (compliant_count + non_compliant_count)) * 100 if (compliant_count + non_compliant_count) > 0 else 0
 }
 
 if total_resources > 0:
 compliance_summary['overall_compliance'] = (total_compliant / total_resources) * 100
 
 return compliance_summary
 
 def get_non_compliant_resources(self, compliance_type=None):
 """Get details of non-compliant resources"""
 
 filters = [
 {
 'Key': 'ComplianceType',
 'Values': [compliance_type] if compliance_type else ['Patch', 'Association', 'Custom:Security']
 },
 {
 'Key': 'Status',
 'Values': ['NON_COMPLIANT']
 }
 ]
 
 non_compliant_resources = []
 
 paginator = self.ssm_client.get_paginator('list_compliance_items')
 
 for page in paginator.paginate(Filters=filters):
 for item in page['ComplianceItems']:
 non_compliant_resources.append({
 'resource_id': item['ResourceId'],
 'resource_type': item['ResourceType'],
 'compliance_type': item['ComplianceType'],
 'status': item['Status'],
 'severity': item['Severity'],
 'execution_summary': item.get('ExecutionSummary', {}),
 'details': item.get('Details', {})
 })
 
 return non_compliant_resources
 
 def remediate_non_compliant_resources(self, resources):
 """Trigger remediation for non-compliant resources"""
 
 remediation_results = []
 
 for resource in resources:
 resource_id = resource['resource_id']
 compliance_type = resource['compliance_type']
 
 try:
 if compliance_type == 'Patch':
 # Trigger patch installation
 response = self.ssm_client.send_command(
 InstanceIds=[resource_id],
 DocumentName='AWS-RunPatchBaseline',
 Parameters={'Operation': ['Install']}
 )
 
 remediation_results.append({
 'resource_id': resource_id,
 'remediation_type': 'Patch Installation',
 'command_id': response['Command']['CommandId'],
 'status': 'INITIATED'
 })
 
 elif compliance_type == 'Association':
 # Re-run association
 associations = self.ssm_client.describe_instance_associations_status(
 InstanceId=resource_id
 )
 
 for assoc in associations['InstanceAssociationStatusInfos']:
 if assoc['Status'] != 'Success':
 self.ssm_client.send_command(
 InstanceIds=[resource_id],
 DocumentName=assoc['Name']
 )
 
 remediation_results.append({
 'resource_id': resource_id,
 'remediation_type': 'Association Re-run',
 'status': 'INITIATED'
 })
 
 except Exception as e:
 remediation_results.append({
 'resource_id': resource_id,
 'remediation_type': 'Failed',
 'error': str(e),
 'status': 'FAILED'
 })
 
 return remediation_results

9. Common Exam Scenarios

Scenario 1: Automated Patch Management

# Complete automated patch management setup
Resources:
 # Production patch baseline
 ProductionPatchBaseline:
 Type: AWS::SSM::PatchBaseline
 Properties:
 Name: Production-Linux-Baseline
 Description: Production patch baseline with strict approval rules
 OperatingSystem: AMAZON_LINUX_2
 PatchGroups:
 - Production-WebServers
 - Production-AppServers
 ApprovalRules:
 PatchRules:
 - PatchFilterGroup:
 PatchFilters:
 - Key: CLASSIFICATION
 Values: [Security]
 - Key: SEVERITY
 Values: [Critical, Important]
 ApproveAfterDays: 0
 ComplianceLevel: CRITICAL
 - PatchFilterGroup:
 PatchFilters:
 - Key: CLASSIFICATION
 Values: [Bugfix]
 - Key: SEVERITY
 Values: [Medium, Low]
 ApproveAfterDays: 7
 ComplianceLevel: MEDIUM

 # Maintenance window for production patching
 ProductionMaintenanceWindow:
 Type: AWS::SSM::MaintenanceWindow
 Properties:
 Name: Production-Patching-Window
 Description: Production patching during maintenance window
 Schedule: cron(0 3 ? * SUN *)  # 3 AM every Sunday
 Duration: 6
 Cutoff: 1
 AllowUnassociatedTargets: false

 # Maintenance window targets
 ProductionPatchTargets:
 Type: AWS::SSM::MaintenanceWindowTarget
 Properties:
 WindowId: !Ref ProductionMaintenanceWindow
 ResourceType: INSTANCE
 Targets:
 - Key: tag:Environment
 Values: [Production]
 - Key: tag:PatchGroup
 Values: [Production-WebServers, Production-AppServers]

 # Patch installation task
 PatchInstallationTask:
 Type: AWS::SSM::MaintenanceWindowTask
 Properties:
 WindowId: !Ref ProductionMaintenanceWindow
 TaskType: RUN_COMMAND
 TaskArn: AWS-RunPatchBaseline
 Targets:
 - Key: WindowTargetIds
 Values: [!Ref ProductionPatchTargets]
 ServiceRoleArn: !GetAtt MaintenanceWindowRole.Arn
 TaskParameters:
 Operation: [Install]
 Priority: 1
 MaxConcurrency: "25%"
 MaxErrors: "5%"

 # Post-patch compliance check
 ComplianceCheckTask:
 Type: AWS::SSM::MaintenanceWindowTask
 Properties:
 WindowId: !Ref ProductionMaintenanceWindow
 TaskType: RUN_COMMAND
 TaskArn: !Ref PostPatchComplianceDocument
 Targets:
 - Key: WindowTargetIds
 Values: [!Ref ProductionPatchTargets]
 ServiceRoleArn: !GetAtt MaintenanceWindowRole.Arn
 Priority: 2
 MaxConcurrency: "50%"
 MaxErrors: "10%"

 # Custom document for post-patch compliance
 PostPatchComplianceDocument:
 Type: AWS::SSM::Document
 Properties:
 DocumentType: Command
 DocumentFormat: YAML
 Name: PostPatchComplianceCheck
 Content:
 schemaVersion: '2.2'
 description: Post-patch compliance verification
 mainSteps:
 - action: 'aws:runShellScript'
 name: ComplianceCheck
 inputs:
 runCommand:
 - |
 #!/bin/bash
 
 # Check if reboot is required
 if [ -f /var/run/reboot-required ]; then
 echo "REBOOT_REQUIRED: System requires reboot"
 # Schedule reboot during next maintenance window
 echo "reboot" | at now + 5 minutes
 fi
 
 # Verify critical services are running
 services=("httpd" "mysqld" "nginx")
 for service in "${services[@]}"; do
 if systemctl is-active --quiet $service; then
 echo "SERVICE_OK: $service is running"
 else
 echo "SERVICE_ERROR: $service is not running"
 systemctl start $service
 fi
 done
 
 # Send compliance status to CloudWatch
 aws cloudwatch put-metric-data \
 --namespace "AWS/SSM/PatchCompliance" \
 --metric-data MetricName=PostPatchCheck,Value=1,Unit=Count

Scenario 2: Cross-Account Systems Management

# Cross-account Systems Manager setup
import boto3
import json

class CrossAccountSSMManager:
 def __init__(self):
 self.sts_client = boto3.client('sts')
 self.ssm_client = boto3.client('ssm')
 
 def setup_cross_account_access(self, target_accounts, cross_account_role_name):
 """Setup cross-account access for Systems Manager"""
 
 setup_results = {}
 
 for account_id in target_accounts:
 try:
 # Assume role in target account
 assumed_role = self.sts_client.assume_role(
 RoleArn=f"arn:aws:iam::{account_id}:role/{cross_account_role_name}",
 RoleSessionName='CrossAccountSSMSetup'
 )
 
 # Create session with assumed role credentials
 target_session = boto3.Session(
 aws_access_key_id=assumed_role['Credentials']['AccessKeyId'],
 aws_secret_access_key=assumed_role['Credentials']['SecretAccessKey'],
 aws_session_token=assumed_role['Credentials']['SessionToken']
 )
 
 target_ssm = target_session.client('ssm')
 
 # Setup resource data sync in target account
 sync_name = f"CrossAccountSync-{account_id}"
 
 try:
 target_ssm.create_resource_data_sync(
 SyncName=sync_name,
 S3Destination={
 'BucketName': 'central-ssm-inventory-bucket',
 'Prefix': f'account-{account_id}/',
 'SyncFormat': 'JsonSerDe',
 'Region': 'us-east-1'
 }
 )
 
 setup_results[account_id] = {
 'status': 'SUCCESS',
 'sync_name': sync_name
 }
 
 except target_ssm.exceptions.ResourceDataSyncAlreadyExistsException:
 setup_results[account_id] = {
 'status': 'EXISTS',
 'sync_name': sync_name
 }
 
 # Setup compliance data aggregation
 self.setup_compliance_aggregation(target_ssm, account_id)
 
 except Exception as e:
 setup_results[account_id] = {
 'status': 'FAILED',
 'error': str(e)
 }
 
 return setup_results
 
 def setup_compliance_aggregation(self, target_ssm, account_id):
 """Setup compliance data aggregation"""
 
 # Create association for compliance data collection
 association_name = f"ComplianceDataCollection-{account_id}"
 
 try:
 target_ssm.create_association(
 Name='AWS-GatherSoftwareInventory',
 Targets=[
 {
 'Key': 'InstanceIds',
 'Values': ['*']
 }
 ],
 ScheduleExpression='rate(1 day)',
 AssociationName=association_name,
 Parameters={
 'applications': ['Enabled'],
 'awsComponents': ['Enabled'],
 'networkConfig': ['Enabled'],
 'windowsUpdates': ['Enabled'],
 'instanceDetailedInformation': ['Enabled']
 }
 )
 
 except target_ssm.exceptions.AssociationAlreadyExists:
 pass # Association already exists
 
 def aggregate_compliance_data(self, accounts):
 """Aggregate compliance data from multiple accounts"""
 
 aggregated_data = {
 'timestamp': datetime.utcnow().isoformat(),
 'accounts': {},
 'summary': {
 'total_instances': 0,
 'compliant_instances': 0,
 'non_compliant_instances': 0
 }
 }
 
 for account_id in accounts:
 try:
 # Get compliance data for account
 account_compliance = self.get_account_compliance(account_id)
 aggregated_data['accounts'][account_id] = account_compliance
 
 # Update summary
 aggregated_data['summary']['total_instances'] += account_compliance['total_instances']
 aggregated_data['summary']['compliant_instances'] += account_compliance['compliant_instances']
 aggregated_data['summary']['non_compliant_instances'] += account_compliance['non_compliant_instances']
 
 except Exception as e:
 aggregated_data['accounts'][account_id] = {
 'error': str(e),
 'status': 'FAILED'
 }
 
 # Calculate overall compliance percentage
 if aggregated_data['summary']['total_instances'] > 0:
 compliance_percentage = (aggregated_data['summary']['compliant_instances'] / 
 aggregated_data['summary']['total_instances']) * 100
 aggregated_data['summary']['compliance_percentage'] = compliance_percentage
 
 return aggregated_data

10. Exam Tips

Understand SSM Agent - Installation, configuration, and troubleshooting
Master Session Manager - Secure access without SSH keys or bastion hosts
Know Parameter Store - Hierarchical parameters, encryption, and CloudFormation integration
Practice Patch Manager - Baselines, maintenance windows, and compliance reporting
Learn Automation - Documents, workflows, and cross-service integration
Understand State Manager - Associations, compliance, and configuration drift
Know Run Command - Document execution, targeting, and result handling
Practice Inventory - Collection, custom inventory, and compliance monitoring
Master troubleshooting - Common issues with agents, permissions, and connectivity
Understand integration - CloudWatch, SNS, Lambda, and cross-account scenarios

CloudWatch — General Notes (DOP-C02)

Sun, 04 Oct 2026 00:00:00 +0000

What it is: The observability backbone of AWS — metrics, alarms, logs, and dashboards. If you learn one service deeply for Domain 4, make it this one.

Exam-testable facts:

Metrics: organized by namespace (AWS/EC2) with up to 30 dimensions per metric. Statistics: Sum, Average, Minimum, Maximum, and percentiles (p50–p99). Standard resolution 60s; high-resolution down to 1s. Retained 15 months.
Alarms: three states — OK, ALARM, INSUFFICIENT_DATA. Metric alarms fire on thresholds; wire SNS actions to state transitions, not just ALARM.
Composite alarms: boolean AND/OR over child alarm states (not metrics). Pattern: suppress SNS on noisy child alarms, put the single page on the composite — one alert per incident instead of an alert storm.
Anomaly detection: ML builds a baseline from past data (hourly/daily/weekly patterns). ANOMALY_DETECTION_BAND(m1, N) draws a band N standard deviations wide; alarm when the metric goes above, below, or either side of the band. Needs roughly two weeks of history for a good model.
Metrics Insights: SQL-like queries across your metric fleet (“top 10 ECS services by CPU”) — for fleets where per-resource widgets don’t scale.
Metric math: combine metrics inside alarms/dashboards (e.g., error rate = errors ÷ requests) without precomputing.
Cross-account observability: a monitoring account linked to source accounts (via Organizations) can view metrics and create alarms/anomaly detectors on source-account data — no account switching.
Logs: log groups need explicit retention (default: never expire = unbounded cost); Logs Insights query language; metric filters turn log patterns into metrics/alarms while subscription filters stream log events to Lambda/Kinesis/Firehose; Embedded Metric Format (EMF) emits custom metrics straight from structured logs.
EC2 action alarms: arn:aws:automate:<region>:ec2:stop|terminate|reboot|recover — remediate without writing a Lambda.
Application Signals (2024): auto-instruments application performance (latency, error rate, availability) using OpenTelemetry — no manual SDK instrumentation needed. Surfaces SLOs (Service Level Objectives) directly in CloudWatch. The modern replacement for manually wiring X-Ray + custom metrics for application health.
Internet Monitor (2023): monitors the health of your application’s internet path from AWS to end users. Detects when an AWS network issue or internet provider problem is degrading connectivity for a subset of users — and tells you which city/ISP is affected. Publishes health events to EventBridge for automated response.

Common traps:

CloudWatch Deep Dive — CrossAccountObservability

Sun, 04 Oct 2026 00:00:00 +0000

What it is: CloudWatch cross-account observability lets a central monitoring account see metrics (and build alarms, dashboards, anomaly detectors) from source accounts — typically wired through AWS Organizations — with no account switching.

Exam-testable facts:

Two roles: source accounts share their observability data; the monitoring account views it. Setup is easiest with Organizations (all accounts in the org can be linked at once).
What flows: CloudWatch metrics, Logs Insights queries across accounts, and cross-account alarms. You create the alarm once in the monitoring account against source-account data.
No data duplication: the monitoring account queries source data in place; metrics aren’t copied over.
Permissions: source accounts grant access via the CloudWatch-CrossAccountSharing IAM role pattern (or Organizations trust); the monitoring account’s users need permission to use the monitoring console views.
Exam signal: “centralized monitoring dashboard for 50 accounts without switching” → cross-account observability. “Aggregate logs centrally for compliance” → that leans toward a central logging account with S3/Kinesis, a different pattern.

Common traps:

CloudWatch Deep Dive — MetricsInsights

Sun, 04 Oct 2026 00:00:00 +0000

What it is: CloudWatch Metrics Insights is a SQL-like query engine for your metrics. Instead of building one dashboard widget per resource, you write a single query that runs across your whole fleet — “show me the 10 ECS services with the highest CPU right now.”

Exam-testable facts:

SQL dialect: SELECT, FROM (namespace + metric name, or SCHEMA for dimension filtering), WHERE, GROUP BY, ORDER BY, LIMIT. Example shape: SELECT AVG(CPUUtilization) FROM SCHEMA("AWS/ECS", ClusterName, ServiceName) GROUP BY ClusterName, ServiceName ORDER BY AVG() DESC LIMIT 10.
SCHEMA() function: the key to fleet-wide queries — it selects metrics by their dimensions instead of naming each resource.
Alarms on queries: a Metrics Insights query can back a CloudWatch alarm, so one alarm covers a dynamic fleet (e.g. “alarm if any service’s error rate exceeds 5%”) — no per-service alarms to maintain.
Use case signal: the exam reaches for Metrics Insights when the scenario involves many resources and per-resource dashboards/alarms don’t scale.
Limits: queries time out after a bounded execution window; results are paginated. It queries metrics, not logs (that’s Logs Insights).

Common traps:

Config — General Notes (DOP-C02)

Sun, 04 Oct 2026 00:00:00 +0000

What it is: The configuration recorder and compliance evaluator — the detective control answering “is my infrastructure still compliant, and what changed?”

Exam-testable facts:

Config rules: managed (AWS-provided, e.g., “S3 buckets must be encrypted”) or custom (Lambda-backed or Guard CUSTOM_POLICY). Triggered on configuration changes, on a schedule, or both.
Conformance packs: a bundle of rules plus remediation actions, deployed as one YAML unit (backed by CloudFormation). Organizational conformance packs roll out from the management/delegated-admin account across the whole Organization.
Remediation: automatic via SSM Automation documents, or manual. Can ship inside the conformance pack — the self-healing compliance loop (Config detects → SSM remediates → Config re-evaluates).
Proactive evaluation: evaluate resource configuration before deployment — the preventative flavor, pairing with Guard in pipelines.
Aggregators: one dashboard for compliance across accounts and Regions.
Prerequisites: a recorder (what to record) + delivery channel (S3 bucket) + IAM role; without the recorder running, nothing is evaluated.

Common traps:

X-Ray — General Notes (DOP-C02)

Sun, 04 Oct 2026 00:00:00 +0000

What it is: Distributed tracing — follow one request across microservices and see exactly where time went and what failed. Complements CloudWatch metrics (what) and logs (details) with the where.

Exam-testable facts:

Trace = one end-to-end request. Segment = one service’s work within that trace. Subsegment = a downstream call inside a segment (DB query, HTTP call, AWS SDK call).
Annotations vs metadata — the classic exam question: annotations are indexed key-value pairs (searchable, filterable); metadata is not indexed (debugging context only, max 64 KB per segment).
Sampling rules: a reservoir (fixed number of traces per second, always kept) plus a rate (percentage of the rest). Rules have priorities; tune them to control cost at scale.
Service map: auto-generated topology of services and edges with latency/error/fault stats per node — the first place to look for “which service is slow.”
Trace header X-Amzn-Trace-Id propagates sampling decisions across service boundaries.
Collection: instrument with the X-Ray SDK; the daemon (UDP) or ADOT collector forwards segments. Lambda needs no daemon — enable active tracing and grant AWSXRayWriteOnlyAccess.
X-Ray Insights: ML-driven anomaly detection over trace data.

Common traps:

Domain 4: Monitoring and Logging on Album69

Amazon CloudWatch Synthetics

1. Overview

2. Canaries

AWS CloudTrail

1. Overview

Key Characteristics

What Problem Does It Solve?

2. Core Concepts

Trail

Event

Management Events

Data Events

Insight Events

Event History

3. Trail Configuration

Basic Trail Configuration

CloudFormation Template

4. Event Structure

Standard Event Format

Key Event Fields

5. IAM Roles & Permissions

CloudTrail Service Role

S3 Bucket Policy for CloudTrail

Cross-Account CloudTrail Access

User Permissions for CloudTrail Analysis

6. Integration with AWS Services

CloudWatch Logs Integration

EventBridge Integration

EventBridge Rule Example

AWS Config Integration

Amazon Athena Integration

Athena Table Creation

7. Security Best Practices

Encryption

S3 Bucket Security

Access Control

Monitoring

8. Monitoring & Troubleshooting

CloudWatch Metrics

CloudWatch Alarms

Common Issues

Trail Not Logging

Missing Events

Log Delivery Delays

Troubleshooting Commands

9. Common Exam Scenarios

Scenario 1: Detect root account usage

AWS CloudWatch Logs

1. Overview

Key Characteristics

What Problem Does It Solve?

2. Core Components

Log Groups and Log Streams

CloudFormation Configuration

3. Log Agents and Collection

CloudWatch Agent Configuration

Container Logging

4. Log Insights and Querying

CloudWatch Logs Insights Queries

Programmatic Log Analysis

5. Subscription Filters and Real-time Processing

Lambda Processing

Kinesis Data Streams Integration

6. Cross-Account Log Sharing

Cross-Account Access Setup

Source Account Configuration

7. Security and Encryption

Encryption Configuration

IAM Policies for Log Access

8. Integration with CI/CD

CodePipeline Integration

Application Deployment Monitoring

9. Common Exam Scenarios

Scenario 1: Centralized Logging Architecture

Scenario 2: Real-time Log Monitoring and Alerting

10. Exam Tips

AWS GuardDuty

1. Overview

Key Characteristics