How to Enable Logging on Every AWS Service in Existence (Circa 2021)

Image for post
Image for post

Cloud security best practices, as well as most compliance programs, require that logging be enabled for all in-scope services. However, that simple requirement — “enable logging” — comes with many followup questions. Is CloudTrail enough? How do I turn on logging for all these services? Aren’t logs collected by default? What. even. is. a. log?

In AWS, logging, like most tasks, isn’t as simple as it seems it could be, due to an inconsistent use of defaults, differing destination logging services, and a variety of configuration options, sometimes hidden in layers of submenus and API parameters. I guess we can blame the pizza teams for this too.

After using a handful of AWS services, you’ll notice that some send their logs to CloudWatch Logs (e.g. Lambda), others go to S3 (e.g. ELB and CloudFront), and still others wind up going to Kinesis (e.g. CloudFront’s new realtime logs).

And so I entered a rabbit hole of Googling, AWS documentation, and spinning up test services with names like “deletethisasap” in order to write the definitive guide to answer the question “how do I enable logging?” for every supported AWS service (although I make no guarantees; AWS released five new services just as I was typing this intro).

TL;DR

If you’re just looking for a list of services: Here.

Prelude

Aside from the first item (“The AWS Account Itself”), the list of AWS services below is specifically regarding logging system-level activity (e.g. access logs to S3 buckets, request logs to ELBs, etc.) and not API-level activity of the services themselves, which go to CloudTrail. There are many services not listed here which do log their API activity to CloudTrail, but do not support any additional logging.

The data below is sourced from hundreds of pages of AWS documentation. Where possible, I’ve included direct links to the source. If you note any inaccuracies or missing services, please email me.

The AWS Account Itself

Let’s start with the single most important logging service: CloudTrail. CloudTrail monitors AWS API activity across nearly every AWS service (although there are a handful that are not yet supported). API activity includes information such as the user agent, IP address, IAM user or role ARN, and other details about the request. CloudTrail is enabled by default for all AWS accounts, with 90 days of history, but should be configured to send logs to an S3 bucket (preferably in a different AWS account) for long-term storage.

CloudTrail also supports “Data Events” for S3 and KMS, which include much more granular access logs for S3 objects and KMS keys (such as encrypt and decrypt operations). This level of detail may not be necessary for most use cases, but can be very helpful in the event of a data breach.

What Gets Logged?
API activity across many AWS services.

Where Do They Go?
S3 and (optionally) CloudWatch Logs.

Are They Enabled by Default?
Yes, but should be configured with S3 for long-term storage or CloudWatch to enable metrics.

How to Enable?
[Link]

AWS ACM (Certificate Manager)

Technically-speaking, ACM doesn’t support logs for the underlying service, other than sending API-level events to CloudTrail. However, AWS publishes certificate transparency logs publicly, so in the interest of completeness, I am including those here. These are enabled by default, but can be disabled in cases of internal or test domains.

What Gets Logged?
Certificate transparency logs

Where Do They Go?
Public certificate transparency logs. These can be searched via crt.sh.

Are They Enabled by Default?
Yes.

How to Disable?
[Link]

Amazon API Gateway

What Gets Logged?
Execution (related to the invocation of API Gateway stages and underlying resources) and access logs (related to requests made to the API Gateway endpoints).

Where Do They Go?
CloudWatch Logs

Are They Enabled by Default?
No.

How to Enable?
[Link]

AWS Amplify

Amplify logs don’t appear to get saved anywhere other than the Amplify console, where they can be downloaded as a CSV. AWS recommends uploading those files to S3, where they can then be queried by Athena, but of course, provides no way of automating that process.

What Gets Logged?
Access logs to the Amplify app.

Where Do They Go?
The Amplify Dashboard

Are They Enabled by Default?
Yes.

How to Enable?
[Link]

AWS App Mesh

App Mesh logs are a bit different than other service logs because they are produced on the virtual nodes and use container-level primitives such as sending logs to stdout and stderr. From there, the logs can be proxied to CloudWatch Logs via an on-host agent.

What Gets Logged?
Envoy access and proxy logs.

Where Do They Go?
/dev/stdout — mixed with Envoy container logs.

Are They Enabled by Default?
No.

How to Enable?
[Link]

AWS AppSync

AppSync supports two types of logs that can be sent to CloudWatch: request-level logs and field-level logs (the link below has more details on what is included in each). AWS will handle the provisioning of log groups and streams accordingly.

What Gets Logged?
Request-level and field-level logs for the GraphQL API.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
No.

How to Enable?
[Link]

AWS Audit Manager

Audit Manager doesn’t appear to export its logs to any additional service, but you can access them via the UI or API.

What Gets Logged?
User activity related to selected controls.

Where Do They Go?
Internal database in the Audit Manager service.

Are They Enabled by Default?
Yes.

How to Enable?
[Link]

Amazon Aurora

Amazon Aurora databases support “activity streams” that can be sent to Kinesis. These are much more detailed logs than the standard RDS logs.

What Gets Logged?
Detailed activity logs.

Where Do They Go?
Kinesis.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon Auto Scaling (EC2)

Bear with me here, because Auto Scaling doesn’t really produce “logs” per se. However, each time an auto scaling event occurs (e.g. scale out, scale in), a notification is produced which looks suspiciously like a log. These events can be viewed in the Auto Scaling console (and retrieved via the API), but they can also be sent to SNS where they can then be streamed to other services.

What Gets Logged?
Auto Scaling events.

Where Do They Go?
An internal Auto Scaling database of event activity, but optionally to SNS.

Are They Enabled by Default?
Yes, but SNS integration must be configured via Auto Scaling notifications.

How to Enable?
[Link]

AWS Batch

What Gets Logged?
Batch job logs.

Where Do They Go?
To the container instance (assuming /dev/stdout), but they can be sent to CloudWatch Logs via the CloudWatch agent. AWS also added support for custom log drivers, such as Splunk or Fluentd.

Are They Enabled by Default?
Yes — to stdout, but integration with CloudWatch Logs or other services must be configured manually.

How to Enable?
[Link] [Another Link]

Amazon Chime

What Gets Logged?
Voice Connector media quality metrics and SIP message logs.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon CloudFormation

Similarly to Auto Scaling, CloudFormation produces events that look like logs. These are available in the CloudFormation console UI under the “Stack Events” tab, or via the API. However, these event notifications can also be sent to SNS topics for proxying to additional locations.

What Gets Logged?
Stack events.

Where Do They Go?
An internal CloudFormation database, however they can be retrieved via the console/API and sent to SNS.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon CloudFront

CloudFront will optionally bundle and deliver logs to an S3 bucket. If you need access logs in realtime, CloudFront recently added support for delivery of logs to a Firehose stream.

What Gets Logged?
Access logs for requests made to the distribution.

Where Do They Go?
A configurable S3 bucket (standard logs) or Kinesis Firehose data stream (realtime logs).

Are They Enabled by Default?
No.

How to Enable?
[Link]

AWS CloudHSM

What Gets Logged?
HSM audit logs.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
Yes. And they cannot be disabled.

How to Enable?
[Link]

AWS CloudTrail

CloudTrail is a unique service because it collects access and activity logs for the AWS account and nearly all services. However, CloudTrail then writes those logs to an S3 bucket. For compliance purposes, access logs to that bucket should then be written (to another S3 bucket). CloudWatch Events is also another AWS service which collects many of the same event types as CloudTrail (write-level logs, not read) but in realtime.

What Gets Logged?
API activity for most API actions within the AWS account.

Where Do They Go?
An S3 bucket (in 15 minute delivery intervals) and the CloudWatch Events service (in realtime).

Are They Enabled by Default?
Yes. Although you need to configure a trail for long-term storage and data-level events.

How to Enable?
[Link]

Amazon CloudWatch Logs

Most of the services listed in this guide send their logs to CloudWatch Logs. However, CloudWatch Logs are both a destination, as well as a source. Using the “export” and “stream” functionalities, CloudWatch Logs can be manually exported to S3 for long-term storage, or streamed to subscriptions such as Lambda, a Kinesis Data Stream, or Kinesis Data Firehose Stream.

What Gets Logged?
Whatever you send to CloudWatch Logs (see the rest of this guide).

Where Do They Go?
If you configure a manual export: S3. If you configure a subscription: Lambda, Kinesis Data Stream, or Kinesis Data Firehose Stream.

Are They Enabled by Default?
No, subscriptions and exports must be manually configured.

How to Enable?
[Link]

AWS CodeBuild

What Gets Logged?
Job build logs.

Where Do They Go?
The CodeBuild UI by default; optionally to S3 and/or CloudWatch Logs.

Are They Enabled by Default?
CloudWatch Logs are. S3 are not. Both can be checked or unchecked during job creation.

How to Enable?
[Link]

AWS CodeDeploy

What Gets Logged?
Deployment logs for EC2 or on-premise deployments. Deployments to Lambda or ECS are not logged.

Where Do They Go?
To a log file in the /opt/codedeploy-agent folder of the EC2 instance. These logs can be sent to CloudWatch Logs using the CloudWatch agent.

Are They Enabled by Default?
Local instance logs are; proxying to CloudWatch Logs must be configured manually.

How to Enable?
[Link]

AWS Config

Config is another service that doesn’t technically produce its own logs, but rather aggregates changes made to other AWS services. These changes are saved in files in an S3 bucket (which should have bucket access logging enabled). However, changes to supported resources can also be routed to an SQS queue, so for the sake of completeness, I am listing the Config service here.

What Gets Logged?
Changes to other supported AWS resources.

Where Do They Go?
Change files are saved in S3. Change notifications can be sent to SQS.

Are They Enabled by Default?
No. Config itself must be enabled, and routing to SQS must then be configured separately.

How to Enable?
[Link]

Amazon Connect

What Gets Logged?
Connect contact flows.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
Yes. A CloudWatch Log group is automatically created for Connect instances. However, depending on how the deployment was made (e.g. via the UI, CloudFormation, API, etc.), you may need to manually enable logging for each contact flow.

How to Enable?
[Link]

AWS Data Pipeline

What Gets Logged?
Pipeline task logs.

Where Do They Go?
If configured: S3. Otherwise they are ephemerally stored in the console for short-term access.

Are They Enabled by Default?
No. Short-term logs can be viewed in the console, but long-term log storage in S3 must be configured at the pipeline-level.

How to Enable?
[Link]

AWS DataSync

What Gets Logged?
Detailed logging for files and objects copied between your NFS servers, SMB servers, Amazon S3 buckets, Amazon Elastic File System (EFS) file systems, and Amazon FSx for Windows File Server file systems. […]

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
No.

How to Enable?
[Link]

AWS Directory Service

What Gets Logged?
Domain controller logs for Managed Microsoft AD directory.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
No.

How to Enable?
[Link]

AWS DMS (Database Migration Service)

What Gets Logged?
Migration task logs.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon DocumentDB

What Gets Logged?
Profiler logs of the execution time and details of operations that were performed on the cluster.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon DynamoDB

It’s not exactly a “log” in the traditional sense, but DynamoDB supports the concept of “streams” of activity that capture changes to items stored in a DynamoDB table. This is helpful for auditing, but may be overkill for most compliance purposes.

What Gets Logged?
Changes to items stored in a DynamoDB table.

Where Do They Go?
Either to DynamoDB or Kinesis, depending on configuration.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon EC2

Instance logging is a complicated subject because there are 10,000 ways to do it. Typically, most organizations use a third-party monitoring service such as Splunk or Sumo Logic which would cover this requirement. However, if you want to keep your logs inside the AWS ecosystem, the CloudWatch agent can be used to forward logs directly to CloudWatch.

What Gets Logged?
Application logs, syslogs, security logs, or whatever else you configure.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon ECS (Elastic Container Service)

Like other container-based services on AWS, logs for ECS are created in the containers themselves and then forwarded to any of several places. ECS is also a bit more complex because it supports both managed container instances (Fargate) and self-hosted via EC2 hosts.

What Gets Logged?
Container application, error, and other logs.

Where Do They Go?
CloudWatch Logs, or a third-party service via support log drivers.

Are They Enabled by Default?
Yes, for Fargate. But EC2 requires the configuration of the CloudWatch agent.

How to Enable?
[Link]

Amazon EKS (Elastic Kubernetes Service)

EKS produces both control plane logs (the control plane is managed by AWS), worker node logs (the compute plane is self-managed via EC2 instances or AWS-managed via Fargate), and task container logs (your application logs). Because of this, there are many logging sources and many ways of collecting logs. I will refer to the control plane logs below. The other log types are either covered by other sections of this guide (see “ECS” above) or via Kubernetes logging guides.

What Gets Logged?
Kubernetes control plane API, audit, controller, authenticator, and scheduler logs.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon ElastiCache for Redis

Although AWS does not provide access to the underlying ElastiCache host logs, it does produce events for different cluster-level activities (failure or success in adding new nodes, modifications to the security groups, etc.). Therefore, for the sake of completeness, I am including those events here.

What Gets Logged?
Cluster events.

Where Do They Go?
SNS.

Are They Enabled by Default?
No.

How to Enable?
[Link]

AWS Elastic Beanstalk

EBS uses EC2 instances, and therefore getting instance logs to CloudWatch Logs requires the use of the CloudWatch agent. Interestingly, the EB CLI appears to have a utility to fetch the instance logs (without the use of CloudWatch), although this is more for debug purposes, not long-term archiving.

What Gets Logged?
Instance application or error logs.

Where Do They Go?
To the local filesystem, although the CloudWatch agent can forward them to CloudWatch Logs.

Are They Enabled by Default?
Yes, but CloudWatch forwarding is not.

How to Enable?
[Link]

AWS ELB (Elastic Load Balancer)

Both ELB and ELBv2 (ALB and NLB) send their access logs to an S3 bucket. You can also configure the delivery interval — either 5 or 60 minutes.

What Gets Logged?
Load balancer request access logs.

Where Do They Go?
S3.

Are They Enabled by Default?
No.

How to Enable?
[Link]

AWS EMR (Elastic Map Reduce)

EMR writes its logs to the node itself, but can also be configured to archive logs to S3.

What Gets Logged?
Step, bootstrap action, and instance state logs.

Where Do They Go?
S3.

Are They Enabled by Default?
Yes, if you launched the EMR cluster via the console. Other deployment methods, such as CloudFormation, may require additional optional parameters to be set.

How to Enable?
[Link]

Amazon Kinesis Data Firehose

What Gets Logged?
Error logs when the Lambda invocation for data transformation or data delivery fails.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
No.

How to Enable?
[Link]

AWS Global Accelerator

Global Accelerator logs are an interesting case — the logs are called “Flow Logs” (not to be confused with VPC Flow Logs, despite doing essentially the same thing), and are sent directly to S3 although they also appear to incur CloudWatch Logs charges.

What Gets Logged?
Flow logs of network traffic across network interfaces in the accelerator.

Where Do They Go?
S3.

Are They Enabled by Default?
No.

How to Enable?
[Link]

AWS IoT Greengrass

What Gets Logged?
Greengrass Core software logs and Lambda functions or connectors (running on the core) logs.

Where Do They Go?
CloudWatch Logs (and the local filesystem).

Are They Enabled by Default?
Yes, but only to the local filesystem. CloudWatch Logs requires configuration.

How to Enable?
[Link]

Amazon Image Builder (EC2)

What Gets Logged?
Build and debug logs.

Where Do They Go?
CloudWatch Logs and (optionally) S3.

Are They Enabled by Default?
Yes (CloudWatch Logs). S3 requires configuration.

How to Enable?
[Link]

AWS Import/Export

What Gets Logged?
Import job logs.

Where Do They Go?
A log file, which can be sent to S3 (or downloaded).

Are They Enabled by Default?
Yes, but S3 upload requires configuration.

How to Enable?
[Link] (PDF Warning) See page 61.

Amazon Kafka (Managed Streaming for Apache Kafka)

What Gets Logged?
Kafka broker logs.

Where Do They Go?
CloudWatch Logs, S3, or Amazon Kinesis Data Firehose.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon Kendra

What Gets Logged?
Errors from your data source that occur while your documents are being indexed.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
Yes.

How to Enable?
[Link]

Amazon Kinesis Data Analytics

What Gets Logged?
Data analytics task, application, and operator logs.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
No.

How to Enable?
[Link]

AWS Lambda

What Gets Logged?
Application logs sent to stdout/stderr from your application.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
Yes (but your function must have permission to create log groups/streams).

How to Enable?
[Link]

Amazon Lex

Note: Lex logs likely don’t need to be enabled for compliance purposes; they are more verbose and related to conversations recorded with your Lex bots. There may also be privacy implications when enabling these logs.

What Gets Logged?
Conversation and audio logs from bots.

Where Do They Go?
CloudWatch Logs (conversation) and S3 (audio).

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon Lightsail

Lightsail is an interesting service because it abstracts many of the underlying resources (EC2 instances, containers, RDS databases, etc.) from the user in favor of a simple interface with predefined application launch types. For this reason, logging appears to be self-contained within Lightsail, only accessible via the Lightsail UI or APIs, and not externally via CloudWatch.

What Gets Logged?
Database, application, container, and error logs.

Where Do They Go?
Somewhere internal to the Lightsail platform, although they can be retrieved from the Lightsail console UI.

Are They Enabled by Default?
Yes.

How to Enable?
[Link — Database Logs] [Link — Container Logs]

Amazon MQ

What Gets Logged?
ActiveMQ and RabbitMQ logs.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon Neptune

Unlike most other services, Neptune logs don’t appear to go anywhere else except the Neptune service, and can only be accessed via the Neptune console by clicking “Download.”

What Gets Logged?
Amazon Neptune DB cluster activity logs.

Where Do They Go?
Internal Neptune service.

Are They Enabled by Default?
No.

How to Enable?
[Link]

AWS Network Firewall

What Gets Logged?
Firewall network flow logs.

Where Do They Go?
CloudWatch Logs, S3, and/or Kinesis Data Firehose.

Are They Enabled by Default?
No.

How to Enable?
[Link]

AWS OpsWorks

What Gets Logged?
Chef and application logs.

Where Do They Go?
Local filesystem, and (if enabled) CloudWatch Logs.

Are They Enabled by Default?
Yes, but only to the local filesystem. CloudWatch Logs requires configuration.

How to Enable?
[Link]

Amazon (QLDB) Quantum Ledger Database

QLDB captures every document revision that is committed to your journal and delivers this data to Amazon Kinesis Data Streams in near-real time.

What Gets Logged?
Document revisions.

Where Do They Go?
Kinesis Data Streams.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon RDS (Relational Database Service)

RDS has supported instance-level logs (including error and slow query logs) for some time. However, these logs can also be streamed to CloudWatch Logs via an opt-in configuration.

What Gets Logged?
General, error, audit, and slow query database logs.

Where Do They Go?
Internal RDS service (accessible via the RDS console or API) and (optionally) streamed to CloudWatch Logs.

Are They Enabled by Default?
Yes, but only to the RDS service. CloudWatch Logs requires configuration.

How to Enable?
[Link]

Amazon Redshift

What Gets Logged?
Redshift instance audit logs.

Where Do They Go?
S3.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon Route53

What Gets Logged?
Public query logs and resolver query logs for private VPCs.

Where Do They Go?
CloudWatch Logs (public query logs and resolver query logs), S3 (resolver query logs), and/or Kinesis Data Firehose (resolver query logs).

Are They Enabled by Default?
No.

How to Enable?
[Link — public query logs][Link — resolver query logs]

Amazon S3

S3 buckets can log their access logs… to another S3 bucket. Which begs the question: should you enable access logs on the access logs bucket? And if so, where do those go? I’ll leave that exercise to your compliance team.

What Gets Logged?
Server access logs.

Where Do They Go?
S3.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon SageMaker

What Gets Logged?
Jupyter notebook job logs.

Where Do They Go?
The local filesystem of the instance, but also (optionally) to CloudWatch Logs.

Are They Enabled by Default?
Yes, but only to the local filesystem. CloudWatch Logs requires additional configuration.

How to Enable?
[Link]

Amazon SNS (Simple Notification Service)

Interestingly, SNS does write to CloudWatch Logs, but only for a subset of the delivery types it supports (SMS).

What Gets Logged?
SMS delivery failure logs.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
Yes.

How to Enable?
[Link]

AWS Step Functions

There are multiple ways to create Step Functions (Standard and Express workflows). The Express Workflow, when configured via the UI, will auto-create the necessary prerequisites to enable CloudWatch Logs. However, the Standard Workflow requires additional configuration.

What Gets Logged?
Execution history events.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
No (unless using the Express Workload creation tool).

How to Enable?
[Link]

AWS WAF (Web Application Firewall)

What Gets Logged?
WAF request and response traffic logs.

Where Do They Go?
Kinesis Data Firehose. You can also configure these to be sent from Kinesis to S3 and then, if you are using Shield Advanced, provide access to the logs to the Shield team.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon WorkDocs

What Gets Logged?
Site-wide activity.

Where Do They Go?
Internal WorkDocs service, accessible via the WorkDocs UI.

Are They Enabled by Default?
Yes.

How to Enable?
[Link]

Amazon WorkMail

What Gets Logged?
Email event tracking logs.

Where Do They Go?
CloudWatch Logs.

Are They Enabled by Default?
No.

How to Enable?
[Link]

Amazon VPC (Virtual Private Cloud)

What Gets Logged?
Network flow logs.

Where Do They Go?
CloudWatch Logs and/or S3.

Are They Enabled by Default?
No.

How to Enable?
[Link]

[The End]

Founder of @CloudSploit , acquired by @AquaSecTeam . Former Infra / Security / Manager @Adobe , @Aviary & @Mozilla intern, @RITtigers grad, @NYC resident

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store