Export Security Incidents to Amazon Security Lake

Tasks

Write security incidents to your Amazon Security Lake as OCSF Detection Findings in Parquet, partitioned the way Security Lake expects. AIControls assumes the custom source's provider role; no AWS access keys are stored.

Before you start

You need the admin role in AIControls, and permission in AWS to create a Security Lake custom source and its IAM role.
AIControls must run with an AWS identity of its own: an IAM role for a service account (IRSA), an EC2 or ECS task role, or the standard AWS environment variables. That identity is the account AWS calls the principal, and it needs permission to assume the provider role (step 3).
Security Lake custom sources accept OCSF 1.3 and earlier. AIControls therefore writes OCSF 1.3.0 to Security Lake, not the 1.9.0 used by the other formats. Fields that 1.3.0 cannot hold are kept under unmapped.
Know which data leaves the product. See what is never exported. The same Redact actors and Include finding summaries options apply.

Create the custom source in Security Lake

Create the IAM role that lets the Security Lake crawler read the source data, as described in Collecting data from custom sources. You need its ARN for the next step.
Add the custom source with the event class DETECTION_FINDING. Choose a source name of at most 20 characters. Set providerIdentity to the AWS account principal that AIControls runs as and an external ID you generate (a long random string).
Note the provider.roleArn and provider.location that Security Lake returns. The role is named AmazonSecurityLake-Provider-<source>-<region>. The location is s3://<bucket>/ext/<source>/; the bucket is the name between s3:// and /ext.

aws securitylake create-custom-log-source \
  --source-name aicontrols \
  --event-classes '["DETECTION_FINDING"]' \
  --configuration crawlerConfiguration={roleArn=arn:aws:iam::111122223333:role/service-role/CrawlerRole},providerIdentity={externalId=YOUR-EXTERNAL-ID,principal=444455556666} \
  --region us-east-1

Security Lake creates the provider role, attaches write access to the source prefix (and KMS permissions when the lake uses a customer managed key), and creates a Glue table and crawler.

Allow AIControls to assume the role

Give the AWS identity that AIControls runs as permission to assume the provider role. The role's trust policy already names that account and the external ID; add the permission to the identity itself:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": "sts:AssumeRole",
    "Resource": "arn:aws:iam::111122223333:role/AmazonSecurityLake-Provider-aicontrols-us-east-1"
  }]
}

AIControls calls sts:AssumeRole with the role ARN and the external ID, then writes objects with s3:PutObject under the source prefix using the assumed credentials. Static access keys are not supported.

Configure the destination

Go to Integrations → Security export and select Add destination.
Enter a unique Name and set Format to Amazon Security Lake (Parquet).
Enter the Custom source name and Region you used above. Leave Account ID partition empty to use external, the value AWS recommends for records that do not belong to an AWS account, or enter a 12-digit account ID.
Enter the Provider role ARN, the External ID and the Security Lake bucket. All three are hidden after you save; leave the masked value to keep the saved one.
Set the Severity filter, Event types, Automatic export, Redact actors and Include finding summaries as for any other destination.
Turn the integration on, select Send Test, then Save. The test assumes the role and writes one empty Parquet object that has the full schema, so permission and prefix mistakes show up without adding a synthetic finding to your lake.

The OCSF version can only be 1.3.0. Any other value is refused when you save, with the reason.

How incidents are written

Trigger When the object is written

Automatic export In batches. At most one flush per destination every 5 minutes, up to 100 incidents each. A flush writes one object per UTC day it covers, so a batch that spans midnight UTC writes two objects. An incident is claimed once per destination across all replicas, and a failed batch is retried after the 5 minutes. Delivery is at least once: if a crash happens after an object is written but before it is recorded, the incident is written again in a later object. Automatic events carry a stable metadata.uid, so queries can deduplicate on it.

Export incident (manual) At once, as one small object, so the result in the drawer is the real result. Many manual exports make many small objects. While an automatic flush is writing to the same destination, a manual export is refused for a few seconds as in progress.

Trigger	When the object is written
Automatic export	In batches. At most one flush per destination every 5 minutes, up to 100 incidents each. A flush writes one object per UTC day it covers, so a batch that spans midnight UTC writes two objects. An incident is claimed once per destination across all replicas, and a failed batch is retried after the 5 minutes. Delivery is at least once: if a crash happens after an object is written but before it is recorded, the incident is written again in a later object. Automatic events carry a stable `metadata.uid`, so queries can deduplicate on it.
Export incident (manual)	At once, as one small object, so the result in the drawer is the real result. Many manual exports make many small objects. While an automatic flush is writing to the same destination, a manual export is refused for a few seconds as in progress.

Objects are zstd-compressed Parquet files, sorted by time, under:

ext/<source>/region=<region>/accountId=<accountId>/eventDay=<YYYYMMDD>/aicontrols-<UTC time>-<id>.parquet

eventDay is the UTC date of the incident's last finding time. Run the Glue crawler after the first export so Security Lake lists the table; run it again if a later release adds columns.

Check what was sent

Every incident in an attempt is recorded in Security → Activity → Configuration as security_incident.exported with the outcome, the trigger, the object key (also for a failed write, because a timed-out write may have left an object), the batch size, the attempt count and, for a failure, a short reason: credentials_unavailable, assume_role_failed, access_denied, bucket_not_found, throttled, timeout, network_error or http_<status>. The role ARN, external ID and bucket are never recorded; the destination appears as security-lake:<region>. A write is retried up to 3 times for network errors, throttling and 5xx responses, and not for other 4xx responses. The metric aicontrols_security_export_total counts each incident.

Troubleshooting

Reason	Check
`credentials_unavailable`	AIControls has no AWS credentials of its own: check the IRSA, task role or environment variables on the pod or host.
`assume_role_failed`	The external ID matches the one in `providerIdentity`; the AIControls identity is the registered principal and is allowed `sts:AssumeRole` on the role.
`access_denied`	The role may write under `ext/<source>/` in the bucket, and the KMS key policy allows it when the lake is encrypted.
`bucket_not_found`	The bucket name and the region of the destination.

See Security Incident OCSF Mapping for how a 1.9.0 event maps to the 1.3.0 Parquet row.

Before you start​

Create the custom source in Security Lake​

Allow AIControls to assume the role​

Configure the destination​

How incidents are written​

Check what was sent​

Troubleshooting​