← Projects

AWS Event-Driven Order Platform

What it is

A serverless order flow on API Gateway, Lambda, DynamoDB, SNS and SQS, provisioned end to end with AWS CDK across multiple stacks. Later migrated to Aurora PostgreSQL to demonstrate the same flow on a relational backend.

Event-driven order platform on AWS: architecture overview A client sends POST /order to API Gateway, which invokes the CreateOrderFunction Lambda; it writes the order to a DynamoDB table and publishes an event to an SNS topic. SNS delivers to an SQS queue that a second Lambda polls to write a JSON receipt to S3, and messages that fail three times move to a dead-letter queue. Separately, a local migration script scans DynamoDB and upserts the data into a normalized Aurora PostgreSQL schema. Event-driven order platform on AWS Serverless order intake, asynchronous receipt processing, and an idempotent DynamoDB-to-Aurora PostgreSQL migration Client Any HTTP caller Migration script Local · boto3 ON CONFLICT DO UPDATE AWS account Region · ap-southeast-2 AWS CDK app · order pipeline, Python Aurora · AWS CLI, express config API Gateway Lambda proxy POST /order Lambda Sync handler CreateOrderFunction DynamoDB On-demand table Orders Failure handling Invalid amount: raise error Not deleted, retried After 3 failed receives: DLQ SNS Pub/sub topic OrderCreatedTopic S3 Order receipts receipts/ SQS Dead-letter queue OrderQueueDLQ SQS Message queue OrderQueue Lambda Async worker ProcessOrderFunction Aurora PostgreSQL orderflow-aurora IAM auth · no VPC Normalized schema customers customer_idUUID PK nameTEXT UNIQUE 1 to many orders order_idUUID PK customer_idUUID FK itemTEXT amountNUMERIC(10,2) created_atTIMESTAMPTZ 1 POST /order 2 3 4 publish 5 push 6 7 put R M1 M2 Legend Compute · Lambda Messaging · SNS, SQS Data · DynamoDB, S3, Aurora API · API Gateway Synchronous Asynchronous One-off migration Outside AWS Flow 1 POST /order over HTTPS 2 Invokes CreateOrderFunction 3 PutItem · grant_write_data 4 Publish · grant_publish 5 SNS pushes to OrderQueue 6 Event source mapping · batch_size=5 7 receipts/<orderId>.json · grant_put R After 3 failed receives, to OrderQueueDLQ M1 Paginated Scan · LastEvaluatedKey M2 Upsert customers, then orders
Order request path, from API call to stored receipt A client sends POST /order to API Gateway, which invokes the CreateOrderFunction Lambda. The Lambda writes the order to the DynamoDB Orders table and publishes to an SNS topic, which pushes the message to an SQS queue. A second Lambda polls the queue and writes a JSON receipt to S3. Order request path From API call to stored receipt Client Any HTTP caller AWS account ap-southeast-2 AWS CDK app API Gateway Lambda proxy POST /order Lambda Sync handler CreateOrderFunction 3 PutItem to DynamoDB Orders SNS Pub/sub topic OrderCreatedTopic SQS Message queue OrderQueue Lambda Async worker ProcessOrderFunction S3 Order receipts receipts/<orderId>.json 1 POST /order 2 invoke 4 publish 5 push 6 poll · batch_size=5 7 put object Legend Compute Messaging Data API Synchronous Asynchronous Outside AWS
SNS-to-SQS delivery with success and dead-letter paths CreateOrderFunction publishes to the OrderCreatedTopic SNS topic, which pushes each message to the OrderQueue SQS queue. From the queue, messages take one of two parallel paths: on success, ProcessOrderFunction polls the queue and writes a receipt to S3; after three failed receives, SQS moves the message to the OrderQueueDLQ dead-letter queue. SNS to SQS delivery One queue, two outcomes, side by side AWS account · ap-southeast-2 AWS CDK app Lambda Publisher CreateOrderFunction SNS Pub/sub topic OrderCreatedTopic SQS Message queue OrderQueue maxReceiveCount=3 Success path Lambda ProcessOrder Function S3 receipts/ Failure path SQS DLQ OrderQueueDLQ Dead-letter queue Failed message kept in the DLQ for review 4 publish 5 push 6 poll R 3 fails 7 put Legend Compute Messaging Data Synchronous Asynchronous R retry exhausted, to DLQ
An order goes from API Gateway to a stored receipt in S3. A message that fails three times moves to a dead-letter queue. Aurora sits outside the CDK app and is filled by a re-runnable migration script.

Decisions

Split the system into multiple CDK stacks instead of one.

Rejected: A single CDK stack for the whole system.

This let the data layer be replaced without touching the API or messaging stacks - it's what made the Aurora migration tractable.

Wrote the migration script idempotent and re-runnable.

Rejected: A one-shot script that needed the target cleaned before every run.

This meant a partial run could be repeated safely instead of requiring the target cleaned first.

Used contract tests comparing API responses either side of the cutover.

Rejected: Verifying only that row counts matched between DynamoDB and Postgres.

Row counts only prove the data arrived, not that every field still means what callers expect - comparing full API responses on both sides catches representation differences a count can't see.

What broke

The contract tests failed on representation, not data. Timestamps stored as ISO strings in DynamoDB came back from Postgres as epoch numbers. The rows were correct and the counts reconciled, so a data-level check would have passed - but every consumer parsing those fields would have broken. Several similar type mismatches surfaced the same way. All caught before staging.

Stack

AWS CDK · API Gateway · Lambda · DynamoDB · Aurora PostgreSQL · SNS · SQS · S3

github.com/nalinka1/aws-order-flow