The Challenge
You need to decouple services so they can evolve independently and scale based on their own workload patterns. In a synchronous architecture, one slow service blocks the entire request chain. Event-driven architecture lets services communicate through events, so producers and consumers operate independently and failures are isolated.
What You'll Build
- → Central event bus for service communication
- → Producer and consumer services
- → Dead-letter queues for failed events
- → Event replay for debugging
- → Monitoring for event flow and latency
Try This Prompt in Pulumi Neo
Run this prompt in Neo to deploy your infrastructure, or edit it to customize.
Best For
Architecture Overview
This architecture uses a central event bus as the communication backbone between services. When something significant happens in one service (an order is placed, a user signs up, a payment is processed), that service publishes a domain event to the bus. Other services subscribe to the events they care about and process them asynchronously. The producer does not know or care which consumers exist, and consumers can be added or removed without changing the producer.
This decoupling provides several advantages. Services scale independently based on their own workload. A spike in order events means the order processing consumer scales up, but the notification consumer continues at its own pace. Failures are isolated: if the notification service is down, order processing continues unaffected, and the notification events queue up until the service recovers.
Dead-letter queues capture events that fail processing after a configured number of retries. Rather than losing failed events or blocking the queue, they are moved to a separate queue where they can be inspected, debugged, and reprocessed. This ensures no events are silently dropped and gives the team visibility into processing failures.
Event Bus
The event bus is the central routing layer. Producers publish events with a type and payload, and the bus delivers them to subscribers that have expressed interest in that event type. The bus handles routing, filtering, and delivery guarantees. Cloud-native options include EventBridge on AWS, Event Grid on Azure, and Pub/Sub on GCP.
Producers and Consumers
Producers are services that emit events when state changes occur. They publish events to the bus without knowledge of who consumes them. Consumers subscribe to specific event types and process them in their own time. A single event can have multiple consumers: an order-placed event might trigger inventory updates, payment processing, and customer notifications, each handled by a separate service.
Failure Handling
Failed events are retried a configurable number of times before being moved to a dead-letter queue. This prevents a single bad event from blocking the entire consumer. The dead-letter queue stores the failed event along with error details, making it straightforward to diagnose the failure. After fixing the issue, you can replay events from the dead-letter queue back to the original consumer.
Common Customizations
- Add workflow orchestration: Extend the prompt to include a state machine service for coordinating multi-step processes where events must happen in a specific order.
- Add event schemas: Request an event schema registry that validates event payloads before they are published, catching malformed events at the producer rather than the consumer.
- Add cross-account events: Ask for event routing rules that forward specific event types to event buses in other AWS accounts or other cloud environments.
- Add analytics on events: Request that events are also delivered to a data lake or analytics service so business analysts can query event patterns without affecting the operational pipeline.