Professional Project
Logistics Order Service
Overview
A high-throughput logistics order-processing service designed for reliable data changes and resilient event-driven workflows. The service combines Dapper-based data access, Azure Service Bus and Kafka messaging, durable background tasks, and New Relic observability.
My Role
- Developed reliable messaging and background-processing capabilities for a .NET logistics service.
- Built reusable Azure Service Bus publishing and consumer infrastructure based on CloudEvents.
- Implemented a SQL-backed persistent-task framework with durable state, retries, and recovery.
- Migrated the team from a custom Kafka integration to the company-standard messaging library and helped harden its initial rollout.
- Used Dapper with explicit transaction control in reliability-critical data-access paths.
Architecture & Technology
- .NET service for logistics order processing
- Dapper-based data access with explicit transaction boundaries
- Azure Service Bus publishers and consumers using CloudEvents
- Dynamic CloudEvent type resolution and dispatch to typed handlers
- Kafka integration through the company-standard Notino.Messaging library
- SQL-backed persistent tasks with atomic batch claiming, retries, and zombie recovery
- Hangfire background processing
- Structured logs, metrics, and production observability through New Relic
Engineering Challenges
- Process high volumes of logistics orders while preserving consistency across data changes and asynchronous work.
- Handle transient messaging failures safely without creating duplicate side effects.
- Prevent slow or failing downstream consumers from blocking message-topic processing.
- Adopt a new Kafka library while addressing early performance and memory issues.
Decisions & Trade-offs
- Used an explicit event-type registry and typed handlers for predictable, safe message processing, where new event types require an intentional code change.
- Used durable tasks to decouple message acknowledgement from slower downstream processing, allowing failed work to be retried without blocking message consumption. Tasks and related database changes were persisted in the same transaction to prevent lost follow-up work.
- Used atomic batch claiming for persistent tasks to support concurrent high-throughput workers without losing durable task state.
Results & Impact
- Reliable Azure Service Bus event delivery and processing with typed CloudEvent handlers, retries, logging, and metrics.
- Durable background-task execution with retries and recovery from abandoned or zombie work.
- Improved correctness through atomic task-and-data persistence, alongside more resilient downstream processing.
- First-team adoption of the company-standard Kafka library, contributing performance and memory improvements during rollout.
- A reliability-focused, observable logistics order-processing service.