Building a Real-Time Fraud Detection System with ML

Real-time fraud detection is one of the hardest ML problems: you need sub-100ms latency, high recall (miss fraud → real loss), controlled precision (too many false positives → angry customers), and the model has to work against adversaries actively trying to beat it.

Here’s how we built one that processes 2 million transactions per day.

Architecture overview

Transaction → Kafka → Feature Enrichment → ML Scoring → Decision → Action
                 ↕
           Feature Store (Redis + offline Hive)

The core challenge: features need to be computed in real-time (e.g., “how many transactions has this card made in the last 5 minutes?”) while the model is trained on offline batch data. The feature store bridges that gap.

Feature engineering

We built ~140 features across four categories:

1. Velocity features — transaction count and sum in rolling windows (1m, 5m, 1h, 24h)
2. Behavioural features — merchant category deviation from user baseline
3. Network features — graph embeddings of shared devices/IPs
4. Contextual features — time-of-day, geolocation delta from previous transaction

The velocity features are the most valuable and the hardest to compute at inference time. We use Redis with sorted sets for O(log N) range queries.

Model architecture

We use a gradient-boosted ensemble (XGBoost) as the primary model — interpretable, fast to serve, robust to feature distribution shifts. A PyTorch neural net runs as a second-stage for uncertain cases (score between 0.3 and 0.7).

Training happens nightly on a Databricks cluster with 90-day rolling data. The feature store is snapshotted to Hive for offline training.

Results

• Latency: p99 = 84ms end-to-end
• Recall: 94.2% (industry baseline: ~85%)
• False positive rate: 0.4%
• Fraud caught: $2.1M in the first 3 months

The system has been running for 8 months without a major incident. If you’re building something similar, let’s talk.