← Back

Fraud Detection V1.0

·Bryan Lai
TechnicalMLFraud

Fraud Detection V1.0

Optimize for cost, not accuracy.

Key Terms

  • ATO: Account takeover. Password or session stolen. Signal: sudden device or IP change.
  • Synthetic identity: Fake person, often built from real identity pieces. The account ages, gets credit, then disappears with cash.
  • First-party fraud: The customer is the thief. They claim goods never arrived or the transaction was unauthorized.
  • Chargeback: The bank reverses the payment and charges a fee. Too many chargebacks can get a merchant banned.
  • Bust out: Build trust, max the credit line, then vanish.
  • Card not present: Online or phone transaction. No physical card makes fraud easier.
  • Friendly fraud: The customer disputes a real purchase.
  • Fraud ring: Many accounts coordinate. They may share a device, address, WiFi, card, or transfer path.

The Mental Model

Accuracy Is Useless

If your model predicts "good" for 99% of transactions, it has 99% accuracy and 0% value.

Precision vs Recall

  • Precision: Of the users you flagged, how many were actually bad? Low precision annoys good users.
  • Recall: Of all bad actors, how many did you catch? Low recall means you are getting robbed.

Move the threshold until the cost of annoying good customers is lower than the cost of letting thieves through.

Build It

Velocity: A card used five times in 10 minutes is a spike. Count recent events per user, card, IP, and device.

Graph signals: Fraudsters create many accounts but reuse devices, WiFi, addresses, cards, and recipients. If Account A is fraud, Account B that shared its device is suspicious too.

LightGBM: For rare fraud labels, set is_unbalanced=True or tune class weights.

Architecture

  1. Kafka/Kinesis: Transactions stream in.
  2. Flink: Calculates velocity and graph signals in real time.
  3. Redis: Store computed features (fastest retrieval).
  4. SageMaker: Scores each transaction from 0 to 100:
    • 90+: approve.
    • 50-89: require selfie or SMS check.
    • <50: deny.

Reality Check

Model drift: Fraudsters change tactics. Retrain from recent chargebacks and confirmed fraud.

Explainability beats complexity: "If country = X and amount > Y, block" can beat a black-box model when ops needs a clear reason.

Make Sure To

  • Engineer features around behavior, not profile fields.
  • Feed confirmed fraud back quickly.
  • Watch false positives like a product metric.
  • Retrain before the fraud pattern goes stale.

Bottom line: a fraud model is only useful if it catches expensive bad behavior without drowning ops and good users.

References