Shamratha G

Problem Statement

• Payment gateways ingest transaction data whose quality issues can silently corrupt downstream risk and settlement decisions.
• In a regulated financial context, an ML model must never be the thing that rejects a transaction — hard rules must enforce, ML can only inform.
• GenAI explanations are useful for humans but unsafe by default — they need deterministic guardrails to stay traceable and auditable.
• Every decision needs an audit trail that attributes responsibility to a specific pipeline layer.

System Overview

• A 15-layer pipeline scores payment-gateway transaction data quality on the philosophy 'Rules Enforce, ML Informs, Humans Decide' — hard business rules can reject, ML and GenAI can only flag.
• I built the core data quality pipeline: deterministic structural/compliance/semantic rule checks combined with Isolation Forest anomaly detection, producing per-dimension quality scores.
• I wrapped the Gemini LLM layer (human-readable issue summaries) in deterministic guardrails so GenAI output stays safe, traceable, and auditable in a regulated financial context.
• I architected the FastAPI service handling incoming transaction payloads, tuning the event loop for tight latency budgets, with async background task processing keeping the main thread responsive under concurrent load.
• I designed the real-time audit logging schema with per-layer liability attribution, enabling root-cause traceability for every decision.
• Later layers synthesize signals through stability, conflict, and confidence checks into a final decision gate, verified by an 88+ test pytest suite.

What I Built

• Built the transaction data quality pipeline combining deterministic rule checks with Isolation Forest anomaly detection, producing per-dimension quality scores and structured audit trails for financial compliance.
• Wrapped the Gemini LLM layer with deterministic guardrails so GenAI outputs stay safe, traceable, and auditable in a regulated context.
• Architected the FastAPI service handling incoming transaction payloads, optimizing the event loop to hit tight latency budgets.
• Designed the real-time audit logging schema with per-layer liability attribution for root-cause traceability.
• Built async background task processing so the main thread stayed responsive under heavy concurrent load.

Key Decisions & Tradeoffs

• 'Rules Enforce, ML Informs, Humans Decide': hard business rules (layers 4.1–4.3) can reject transactions; ML and GenAI layers (4.4–4.5) can only flag.
• Hybrid intelligence — deterministic structural/compliance/semantic checks alongside Isolation Forest anomaly detection across 35+ extracted feature dimensions.
• Gemini used strictly for human-readable summarization behind deterministic guardrails, never for enforcement.
• Per-layer responsibility and liability attribution baked into the audit schema so every decision is root-cause traceable.

Why It Matters

What I'd Improve Next

• Replace synthetic generated transactions with anonymized real payment data to validate rule thresholds and anomaly-detection sensitivity.
• Benchmark and publish end-to-end latency percentiles under production-scale concurrent load.
• Add drift monitoring and scheduled retraining for the Isolation Forest as transaction patterns evolve.
• Expose the engine as a streaming service (e.g. Kafka consumer) for continuous scoring rather than batch runs.
• Add a human-review console over the decision gate so flagged transactions complete the 'Humans Decide' loop in-product.