Back to Insights
November 18, 2026
5 min read
Enterprise AI

2% of What? Architecting Real-Time AI Fraud Detection Beyond the Model

How an Enterprise AI Architect turns a simple fraud detection requirement into a reliable, governed and continuously learning AI system.

177 views13 likes

Imagine a business leader walking into a discovery meeting and asking a question that sounds simple enough.

Can AI detect fraudulent transactions in real time?

There is an easy way to start this conversation. We could talk about the latest foundation model, a sophisticated fraud platform or an agent that investigates transactions. All of those might sound impressive. But they would be answers to a technology question that the business has not yet fully asked.

I would rather begin with the outcome. What kind of fraud are we trying to catch? Which legitimate customers are we prepared to inconvenience? How quickly must the decision be made? And how will we learn whether the decision was correct?

This case study follows that thinking from the initial business request to the architecture of a system that can actually operate in production.

The scenario is a fictional composite informed by common financial services patterns. The numbers and targets are illustrative. They are not claims about a particular client or actual project performance.


A reasonable request with a difficult condition

Consider a firm handling approximately 800,000 transactions each day across retail and corporate accounts. Its existing fraud detection relies heavily on rules built around known patterns. Rules are useful. They can be tested, understood and enforced consistently. But fraud changes. A new device, an unusual merchant, rapid successive transactions or activity across related accounts may not match any one historical rule.

Leadership asks for AI to identify emerging fraud without increasing the inconvenience to genuine customers. Then somebody adds what sounds like a clear target.

Keep false positives below 2 percent.

My next question would be a short one.

2 percent of what?

The answer could change the entire solution. A business stakeholder may mean the proportion of genuine transactions being wrongly declined. A fraud operations manager may mean the proportion of alerts that turn out to be legitimate. Those are different denominators, different experiences and different workloads.

Suppose two hundred transactions are flagged and only forty are genuine fraud. The alert precision would be 20 percent and the proportion of false alerts would be 80 percent. That does not mean 80 percent of all legitimate transactions were incorrectly declined. The flagged population is only a small subset of total activity.

Equally, a 2 percent share of 800,000 daily transactions would represent 16,000 events. That simple calculation shows why a loosely defined percentage can become an expensive misunderstanding. The example is not a proposed operating target. It is a reminder to agree on the denominator before discussing model performance.

This is where architecture begins for me. The first job is not to choose the intelligence. It is to make the outcome measurable.



The business capabilities have to work together

The business architecture view makes an important point. Fraud prevention is not a single prediction. It is a connected value stream. A transaction is initiated, risk is evaluated, an action is authorized or challenged, an exception may be investigated, and the outcome eventually informs future decisions.

I would bring the payments team, fraud operations, risk, compliance and customer experience owners into the same discussion. They need to agree on what counts as a prevented fraud, what counts as a wrongful decline, what is merely an alert, and which decisions can be automated.

I would also distinguish fraud recall, legitimate transaction decline rate, alert precision and analyst workload. Better detection is not valuable if the improvement overwhelms investigations or disrupts too many genuine customers. The business case should reflect avoided fraud losses, customer friction, operating expense and the cost of running the models and controls.

Even before a solution diagram exists, those definitions start shaping the design.

One transaction, several kinds of intelligence

Take a fictional customer who normally makes modest purchases from familiar devices and merchants. Within minutes, three transactions arrive from a new device. Each is small enough to look ordinary in isolation. Together, they may tell a different story.

Now ask what we need the technology to do. We need a rapid signal about the transaction itself. We need to recognize unusual behavior that does not match a known fraud label. We need supporting history. We need a governed decision about approving, challenging or declining the payment. We may need a clear explanation for an analyst. And eventually, we need to know whether the assessment was right.

That is not one model call. It is a sequence of related capabilities operating under different constraints.



The six capabilities behind the fraud detection system

1. Recognize learned patterns and unfamiliar anomalies

The immediate task is transaction classification. For each payment, we need a risk signal derived from information such as transaction amount, merchant category, device, location, velocity and account behavior.

For this part, I would benchmark classical machine learning before proposing a large language model. Gradient boosted trees and suitable neural models are often strong candidates for structured transactional features, particularly when inference must be fast and predictable. The right choice depends on the available labels and actual performance testing, not on a blanket assumption that any one technique always wins.

There is a second requirement that classification alone may not cover. A supervised model learns from historical examples and can generalize, but it may be less effective when new fraud behavior differs substantially from the training data. An unsupervised anomaly detector, such as an Isolation Forest or a suitable autoencoder, can provide a complementary view of unusual activity without requiring every anomaly to have been previously labeled as fraud.

Combining those signals is more useful than expecting one model to recognize every pattern. They must still be calibrated, tested and monitored against real business outcomes.

2. Retrieve the evidence that matters

A score tells an analyst that something is unusual. It rarely tells the complete story.

Was the device used earlier today? Are there related transactions in the same session? Is this merchant consistent with past customer activity? Have similar cases appeared elsewhere?

The system needs authoritative transaction history, account context and relevant behavioral patterns. Much of this can come from structured lookups, event data and feature stores. Similarity retrieval may also help where a use case warrants it. I would not start by placing an entire transaction history into an LLM context window.

This is Context Engineering in practice. Retrieve the facts that support the decision, make their freshness and provenance clear, and avoid drowning the important signals in irrelevant history. A generated summary should never take precedence over authoritative payment facts.

The data architecture behind the decision

The data view separates two very different responsibilities. The real time decision needs trusted, low latency features. The learning process needs historical evidence and carefully matured fraud labels.

For the immediate path, data from payments, customer profiles, devices, sessions and merchant categories should be validated and converted into features that the scoring service can retrieve quickly. Feature engineering and inference must fit within an agreed latency budget.

For the longer path, every decision needs enough linked evidence to explain what happened later. Chargebacks, customer disputes and analyst investigations may arrive well after authorization. Those eventual outcomes become training labels only when the evidence is sufficiently reliable.

This is why I would make provenance, data quality, retention, privacy and feature consistency architectural concerns from day one. It is entirely possible to build a high performing model with poor production data controls and still produce an unreliable fraud system.



Turn risk signals into understandable evidence

A useful fraud system may combine a supervised fraud score, an anomaly score and contextual indicators into a governed risk assessment. Those numerical outputs must come from evaluated scoring methods and documented business logic. An LLM should not be asked to invent a probability of fraud simply because it can write a persuasive explanation.

Where the LLM adds value is in helping the analyst understand the evidence. For example, it could summarize that a transaction originated from a new device, followed an unusual sequence and differed from the account's normal merchant behavior. The explanation should be concise, grounded in retrieved evidence and tied to specific source references.

If the system cannot support a statement, the explanation should not confidently invent one. In my view, the LLM belongs in the explanation and investigation experience, not automatically in the payment authorization decision.

4. Keep important business decisions deterministic

Fraud operations need clear actions. Approve the transaction, challenge the customer, decline under an approved policy or route the case for investigation.

Those actions are not simply whatever a language model recommends. They depend on business thresholds, risk appetite, transaction value, customer impact, available evidence and policy. A deterministic policy engine can enforce the agreed decision boundaries while the models supply risk signals.

Human review also needs realistic design. If the solution generates 5,000 reviews a day but the team can reasonably process only 2,000, we have not solved the problem. We have moved the bottleneck. The architect has to connect review capacity, escalation service levels and the thresholds that generate work.

Not every uncertain transaction can wait for a human. In the authorization flow, the system may need to choose a preapproved challenge or fallback action and send the case for later investigation.

5. Make every consequential action traceable

Imagine a genuine customer disputes a declined payment three weeks later. Can we explain what the system saw at the moment of decision? Which model version was running? Which rule caused the decline? What did the analyst conclude after review?

An operational log alone may not answer those questions.

I would design a decision record that ties together the transaction reference, relevant features, score, model version, policy result, response and subsequent review. Logs must be protected, retained only as required and accessible to authorized teams. Any AI generated explanation should also be traceable to the evidence used.

For a regulated organization, auditability is not a final implementation task. It is part of how we define a trustworthy decision.

6. Learn from confirmed outcomes, not guesses

This is one of the less visible challenges in fraud detection. The system must respond almost immediately, but the truth about a transaction may arrive much later. A customer dispute, a chargeback or an analyst investigation could change what we know about a payment weeks or months after it occurred.

That delay matters. Retraining on premature or inaccurate labels can weaken a model rather than improve it. And leaving a model unchanged indefinitely can allow performance to drift as customer behavior and fraud patterns evolve.

A practical feedback loop captures outcomes, applies label maturity and data quality checks, monitors performance and tests candidate models before replacing the current model. A champion and challenger process helps keep deployment decisions controlled and observable.

This is what turns a fraud model into a fraud capability that can continue learning responsibly.

The system runs on two clocks

One detail makes the architecture particularly interesting. The fraud system has to work at two different speeds.

The first is the fast path. It supports the time sensitive authorization decision. A payment event arrives, essential features are obtained, supervised and anomaly signals are calculated, and a policy engine determines the allowed action. The precise time budget must be confirmed with the payment platform and scheme requirements. A budget around 300 milliseconds is only an illustrative scoping assumption, not a universal industry limit.

The second is the slow path. It supports evidence retrieval, LLM assisted summaries, investigations, delayed outcomes, feedback and model evaluation. Those tasks can be asynchronous because they should not hold up a time critical authorization.

That separation avoids an architectural trap. A compelling LLM narrative is not worth increasing payment latency or allowing an unverified text generation step to control a financial transaction. The model can help investigators understand why a case was flagged without becoming the synchronous authorization engine.





Where application and technology architecture meet

The application and technology view shows what must actually run behind those two clocks. On the synchronous side we need a payment API, fraud scoring service, low latency feature access, model serving and a deterministic risk policy engine. The authorization response must be reliable even when a nonessential dependency is unavailable.

On the asynchronous side, an event bus can capture decision events. Investigation applications can retrieve supporting context, request a controlled LLM explanation and present cases to human analysts. Confirmed outcomes then feed the review and model lifecycle process.

The platform underneath needs secure integration, identity and access controls, encryption, resilient serving, monitoring and audit evidence. Model versioning, deployment controls and operational observability belong here too.

The objective is not to add every fashionable platform component. It is to introduce the capabilities that this business decision genuinely requires. Existing payment and risk platforms may already provide significant parts of the architecture.

What I would clarify before committing to a design

By now the conversation is no longer about selecting a model. I would want several uncertainties resolved before estimating or promising outcomes.

First, how much transaction history is available and how trustworthy are the fraud labels? Next, what is the real authorization latency budget, including peak load and dependency timeouts? What are the existing decline and detection baselines? What risk thresholds does the business actually approve? How much work can analysts handle? What data and decisions are subject to specific legal, privacy and regulatory obligations? And how long do confirmed labels normally take to arrive?

These questions may feel less exciting than a model demonstration. But they save the team from building a solution that is fast in a demo and impractical at scale.

I would write the agreed assumptions and unresolved dependencies into the scope. That is how we make the next architecture discussion honest.

A delivery path that protects the business

I would not switch the entire transaction flow to a new AI driven decision process overnight. A safer first release can begin by measuring the current rules based system and then testing new detection signals on historical data.

From there, run the new models in shadow mode alongside existing controls. Compare fraud recall, legitimate declines, alert precision, throughput and latency without changing customer outcomes. Use those findings to adjust thresholds and identify which actions can be introduced under controlled rollout.

Only then would I expand automated decisions, introduce analyst explanations where they improve review quality, and connect the confirmed outcomes into retraining. Every phase should have clear acceptance criteria and a rollback path.

The business case should include more than prevented fraud. It must account for wrongful declines, operational review effort, platform and inference cost, support burden and the governance needed to operate the solution well.

What the example really teaches

We started with a perfectly reasonable question. Can AI detect fraudulent transactions in real time?

We ended up with a system that needs multiple kinds of intelligence, authoritative data, business rules, separate processing paths, human investigation, traceability and a responsible learning lifecycle.

The most valuable question in the early discussion was not which model to use. It was 2 percent of what? That question exposed the business outcomes, the operational risks and the measures that would determine whether the entire solution was successful.

When I think about the role of an Enterprise AI Architect, four responsibilities come together here. Feasibility establishes whether the problem, data and operating conditions justify the use of AI. End to end design connects the capabilities and failure paths. Governance makes the decisions controlled and explainable. Business communication connects everything back to customer experience, fraud losses, analyst capacity and total cost.

The scoping exercise did not make the problem more difficult. It made the problem visible before the architecture was built.

That is the difference between adding an intelligent model to a transaction flow and architecting a dependable business decision system.


Related Topics

Fraud DetectionAI ArchitectClassical MLAnomaly DetectionLLMRisk ScoringContext EngineeringMLOpsAI GovernanceReal-Time AIDecision Intelligence