Starting point: Define the first use case

The first question is not, “How do we build a large model?” It is, “Where can reusable transaction intelligence create measurable value first?” Pick something specific, measurable and close to production. Fraud detection is often a strong starting point because the value case can be quantified through fraud loss, recall, precision, false positives, manual review volumes and decision latency.

Good first questions look like: “Can TFM embeddings improve fraud detection for account-to-account payments and reduce false declines without increasing fraud?” This step defines the business objective, baseline performance, target metric, decision workflow, available data and production constraints.

Step 1: Data preparation

  • Assemble the transaction and event data foundation

A TFM depends on rich, well-governed transaction data. The minimum dataset includes transaction amount, timestamp, merchant or counterparty, transaction type, channel, currency, customer or account identifier and outcome labels. Advanced use cases add device, location, IP, customer profile, merchant metadata, payment rail, authorization response, fraud outcomes, chargebacks, disputes, sanctions outcomes, servicing events and app behavior.

The aim is not to collect every field on day one. The aim is to create a high-quality, sequenced view of behavior over time, with strong data engineering underneath: schema mapping, lineage, quality checks, consent and privacy controls, entity resolution, deduplication and temporal ordering. In payments, time matters — the model must be trained and evaluated in a way that reflects how decisions would have been made in the real world (point-in-time correctness).

  • Normalize, sequence and prepare the data

Raw transaction data is not ready for transformer training. It must be converted into coherent sequences. Group events by the right entity — customer, account, card, merchant, device, counterparty — and order by time. Define windows that match the use case: fraud needs recent high-resolution sequences; credit needs longer-term income and spend patterns; operations needs sequences of payment failures, retries, repairs and settlement events.

This is where data science and payments domain expertise have to work together. The model should learn from patterns that make sense in the payments context, not from arbitrary rows in a table.

Step 2: Tokenize transaction data

Tokenization is the most distinctive technical decision in a TFM build, and a small set of choices has a large effect on model quality:

  • Amount fields: Bucket using log-spaced or decile thresholds; never feed raw values.

  • Time-delta tokens: Choose a granularity (minutes for fraud, days for credit) that matches the downstream task.

  • High-cardinality fields (merchant ID, counterparty, device): Use top-K + UNK, hashing or learned embedding tables.

  • Categorical fields (channel, payment rail, status): Implement direct token mapping.

  • Outcome tokens (approved, declined, disputed, fraudulent): Inject explicit tokens so the backbone learns the supervisory signal in pre-training.

A good tokenizer balances domain meaning, privacy, scale and model efficiency. The NVIDIA Transaction Foundation Model developer example provides a custom NVIDIA CUDA-X -powered tokenizer for this stage, which is useful because tokenization and preprocessing routinely become the bottleneck at large transaction-data scale.

Step 3: Train the transformer backbone

Once sequences are tokenized, the foundation model can be trained. The NVIDIA reference uses NeMo AutoModel to train a decoder-only transformer with causal language modeling — the model learns what tends to happen next in a transaction sequence. Other approaches use masked event modeling, where the model reconstructs missing events or attributes. Model size is a function of data scale and task complexity. 

The goal of pre-training is to produce a model that learns useful representations of financial behavior: recurring patterns, abnormal shifts, behavioral similarity, merchant relationships, temporal context and risk signals.

Step 4: Generate reusable embeddings

After training, the model is used to generate embeddings, which are compact numerical representations of a transaction sequence, customer, account or merchant. A few decisions shape how reusable they are:

  • Pooling strategy (last hidden state, mean pool, CLS-token analog): Controls what the embedding captures.

  • Granularity: Per-transaction, per-customer rollup over a window, or per-merchant aggregate.

  • Frozen vs fine-tuned: Frozen embeddings are simpler to govern; fine-tuned embeddings give more lift per downstream task.

  • Dimensionality: Typically 128–768; lower dimensions cut serving cost, higher dimensions preserve more signal.

The embeddings are stored in a vector database and served through an embedding service. From this point on, fraud, credit, operations and personalization teams all consume the same intelligence layer. This is the architectural pivot that turns isolated AI pilots into a shared bank-wide asset.

Step 5: Evaluate against a baseline

The right evaluation is a three-way comparison: existing raw features only, TFM embeddings only and raw features + TFM embeddings. The third typically wins, and the lift is what unlocks budget for the next use case. The NVIDIA Transaction Foundation Model developer example demonstrates this pattern using XGBoost on top of raw features, TFM embeddings and combined features.

The final mile: Integrate into decisioning and workflows

A TFM creates value only when its outputs are embedded into business processes. In fraud, this means feeding embeddings into a fraud-scoring model, case-management system or real-time decision engine. In authorization, it supports approval optimization and false-decline reduction. In operations, it supports exception routing, payment repair or reconciliation. In credit, it augments affordability models or early-warning indicators.

The integration pattern depends on latency. Card authorization and real-time payments need sub-100ms end-to-end inference, solved on the NVIDIA platform with NIM, Dynamo-Triton and TensorRT-LLM. Credit decisioning and customer engagement can run near-real-time or batch; operations can run offline. The architecture must also include fallback logic, human-in-the-loop review for high-impact decisions, A/B or shadow deployment, and continuous drift monitoring on both embeddings and predictions.

Under the hood: A TFM reference architecture

A practical TFM reference architecture can be structured across nine layers. Each layer is modular. The TFM should not force every downstream team onto the same model or workflow; it should provide a shared intelligence layer that can be consumed through controlled interfaces. The color zoning below maps the layers to five functional groups: data (green), tokenization (yellow), model training and serving (blue), decisioning (purple) and governance (black).