Architecture
Six decisions define Vault's architecture: where data physically sits, how the model gets customized, how PII is handled, how the network is isolated, how compliance teams audit it, and how it fails over without violating the residency guarantee it exists to provide.
1. Residency split
This is the signature feature of Vault v2.0. Documents, the vector index, and the PII lookup table are held at rest physically inside the AWS Hanoi Local Zone. Only a Comprehend-redacted inference payload transits — over PrivateLink, never the public internet — to Amazon Bedrock in the parent region (ap-southeast-1).
| Component ▲ | Location | Crosses region boundary? |
|---|---|---|
| Source documents (S3) | Hanoi Local Zone | Never |
| Vector index | Hanoi Local Zone | Never |
| PII lookup table (DynamoDB, KMS CMK) | Hanoi Local Zone | Never |
| Redacted inference payload | Local Zone → parent region | Yes, via PrivateLink |
| Model weights / inference execution | ap-southeast-1 (Bedrock) | — |
2. Customization: RAG vs. fine-tuning vs. prompt engineering
Vault defaults to RAG for anything that must be auditable, and treats LoRA fine-tuning as strictly a tone/vocabulary tool — never a way to inject facts.
A regulator can inspect exactly which retrieved sources produced an answer. This auditability is why RAG is the default path for anything fact-bearing — rates, policies, product terms.
Reserved strictly for tone and domain vocabulary, never for fact injection. Fine-tuned facts can't be audited the way retrieved sources can, and they go stale silently as policy changes.
The cheapest lever and the first one tried. Reached for before fine-tuning or expanding the retrieval corpus.
LoRA fine-tuning on a Vietnamese-legal-domain corpus reduced hallucination on domain-specific terminology by 40% compared to prompt engineering alone. That gain is scoped to vocabulary and phrasing fidelity — it is not used, and was never evaluated, as a mechanism for injecting facts the model should instead retrieve.
3. PII handling
Original PII never leaves the isolated VPC and never reaches the model. The flow: Comprehend detects PII in the incoming query, tokenizes it before any cross-region call is made, stores the mapping in a KMS-customer-managed-key-encrypted DynamoDB table, and re-hydrates the original values into the response after it returns from Bedrock.
| Step ▲ | Service | What happens |
|---|---|---|
| 1 | Amazon Comprehend | Detects PII entities in the incoming query |
| 2 | App tier | Tokenizes detected entities before constructing the cross-region payload |
| 3 | DynamoDB (KMS CMK) | Stores the token → original-value mapping, encrypted at rest with a customer-managed key |
| 4 | Bedrock (ap-southeast-1) | Receives only the tokenized/redacted payload — never sees raw PII |
| 5 | App tier | Re-hydrates tokens back to original PII in the response, after it returns from the model |
4. Network isolation
The inference path has zero public egress. VPC interface endpoints are used for Bedrock, Comprehend and KMS — traffic to these services never touches the public internet. The bank's data center connects to the VPC via Direct Connect.
Interface endpoints (PrivateLink) terminate Bedrock, Comprehend and KMS calls inside the VPC. Combined with Direct Connect on the bank side, no request in the inference path is ever routed through a public gateway.
5. Governance
CloudTrail logs every inference call. Those logs feed a Glue Data Catalog, queryable through an Athena compliance workgroup — so compliance teams can self-serve questions like "show me every call touching customer X's data in the last 90 days" without filing a ticket to engineering.
A governance model that requires an engineer to run a query for every audit request doesn't scale and creates a bottleneck exactly when compliance needs speed. Routing CloudTrail through Glue/Athena turns audit requests into self-service SQL.
6. Disaster recovery — an open trade-off
This is presented honestly as an unresolved tension, not a solved problem. A standard DR playbook would fail over to the parent region for availability — but that risks violating the very residency requirement this architecture exists to satisfy.
| Tier ▲ | DR posture | Residency impact |
|---|---|---|
| Stateless app tier | Warm standby — ready immediately | None — holds no customer data |
| Data tier (documents, vector index, PII table) | Pilot light — inactive, encrypted, requires activation | Activation only under documented, compliance-approved "degraded mode" exception |