---
# SAP HANA Databricks migration
**URL:** https://www.sigmoid.com/sap-hana-databricks-migration/
Date: 2026-11-12
Author: Joha Momin
Post Type: page
Summary: SAP HANA to Databricks migration playbook Sort the estate, rebuild what survives, and switch only when numbers match. Most SAP HANA estates...Read More...
---

# SAP HANA
to Databricks
migration playbook
Sort the estate, rebuild what survives, and switch only when numbers match.
Most SAP HANA estates carry views nobody reads, copies of copies, and logic that won’t run well anywhere else. This is Sigmoid’s playbook for moving only what earns its place on Databricks, in ten gated steps.
Sort the models
Start again
Figures show a representative 500-model estate. Your own split comes out of the assessment.
**Your HANA estate today**
Carried to Databricks **500** of 500
Migrate as-is0
Redesign0
Merge0
Retire0
500 calculation views, all running on HANA. Sort them to see what happens to each.
[Context](#overview)[Method](#steps)[Decide](#tests)[Design](#journey)[Build](#datamove)[Prove](#parity)[Plan](#timeline)[Control](#control)[Run](#operate)[Glossary](#glossary)Pause animations
Context
## Harder than copying code
The platform is going away, and every report on it must keep working.
1
### The logic is hard to read
Calculation views hide currency conversion, hierarchies and security rules, and most are thinly documented. The first job is finding out what each one does.
2
### Nobody should migrate everything
Some models are unused, duplicated or obsolete. Retiring or merging them works only if each call is checked with its business owner.
3
### The business can’t go dark
HANA stays live beside Databricks until each package matches and its owners sign. Period-end reporting must never break.
#### Before and after, in one picture
TODAY
SAP sourcesystems
Legacy HANA500 models, 15 packagesretired by SAP
Qlikreports and apps
Business usersfinance, sales, safety
AFTER (the target)
SAP sourcesystems
Bronzeraw copybuilt by the customer
Silvercleaned and keyedbuilt by Sigmoid
Goldbusiness entitiesbuilt by Sigmoid
Qlik, via compatibilityviews with the old names,so reports need no edits
Parity gate: Databricks results must match legacy before a package is signed offLegacy stays live in parallel until then, so a wrong call can be undone
The platform and ingestion (everything up to Bronze) already exist or belong to the customer. Sigmoid designs and builds Silver, Gold and the compatibility layer, and proves the match.
### The approach in brief
Understand first, decide second, build third, prove last. Pick a stage.
## Is this method a fit for you?
Check your situation before reusing the playbook.
## Eight rules the method doesn’t bend
When a project is under pressure, these are checked first.
Method
## Ten steps, and none of them is skipped
Every step ends at a gate with evidence and a named signer.
UnderstandDesignProveSwitch
###
Previous step
Next step
Exit gate
Gate checklist
## Who does what, step by step
Roles stay the same on every engagement. Pick a role to follow it.
A accountable, signs
R responsible, does
C consulted
I informed
Where a step shows two A entries, the domain owner is accountable for acceptance and the delivery manager for execution.
## Why the method works this way
The choices teams question most, and when to revisit them.
Decide
## Three questions decide the fate of every view
Checked in a fixed order, cheapest answer first. Try a sample view.
Monthly sales by region
Promo report 2019
Customer margin (copy)
Stock by plant, hard-coded
Year-end accruals
1
Is it needed?
Someone reads it, or it’s needed at period-end or by regulation.
YesNo
2
Is it a duplicate?
Another view gives the same result at the same grain.
YesNo
3
Does its logic need to change?
Hard-coded values, slow joins, or bad data hidden inside.
YesNo
The business owner can overturn any answer, and every decision goes in a log with its reason.
Answer the questions to see the treatment.
Pending
Each view ends up in exactly one of four treatments: migrate as-is, merge, redesign or retire.
### The eight things we look at
Six classification factors, plus ownership and currency. Thresholds are tuned in the pilot.
### Rules that override the three questions
Some views are protected, whatever the answers say.
### What a wrong call costs
Every treatment can be wrong. HANA stays live, so each one is reversible until its package retires.
## Treatment says what. Complexity says how hard.
Every model also gets a complexity score. Tick what a view contains.
Complexity score
0 points
**Low**
Classes: 0 to 2 Low, 3 to 5 Medium, 6 to 8 High, 9 or more Very High. Each class gets an effort per model, measured in the pilot.
Design
## How one package moves across
Every package takes the same gated path. The pilot builds the templates.
Inventoryviews, owners, usage
Treatment1 of 4 paths
BuildSilver + Gold
Parallel runLegacy + Databricks
Parity gatevs legacy output
UAT sign-offCustomer business owner
Cutoverrepoint Qlik
Hypercarethen legacy objects retired
SCOPE ✓
PATH ✓
PARITY ✓
GO/NO-GO
BAU
HANDOVER
parity fails → triage, fix, re-run
issue after cutover → roll Qlik back to legacy (still live)
**Inventory:** list the package's views, tables, owners and usage.
Rollback only works because the legacy platform keeps running. That is why each package's legacy objects can't be retired until hypercare closes, and why the parallel run costs money on both platforms at once.
Rollback works only because HANA keeps running. That’s why a package’s legacy objects stay until hypercare closes, and why the parallel run costs money on both platforms at once.
## Two paths run side by side until cutover
Reports keep reading HANA while the new path is built alongside.
Switch between the three moments, then pick any box.
Today
During migration
After cutover
Play the sequence
**During migration:** both paths run. Qlik still reads the legacy platform while each package is rebuilt and checked against it. Records failing DQ checks in Silver go back to SAP to be fixed at source.
The Customer-owned · must already exist
SIGMOID BUILD SCOPE
CURRENT PATH · stays live
TARGET PATH · Databricks on AWS
SAP sources
SAP ECC
BW / BW4HANA
SAP CRM
also: Salesforce,
AWS Redshift
DQ issues
fixed here
Existing feeds
into legacy
SAP legacy HANA
500 models · calc views
ETL jobs · SQL engine
sunset
Qlik reads legacy until its package cuts over
AI reverse-engineering
classify · translate to Databricks SQL
zoom in ↓
view logic & lineage
extracted
rebuilt logic
Customer ingestion
existing, Customer-run
S3 landing
existing
Bronze
existing, Customer-run
[zoom in ↓](#components-sec)
Silver
clean · dedupe
[zoom in ↓](#components-sec)
Gold
data products
[zoom in ↓](#components-sec)
Semantic layer
Qlik-facing
[zoom in ↓](#components-sec)
Parity gate
+ UAT sign-off
cutover →
compare output
against legacy
fails DQ rules
DQ quarantine
rule · severity · record keys · owner
routed back to SAP data owners to correct at source
corrected records flow in again through replication (CDC)
Consumers
Qlik Sense
QlikView
reports: Customer scope
Databricks SQL
Downstream
via Unity Catalog
Runs across every layer
Unity Catalog (catalog, lineage, access) · DQ monitoring & alerts · Databricks Workflows · AWS IAM / Secrets (Customer-owned)
Legacy decommissioned
The dashed line from the legacy platform into Silver and Gold is where legacy logic gets rebuilt. The red loop is the data quality feedback path: records that fail DQ rules in Silver are held in quarantine rather than passed to Gold, and each exception goes back to the SAP data owner to be corrected in the source system, so the fix is permanent and the legacy platform and Databricks stay comparable. The dashed line down into the parity gate is where each package has to prove it matches the legacy platform before Qlik is moved over. The semantic layer is the Qlik-facing boundary. Sigmoid migrates it, and any Qlik report change that follows is the Customer's responsibility, so keeping field names stable limits that work.
## Zoom in on each component
Pick a component, then step through it or press Play.
Each tab opens one box from the architecture diagram, with an animation, the steps in detail, the scope and who owns what. The AI reverse engineering box and the DQ quarantine and parity gate boxes already have their own sections: [Reverse engineering](#ai) and [DQ & parity gate](#dq). Dictionary and model design is in [Dictionary & data model](#entities), and the table-by-table route is in [Data migration](#datamove).
Examples and values are illustrative. Items labelled to confirm come from general SAP and Databricks practice, not from the Customer's documents.
**0**Current state: sources, the legacy platform, Qlik**1**Bronze handover and Silver input contract**2**Silver layer engineering**3**Gold data products**4**Qlik-facing semantic layer**5**Unity Catalog: where it sits and when it acts**6**Governance, quality monitoring, orchestration**7**Migration factory and release pipeline**8**Cutover, coexistence and hypercare**9**Steady state after cutover: change data flow**10**No duplicates: CDC, quarantine and reloads
This is the part of the architecture that exists today and that the Customer owns. Sigmoid does not build it or change it, but every migration decision starts from understanding it, so the first tab shows how data gets from the SAP sources to Qlik and which parts Sigmoid reads in Phase 1.
**1**SAP sources**2**Feeds into HANA**3**Base tables**4**Models**5**XS jobs and scripts**6**Consumption**7**What Sigmoid analyses
‹ BackPlayNext ›
SAP sourcesECCBW / BW4HANACRM+ custom Z objects
Feeds into HANASAP SLTSAP SDIODBC / JDBCto confirm per package
Base tablesreplicated copiesof source tables
Legacy modelsattribute viewsanalytic viewscalculation views500 in 15 packages
XS jobs and scriptsscheduled jobs, XSJSfeed or reshape data
QlikQlik SenseQlikViewreads over ODBC/JDBC
Analytic privilegesrow-level accessby user or role
SOURCE · CUSTOMERFEEDS · CUSTOMERSIGMOID READS (read-only)SIGMOID READS jobsSIGMOID READS the feed set-upSIGMOID READS the app-to-view mapAmber boxes stay live until each package cuts over. Nothing here is changed by the migration.
The path shown in amber stays live until each package cuts over. The feeds in use are SLT, SDI and ODBC/JDBC. Which feed serves which package is not yet known.
### The steps in detail
1
### SAP sources
Customer
**What happens.** ECC, BW and CRM hold the transactional, master and reference data. Custom Z tables and Z reports in ECC can hold extra logic.
**Example.** Billing sits in the ECC tables VBRK and VBRP and customers in KNA1. A Z report may reshape billing before it reaches HANA, and no HANA view will show that.
2
### Feeds into HANA
Customer
**What happens.** SAP SLT, SAP SDI and ODBC/JDBC pulls copy the data into HANA. A feed can filter or change data on the way. Which feed serves which package is to confirm.
**Example.** If a feed excludes one company code, every model above it silently misses that data, and the HANA views look correct.
3
### Base tables
Customer
**What happens.** Replicated raw copies of the source tables sit in HANA schemas. These are the tables the models read.
**Example.** KNA1 in HANA holds the customer rows. Whether deletes in SAP also disappear here depends on the feed.
4
### Models
Customer
**What happens.** Attribute views hold master data, analytic views join facts to dimensions, and calculation views (graphical or SQLScript) add logic. There are 500 models in 15 packages, two thirds of them undocumented.
**Example.** A billing view (illustrative) joins billing items to customers, filters company codes and converts currency.
5
### XS jobs and scripts
Customer
**What happens.** Scheduled XS jobs and XSJS scripts refresh or reshape data that the models read. The XS applications are out of scope, but the jobs that feed models change what the views return.
**Example.** A nightly job fills a summary table that a calculation view reads. The view definition does not show the job.
6
### Consumption
Customer
**What happens.** Qlik apps read the models over ODBC/JDBC. Analytic privileges limit which rows each user sees.
**Example.** An EMEA user opens a billing app and sees EMEA rows only, because of an analytic privilege.
7
### What Sigmoid analyses
Sigmoid, Phase 1
**What happens.** Read-only. Sigmoid reads model definitions and dependencies, the XS jobs, the feed configuration and the Qlik app-to-view map. Nothing in the legacy platform changes, and the legacy platform stays live until each package is signed off.
**Example.** The asset list by tier is in [Reverse engineering](#ai), under "What to analyse in HANA".
### Scope and owners
Item
Scope source
When
Owner
**Legacy models, jobs and dependencies**
Assess all 500 models and document logic, lineage and business rules
Phase 1
Sigmoid analyses, the Customer provides access
**SAP sources and the feeds into HANA**
Out of scope: changes to source systems (ECC, BW, CRM)
None
Customer
**XS applications, HTML5 and UI5**
Out of scope: migration of other applications on the legacy platform
None
Customer
**Qlik apps and reports**
Read access to see which views feed which reports. Report refactoring is out of scope.
Phase 1 and later
Customer
### What to watch
Severity
Watch-out
What to do
High
**Logic outside the views.** Z reports, feed filters and XS jobs change the data but are not in the view definitions.
Read the feed configuration, the XS jobs and the sample zTables and zReports requested before kickoff.
Medium
**Feed per package is unknown.** It is not yet known whether SLT, SDI or ODBC/JDBC serves each package.
Ask the Customer for the feed map in Phase 1.
Medium
**Hidden consumers.** A view may feed something other than Qlik.
Check the audit log and ask owners before any retire decision.
Bronze is the Customer's existing, managed layer, and Sigmoid's scope starts at Silver. This tab shows the handover between the two: how Sigmoid checks that Bronze holds what the models need, what happens to gaps, and the signed contract that tells Silver which tables to read.
**1**Bronze is in place**2**Profile the structure**3**Read what the models need**4**Compare and find gaps**5**Route the gaps**6**Sign the contract
‹ BackPlayNext ›
CUSTOMER · BRONZESIGMOID · ANALYSEOUTPUTS
Bronze (existing)
KNA1 customer masterKNVV sales area dataVBRK / VBRP billingT005 countries✓✓✓missinghistory from 2019 · CDC columns
Profile structurecolumns · keys · typeshistory · delete flags
What models needtables, columns, keys,history per package
Compareneeds againstwhat Bronze holds
Gap listT005 missingsales area from 2022
Schema contractper package, signedSilver reads only this
Silvercontractedtables only
gaps go back to the CustomerBronze data quality validation stays with the Customer. Sigmoid checks structure and fit, not content.
Red dashed line: a gap is an action for the Customer, or a change request if the dataset is new. Sigmoid analyses the Bronze structure and confirms the Silver input schemas.
### The steps in detail
1
### Bronze is in place
Customer
**What happens.** The Customer's ingestion team keeps Bronze loaded and current. Sigmoid gets read access in the first two weeks.
**Example.** KNA1, KNVV and the billing tables are loaded, with history from 2019 and change-data-capture columns that show inserts, updates and deletes.
2
### Profile the structure
Sigmoid
**What happens.** Sigmoid reads the structure: columns, types, keys, history depth, delete flags and CDC columns. This is a structural check. Validating Bronze data quality is the Customer's job.
**Example.** KNA1 has key KUNNR, a delete flag and history from 2019. KNVV starts in 2022.
3
### Read what the models need
Sigmoid
**What happens.** From the Tier 2 analysis of surviving models, list for each package the source tables, columns, keys and history they read.
**Example.** The pilot package needs KNA1, KNVV, ADRC and T005, and history back to 2019 for the year-end views.
4
### Compare and find gaps
Sigmoid
**What happens.** Compare what the models need with what Bronze holds. A gap is a missing table, a missing column, a shorter history or a different key.
**Example.** T005 is missing from Bronze, and the sales-area table starts in 2022 while the year-end views need 2019.
5
### Route the gaps
Customer
**What happens.** A missing table that models need is a Customer action to load it, or a change request if it is a new dataset. A history gap goes to the report owner. Bronze data quality validation stays with the Customer.
**Example.** The Customer agrees to load T005. The 2022 start is raised with the owner of the year-end views as a history decision.
6
### Sign the contract
Sigmoid drafts, the Customer confirms
**What happens.** For each package, record the tables, columns, keys, history, refresh and who owns the feed. Silver reads only contracted tables. Later changes go through scope control.
**Example.** The pilot package contract: 4 tables, 61 columns, history from 2019, refreshed nightly, feed owned by the Customer ingestion team. (Illustrative.)
### Scope and owners
Item
Scope source
When
Owner
**Analyse Bronze structure, confirm Silver input schemas**
Phase 1
Phase 1
Sigmoid
**Bronze ingestion and existing tables**
Out of scope: established and maintained by the Customer
All phases
Customer
**Bronze data quality validation**
The Customer's responsibility, excluded from Sigmoid scope
All phases
Customer
**Datasets not in Bronze**
A change request
As found
Customer action or change order
### What to watch
Severity
Watch-out
What to do
High
**Gaps found late.** A missing table found during the build holds up a wave while the gap is fixed.
Run the comparison for each wave before the build starts, starting with the pilot.
High
**History depth is undefined.** How many years Bronze holds is not stated.
Agree depth per package in the contract, and keep period-end data for at least one close.
High
**Bronze storage is undocumented.** Format, Unity Catalog registration, history depth and how deletes are stored are not stated.
Ask for a sample Bronze table and the table inventory at kickoff. The known and unknown items are listed in Data migration.
Medium
**Contract drift.** The Customer changes a feed after the contract is signed.
Record changes against the contract, and log any effect through scope control.
Silver is where Sigmoid's scope starts. It cleans and standardises Bronze data, removes duplicates and holds the business logic rewritten from HANA SQL. The animation follows one small sample of records through each stage.
**1**Start from Bronze**2**Standardise**3**De-duplicate**4**Rewrite the business logic**5**Run the DQ rules**6**Write the Silver table**7**Register in the catalog
‹ BackPlayNext ›
Bronze tableraw, with CDCcolumns
Standardisetypes · codes ·dates · units
De-duplicateone row per key,latest change
Business logicHANA SQL rewrittenas Databricks SQL
DQ rulespass or holdsee quarantine
Silver Deltapartitioned,named, enforced
Catalogowner,lineage
held rows go to quarantine
SAMPLE RECORDS (ILLUSTRATIVE)
100234 'DE ' 20240301 op=I100234 'DE ' 20240301 op=U100871 ' ' 20240302 op=ISilver is built per package,only from contracted Bronze tables,and only for models with a signed treatment.
The DQ rules stage is shown in full, with the quarantine route, in the DQ quarantine zoom-in. The stages and examples are illustrative.
### The steps in detail
1
### Start from Bronze
Customer data
**What happens.** Raw rows exactly as loaded, with change-data-capture columns for insert, update and delete.
**Example.** Customer 100234 appears twice, as an insert and then an update. Customer 100871 has a blank country.
2
### Standardise
Sigmoid
**What happens.** Cast types, trim and map codes, convert dates to one format, and align currency and unit codes.
**Example.** 'DE ' becomes DE, the date becomes 2024-03-01, and a blank country becomes null so rules can see it.
3
### De-duplicate
Sigmoid
**What happens.** Keep one row per key, using the latest change. Apply an agreed rule for deletes and late-arriving changes.
**Example.** The insert and update for 100234 collapse into one row, the latest.
4
### Rewrite the business logic
Sigmoid, with the AI
**What happens.** The HANA logic for surviving models is rebuilt as Databricks SQL: derived columns, joins, filters and conversions, following the signed treatment and target model.
**Example.** A region column is derived from the company code, so both customers get EMEA.
5
### Run the DQ rules
Sigmoid and the Customer
**What happens.** Rules check completeness, uniqueness, references and business rules. Passing rows continue. Failing rows are held in quarantine, not dropped.
**Example.** 100234 passes. 100871 fails "country must not be blank" and is quarantined for the SAP owner.
6
### Write the Silver table
Sigmoid
**What happens.** Passing rows are written to a Delta table with a partition strategy, the naming standard and an enforced schema.
**Example.** silver.customer is partitioned by region. One row written, one held.
7
### Register in the catalog
Sigmoid, Unity Catalog
**What happens.** The table is registered in Unity Catalog with an owner, a description, lineage back to Bronze and to the legacy platform model it replaces, and an access policy.
**Example.** silver.customer: owner the EMEA customer master team, lineage to Bronze KNA1 and to the legacy platform customer view, row filter by region.
### Scope and owners
Item
Scope source
When
Owner
**Silver engineering: de-duplication, simplification, rewriting HANA SQL**
All view types (calculation, analytic, attribute)
Phases 2 and 3
Sigmoid
**Delta structure, partition strategy, naming, Unity Catalog standards**
Designed in Phase 1
Phase 1
Sigmoid
**Pilot Silver layer (Customer domain)**
Phase 2
Phase 2
Sigmoid
**Silver for the remaining 14 packages**
Phase 3, Waves 1 to 3
Phase 3
Sigmoid
**Workspaces, access and cloud set-up**
Out of scope: the Customer provisions
Before kickoff and within 2 weeks
Customer
### What to watch
Severity
Watch-out
What to do
High
**De-duplication drops real history.** A latest-change rule can lose late-arriving changes or valid deletes.
Agree the rule per table and test it on a period-end in the pilot.
High
**Logic rewritten without a signed treatment.** Rebuilding a model that should have been merged or retired wastes effort.
No model enters Silver build without a signed treatment and target entity.
High
**Replays create duplicates.** A re-sent change file, a job re-run or a fixed record returning from quarantine can add a second row for the same key.
Merge by key and apply only newer changes, so a replay does nothing. The design is in the duplicate-protection tab.
Medium
**Partition strategy chosen too early.** The right choice depends on real volumes.
Decide in Phase 1 from the Bronze analysis, and re-check in the pilot.
Gold is where the 500 models become reusable data products. There does not need to be one Gold table for each legacy view: several views can feed one dimension, and one product can serve many reports. This tab covers how Gold is built and published. The design rules are in the dictionary and model section.
**1**Start from Silver**2**Conformed dimensions first**3**Facts at a declared grain**4**Consolidate**5**Publish as a data product**6**Consume, then prove
‹ BackPlayNext ›
Silver · customer
Silver · billing
Silver · material
Silver · calendar, plant
Conformed dimensionscustomer · materialplant · calendarshared by all domains
Facts at a grainbilling lineopen itemone row means one thing
Consolidateseveral legacy viewsbecome one entitymeasures defined once
Gold data productowner · definitionrefresh · accessDQ rules attachedcatalogue entrydocumentation
Semantic layerQlik-facing
Databricks SQLanalysts
Other consumersdownstream
Parity gate and UAT sign-off sit between Gold and cutover. See the DQ and parity gate section.
The consolidation step is where logic changes can hide, so every consolidated entity is tested against all the legacy platform views it replaces at the parity gate.
### The steps in detail
1
### Start from Silver
Sigmoid
**What happens.** Only entities that survived the Decide gate are built. Silver entities come from several domains.
**Example.** Customer, billing, material and calendar and plant data are all in Silver for the pilot and first wave.
2
### Conformed dimensions first
Sigmoid, owners sign off
**What happens.** One customer, material, plant and calendar dimension is shared by every domain, so numbers agree across reports. These are designed in Phase 1.
**Example.** Finance and Sales both read the same customer dimension, so a customer's country is defined once.
3
### Facts at a declared grain
Sigmoid
**What happens.** Each fact states what one row means, and carries measures defined once.
**Example.** The billing fact has one row per billing document item. Net value and tax are measures on that row.
4
### Consolidate
Sigmoid, with the AI
**What happens.** Several legacy views that serve the same purpose are rebuilt once. The old names return later as compatibility views, covered in the semantic layer zoom-in.
**Example.** The EMEA and NA billing views differ only by a company-code filter. They become one billing fact with a region column.
5
### Publish as a data product
Sigmoid, Unity Catalog
**What happens.** Each product has an owner, a definition, a refresh schedule, access rules, attached quality rules and a catalogue entry with documentation.
**Example.** Customer dimension: owner the EMEA customer master team, nightly refresh, six DQ rules, row filter by region.
6
### Consume, then prove
Sigmoid and the Customer
**What happens.** The semantic layer, SQL analysts and other consumers read the product. No package cuts over until the parity gate and UAT sign-off pass.
**Example.** Qlik reads through the semantic layer, analysts query in SQL, and the gate compares Gold with the legacy platform by region and period.
### Scope and owners
Item
Scope source
When
Owner
**Reconcile, rationalise and migrate 500 models into domain-aligned, reusable data products**
Gold layer engineering
Phases 2 and 3
Sigmoid builds, domain SMEs sign off
**Pilot Gold layer (Customer domain, first data-as-a-product)**
Phase 2
Phase 2
Sigmoid
**Gold for the remaining packages**
Phase 3, Waves 1 to 3
Phase 3
Sigmoid
**Data model inventory and data catalogue**
Post-implementation documentation
Phases 3 and 4
Sigmoid hands over
**Ownership after the programme**
Knowledge transfer to the Customer governance team
Phase 4
Customer
### What to watch
Severity
Watch-out
What to do
High
**Consolidation hides logic changes.** Merging views can change results for one region.
Test each consolidated entity against every original view at the parity gate, region by region.
Medium
**Over-modelling.** Modelling beyond what reports consume delays the pilot.
Model only entities that surviving views and reports use, one wave ahead.
Medium
**Product owners are unnamed.** A data product with no owner has nobody to approve changes or fix quality issues.
Name an owner per product before it is published, and record it in the catalogue.
Qlik reads the old legacy views today, and Sigmoid migrates the layer Qlik will read tomorrow. This tab shows how the data models in front of Qlik are rebuilt on Gold, who does what, and why the legacy platform stays reachable until a package is signed off.
**1**Start from Gold**2**Compatibility views**3**Map the fields**4**The Customer repoints**5**Continuity check**6**Switch, with a way back
‹ BackPlayNext ›
Gold productscustomerbillingmaterial
Compatibility viewsold view namesold column namesnew Gold logic
Field mappingQlik field toGold columnone row per field
Qlik appsrepoint the connectionfix what breaks
Continuity checksame charts, samenumbers, same filters
Legacy (still live)Qlik can switch backuntil sign-off
CutoverQlik reads DatabricksLegacy retired later
CUSTOMER WORKSIGMOID BUILDSSIGMOID BUILDSSIGMOID VALIDATESData models only. Qlik report refactoring is out of scope.
Red dashed line: if a check fails, Qlik points back to the legacy platform. The compatibility-view approach is proposed. The commitment is to migrate the data models and validate Qlik Sense report continuity.
### The steps in detail
1
### Start from Gold
Sigmoid
**What happens.** The semantic layer reads Gold data products only, never Silver or Bronze directly, so there is one governed route to the data.
**Example.** The billing and customer products are the source for the billing app's data model.
2
### Compatibility views
Sigmoid
**What happens.** Views expose the old view and column names on top of the new logic, so apps that used a legacy view need little change. A view that was merged in Gold gets a compatibility view for each old name.
**Example.** The old EMEA and NA billing views both exist as names, and both read the single Gold billing fact with a region filter.
3
### Map the fields
Sigmoid, the Customer
**What happens.** Using the Qlik read access, list which views feed which apps and map each Qlik field to a Gold column. Any field with no match is listed for a decision.
**Example.** Field NetValue maps to fact_billing.net_value. Field LegacyFlag has no Gold column and goes to the app owner.
4
### The Customer repoints
Customer
**What happens.** The Customer changes the Qlik connection and any report-level fixes. This is outside Sigmoid's scope, so it needs a named Qlik owner and a slot in the wave plan.
**Example.** The billing app's connection points to the new views. The owner fixes one chart that used a field that was renamed.
5
### Continuity check
Sigmoid validates, the Customer signs
**What happens.** Open each app on the legacy platform and on Databricks and compare charts, totals and filters. This sits alongside the data parity gate and covers what users see.
**Example.** The EMEA revenue chart shows the same total and the same top ten customers on both. One filter list is missing a country, so the check fails.
6
### Switch, with a way back
The Customer and Sigmoid
**What happens.** Qlik switches at the package cutover. The legacy platform stays live and Qlik can point back to it until final sign-off. The rollback criteria are agreed before cutover.
**Example.** After a clean check the billing app switches. A week later a regional total is questioned, and the team can compare against the legacy platform the same day.
### Scope and owners
Item
Scope source
When
Owner
**Migrate the Qlik-facing semantic layer for the Customer domain (pilot) and validate Qlik Sense report continuity**
Phase 2
Phase 2
Sigmoid
**Qlik-facing semantic layer migration and optimisation for the remaining packages (data models only)**
Scope table
Phase 3
Sigmoid
**Downstream Qlik report refactoring and migration**
Out of scope. Any report change is the Customer's
All phases
Customer
**Qlik Sense / QlikView read access to see which views feed which reports**
The Customer prerequisite
Within 2 weeks of kickoff
Customer
### What to watch
Severity
Watch-out
What to do
High
**Qlik workload is out of scope.** A small data-model change can mean a lot of report work, and Sigmoid has no hours for it.
Size the report impact per package in Phase 1 and name the Customer Qlik owner. Anything more goes through scope control.
High
**Unmapped fields.** Fields with no Gold column surface late, in UAT.
Complete the field map before the wave build starts and review gaps with app owners.
Medium
**Compatibility views become permanent.** They can outlive their purpose.
Record a retire date for each compatibility view in the data catalogue.
Unity Catalog is the governance layer in Databricks. It is not a pipeline stage, so no data flows through it. It decides who can see what, records where every column came from, and holds the names and owners of every table. This tab shows where it sits, when each capability acts, and who owns which part.
**1**Where it sits**2**Namespace and layout**3**At build: register**4**At query: permissions and filters**5**As jobs run: lineage**6**Audit and quality results**7**Qlik and analysts**8**Hand over
‹ BackPlayNext ›
Unity Catalog: one governance layer over every layer
namespaceaccess controlrow filters, maskslineageauditdiscovery, tagsThe Customer provisions and administers it. Sigmoid designs the standards and builds into it.
BronzeCustomer-run
SilverSigmoid builds
GoldSigmoid builds
SQL warehousesemantic layer
Qlik and analystsconsumers
Workflowsscheduled loads
WHAT UNITY CATALOG DOES AT THIS POINT (ILLUSTRATIVE)
Unity Catalog is set up and administered by the Customer. Sigmoid provides the governance standards, the lineage documentation and the quality monitoring. Specific settings below are to be confirmed in Phase 1.
### The steps in detail
1
### Where it sits
The Customer provisions
**What happens.** Unity Catalog sits over the whole Databricks platform, beside the pipelines rather than inside them. Every table in Bronze, Silver and Gold, every query from the SQL warehouse and every Workflows job goes through it. Access to the S3 storage underneath is also governed through it.
**Example.** A Workflows job writing silver.customer and an analyst reading gold.fact_billing are both checked by the same catalog.
2
### Namespace and layout
Sigmoid designs in Phase 1
**What happens.** Every object has a three-part name: catalog, schema, table. The layout is part of the Phase 1 design: for example a catalog per environment and a schema per layer or domain. This is a recommendation to settle with the Customer.
**Example.** prod.silver.customer and prod.gold.dim_customer are in the same catalog. dev.silver.customer holds the test copy.
3
### At build: register
Sigmoid, with the pipeline
**What happens.** When a pipeline first creates a table, it is registered with an owner, a description and tags. Doing this in the build, not afterwards, is what makes the catalogue usable at handover.
**Example.** gold.dim_customer is created with owner "EMEA customer master", a plain-English description, and the tag domain=customer.
4
### At query: permissions and filters
Unity Catalog, rules by Sigmoid
**What happens.** On every query, the catalog checks the user's grants. Row filters and column masks then shape what comes back. This is where the old HANA analytic privileges are rebuilt. The rules are designed by Sigmoid and approved by the Customer.
**Example.** An EMEA analyst queries gold.fact_billing and receives EMEA rows only. A column with a bank detail comes back masked.
5
### As jobs run: lineage
Unity Catalog, automatic
**What happens.** When Workflows and queries run on Unity Catalog tables, the catalog records which tables and columns each one read and wrote. Nobody has to maintain that by hand. The link back to the legacy platform view is not automatic, so Sigmoid records it as a tag or description.
**Example.** gold.fact_billing.net_value traces back to silver.billing and on to the Bronze billing table. A tag names the legacy platform view it replaces.
6
### Audit and quality results
Unity Catalog, Sigmoid reads
**What happens.** Access and activity are recorded in system tables, so who read what and when can be queried. Data quality results are stored as tables in the same catalog, so the dashboard sits under the same permissions.
**Example.** An audit query lists every user who read gold.fact_billing in the last 30 days, which also answers whether a legacy view still has consumers.
7
### Qlik and analysts
The Customer and Sigmoid
**What happens.** Qlik reaches the data through a SQL warehouse and is identified by the account it connects with. If Qlik uses one shared service account, per-user row filters cannot apply. How each Qlik app authenticates is a decision to take in Phase 1.
**Example.** With a shared account, every Qlik user sees what that account sees. HANA applied each user's own analytic privileges.
8
### Hand over
Customer governance team
**What happens.** At handover the governance team owns grants, owners, reviews and the audit queries, using Sigmoid's standards and runbook. The catalog is what stays when the programme ends.
**Example.** A new analyst group is added by a governance team member following the runbook, with no Sigmoid involvement.
### Scope and owners
Item
Scope source
When
Owner
**Unity Catalog governance standards, naming and structure**
Phase 1 (Silver and Gold architecture design)
Phase 1
Sigmoid
**Databricks workspace and Unity Catalog set-up, access provisioning**
Out of scope: The Customer's cost and responsibility
Before kickoff
Customer
**Data catalogue and lineage documentation**
Deliverables 4.3 and 4.4
Phases 3 and 4
Sigmoid
**Row filters and masks replacing HANA analytic privileges**
Falls under security and governance design
Phases 1 to 3
Sigmoid designs, the Customer approves
**Ongoing grants, owners and audit review**
Knowledge transfer on platform operations
Phase 4 onwards
Customer
### What to watch
Severity
Watch-out
What to do
High
**Bronze may not be in Unity Catalog.** It is not known whether the Customer's Bronze tables are registered in it. If they are not, lineage stops at Silver and Bronze access is governed elsewhere.
Ask in the first two weeks whether Bronze is registered, and plan the lineage break if it is not.
High
**Qlik identity.** A shared service account defeats per-user row filters, which would be a security regression from HANA.
Confirm how each Qlik app authenticates before designing the filters, and agree the answer with security.
Medium
**Unity Catalog settings are owned by the Customer.** A late or different design (catalog layout, metastore) moves Sigmoid's standards.
Agree the catalog layout in Phase 1 and treat any change after the pilot as a scope-control item.
Medium
**Registering later.** Owners and descriptions added after the build rarely get added at all.
Make owner, description and tags part of the definition of done for each table.
This tab covers what keeps the migrated platform trustworthy after each package goes live: the catalog and standards, lineage, access, scheduled loads and quality monitoring. The workspace itself belongs to the Customer, and Sigmoid designs the standards and monitoring that run on it.
**1**Unity Catalog**2**Standards**3**Lineage**4**Access**5**Orchestration**6**Quality monitoring**7**Hand over to operate
‹ BackPlayNext ›
Unity Catalogcatalogs, schemasowners, descriptionsset up by the Customer
Standardsnaming, structurehow products areregistered
LineageBronze to Silverto Gold to Qlikplus the legacy view
Accessrow filters fromanalytic privilegesroles and groups
OrchestrationDatabricks Workflowsorder, retries,alerts
Quality monitoringrule results per runtrend by tablequarantine counts
Quality dashboardlive from Wave 1for owners and Customergovernance team
OperateCustomer governanceteam takes overrunbooks, handover
Workspace, cloud and Unity Catalog set-up are the Customer's cost and responsibility. Sigmoid designs the standards.
The one-way arrow to Operate is the knowledge transfer in Phase 4. Orchestration details beyond Databricks Workflows are to confirm in Phase 1.
### The steps in detail
1
### Unity Catalog
The Customer provisions
**What happens.** The Customer creates the Databricks workspace and Unity Catalog and provisions access. Sigmoid cannot start Silver build until this exists.
**Example.** Catalogs per environment and schemas for Silver and Gold, created before the pilot build.
2
### Standards
Sigmoid, Phase 1
**What happens.** The Phase 1 design sets naming conventions, table structure, partition strategy and how a product is registered, so every package is built the same way.
**Example.** silver.customer and gold.dim_customer follow one pattern, with an owner and a description recorded at creation.
3
### Lineage
Sigmoid
**What happens.** Lineage links each Gold column to its Silver and Bronze sources, and to the legacy platform view it replaces. The catalogue and lineage documentation is a deliverable.
**Example.** gold.fact_billing.net_value traces back to Bronze VBRP and to the legacy platform billing view it replaces.
4
### Access
Sigmoid designs, the Customer approves
**What happens.** HANA analytic privileges become Databricks row filters and column rules, so users see the same rows as before. Security design needs Customer approval.
**Example.** The EMEA analyst group sees EMEA rows only, matching the old privilege.
5
### Orchestration
Sigmoid
**What happens.** Loads run in order through Databricks Workflows, with retries, alerts and dependencies between Silver and Gold. The trigger from Bronze load is to confirm.
**Example.** The Gold billing refresh waits for the Silver billing job to finish, and an alert goes to the on-call channel if it fails.
6
### Quality monitoring
Sigmoid
**What happens.** Every run records rule results, trends and quarantine counts per table. Monitoring is live across migrated domains from Wave 1 onwards.
**Example.** The dashboard shows billing completeness at 99.8% this week against 99.9% last week, and the held-row count by rule.
7
### Hand over to operate
Customer governance team
**What happens.** In Phase 4, Sigmoid hands over runbooks, dashboards and monitoring in knowledge transfer, accepted by the Customer at the final milestone.
**Example.** The governance team runs the dashboard review in the first week with Sigmoid watching, then on its own.
### Scope and owners
Item
Scope source
When
Owner
**Unity Catalog governance standards, Delta structure, naming and partition design**
Phase 1
Phase 1
Sigmoid
**Cloud infrastructure, Databricks workspace and Unity Catalog set-up**
Out of scope: The Customer's cost and responsibility
Before kickoff
Customer
**Data quality monitoring across migrated domains**
Phase 3, from Wave 1 onwards
From the first wave
Sigmoid
**Data catalogue, lineage documentation, operations runbook**
Deliverables 4.3 and 4.4
Phases 3 and 4
Sigmoid hands over
**Knowledge transfer on platform operations and quality monitoring**
Phase 4, accepted at the final milestone
Phase 4
Sigmoid to the Customer
### What to watch
Severity
Watch-out
What to do
High
**Workspace not ready.** Late provisioning delays the pilot, and dates extend day for day.
Confirm workspace, Unity Catalog and IAM dates before kickoff and track them as a dependency.
High
**Security parity.** Analytic privileges differ from Databricks filters and gaps can leak data.
Convert privileges per package, test with real user groups and have security approve them.
Medium
**Monitoring without an owner.** A dashboard nobody reviews does not catch anything.
Name a quality owner per domain and review the dashboard in the governance forum.
Rebuilding 500 models by hand would not fit the timeline, so the plan relies on a repeatable process: patterns, generation, automated tests and a release pipeline. This tab shows that loop, and where one open question remains: who owns the CI/CD pipeline.
**1**Patterns first**2**Intake**3**Generate and review**4**Automated tests**5**Fail and loop back**6**Pattern fix, then promote**7**Record the outcome
‹ BackPlayNext ›
Patternstemplates for thecommon view typesbuilt in Phase 1 and 2
Intakeview with signedtreatment andtarget entity
GenerateAI drafts SQL fromthe view and patternengineer reviews
Automated testsschema, rules,row counts, valuesagainst legacy output
Failfix the viewor the pattern
Promotedev to test to prodthrough a pipelineowner: to confirm
Pattern fixone change correctsevery similar view
Migration log500 models, eachwith its outcome
Automated test cases are finalised by the Phase 2 kickoff milestone, and the pipeline is built in Phase 2.
The red dashed lines are failure loops. Automated test cases and an automated DQ rule engine are Phase 2 outputs, and a complete model migration log is a deliverable. Who builds and runs the CI/CD pipeline is not stated.
### The steps in detail
1
### Patterns first
Sigmoid
**What happens.** The pilot and the hardest views produce templates for common view types. The patterns become the base for the waves.
**Example.** A pattern for a calculation view that joins billing items to customers with a currency conversion, built once in the pilot.
2
### Intake
Sigmoid
**What happens.** A view enters the factory only with a signed treatment, a target entity and its Bronze contract. Anything else waits.
**Example.** View 142 is marked "translate", maps to the billing fact and has its Bronze tables contracted, so it enters the queue.
3
### Generate and review
Sigmoid, with the AI
**What happens.** The AI tool drafts Databricks SQL from the view definition and the pattern, and an engineer reviews it. The AI drafts, and a person decides.
**Example.** The draft for view 142 rewrites the currency conversion. The engineer corrects one rounding rule.
4
### Automated tests
Sigmoid
**What happens.** Tests check schema, DQ rules, row counts and values against the frozen legacy output. Automated tests are what allow 500 models to be proved, not just built.
**Example.** Row counts match for all regions, and one value differs by 0.02 in one company code.
5
### Fail and loop back
Sigmoid
**What happens.** A failure goes back to the view, or to the pattern if the cause is shared. The fix is retested before anything moves on.
**Example.** The 0.02 difference comes from rounding in the pattern. Fixing it in the pattern corrects all views that use it.
6
### Pattern fix, then promote
Sigmoid
**What happens.** A pattern fix is applied to every similar view and retested. Passing views are promoted from dev to test to prod through a pipeline.
**Example.** The rounding fix corrects 40 similar views and they are retested together. Promotion runs from a pipeline, not by hand.
7
### Record the outcome
Sigmoid
**What happens.** Every model gets a log entry with its treatment, its tests and its sign-off, building toward the complete model migration log.
**Example.** Model 142: translated, 14 of 14 tests passed, signed off during the pilot build.
### Scope and owners
Item
Scope source
When
Owner
**Automated test cases finalised and automated DQ rule engine designed**
Phase 2 milestone 1 (kickoff)
By the Phase 2 kickoff milestone
Sigmoid
**Pilot build, then waves 1 to 3 at scale**
Phases 2 and 3
Phases 2 and 3
Sigmoid
**Complete model migration log**
Phase 3 deliverable
Phase 3
Sigmoid
**CI/CD pipeline build and operation**
Not stated in the Sigmoid role list
To decide
Open: see the watch-outs
### What to watch
Severity
Watch-out
What to do
High
**CI/CD owner is undefined.** Automated tests are listed but no pipeline engineer is, and the Customer platform role covers set-up only.
Ask in Phase 1 who builds and runs the pipeline, and whether tools and repos are the Customer's or Sigmoid's. The detail is in the watch-outs section.
High
**Patterns spread a mistake.** One wrong pattern is copied into many views.
Test a pattern on the hardest sample first, and retest every view it touches after a fix.
Medium
**Review fatigue.** Engineers stop checking AI drafts closely.
Sample deep reviews per wave and track the defect rate found after promotion.
Cutover happens fifteen times, once per package. This tab shows what must be true before each one, how the legacy platform and Databricks coexist until then, and what Sigmoid hands over at the end. The final step, switching the legacy platform off, belongs to the Customer.
**1**Plan around business cycles**2**Readiness check**3**Parallel run**4**Cutover**5**Rollback if needed**6**Hypercare**7**Sign off and decommission readiness
‹ BackPlayNext ›
Wave planavoids financialclose and budgetwindows
Readiness checkgate passedUAT signedrollback agreed
Parallel runLegacy and Databricksboth livePackage D runs longer
CutoverQlik readsDatabricksone package at a time
Hypercare30 days aftercutoverdefect SLA
Sign-offfinal parity,decommissionreadiness
RollbackQlik points backto legacy
Legacy decommissioncertificate issued by Sigmoid; the Customerturns legacy off and removes objects
15 production cutovers by the final cutover milestone, hypercare to the handover milestone.
The red dashed line is the rollback route. Rollback criteria are required, and their content is to be agreed in Phase 1.
### The steps in detail
1
### Plan around business cycles
Sigmoid and the Customer
**What happens.** The wave and cutover plan produced in Phase 1 accounts for the Customer's financial close cycles and budgeting windows, so BAU is not disrupted.
**Example.** Package B does not cut over in the final week of a quarter. The Package B package moves to a quieter week.
2
### Readiness check
Sigmoid and business owners
**What happens.** Before a cutover the parity gate has passed, UAT is signed by the business owner, the Qlik continuity check is clean and rollback criteria are agreed.
**Example.** The Package A package has a passed gate, UAT signed by the domain owner and a rollback trigger agreed.
3
### Parallel run
The Customer and Sigmoid
**What happens.** The legacy platform and Databricks both run, and results are compared. Package D, a very high complexity package of 129 objects, needs an extended parallel run and dedicated owner time.
**Example.** Package A runs two close cycles on both sides before it moves. Package D runs longer.
4
### Cutover
The Customer and Sigmoid
**What happens.** Qlik apps for the package read Databricks, and the package is declared live. There are 15 production cutovers in the final milestone.
**Example.** Package A goes live on a Monday after a weekend refresh.
5
### Rollback if needed
The Customer decides
**What happens.** If a pre-agreed criterion is hit, Qlik points back to the legacy platform, which is why the legacy platform must stay live and fully refreshed through the parallel period. The criteria are defined before cutover.
**Example.** A criterion: any revenue total that differs from the legacy platform by more than the agreed tolerance, unresolved for two days, triggers rollback.
6
### Hypercare
Sigmoid
**What happens.** Thirty days of post-cutover hypercare cover defect triage and resolution under an SLA for migrated domains. A further 20 days apply for defects and 10 days for deemed acceptance.
**Example.** A user reports a missing company code on day four. It is triaged, fixed in the pattern and promoted the same week.
7
### Sign off and decommission readiness
Sigmoid certifies, the Customer decides
**What happens.** Final reconciliation sign-off and the decommission readiness certificate close the programme. The Customer turns the legacy platform off, and removing legacy objects is out of scope.
**Example.** The certificate lists each package, its sign-off date and the last legacy read date. The Customer schedules the shutdown.
### Scope and owners
Item
Scope source
When
Owner
**Wave planning, cutover support and BAU continuity management**
Scope table
Phases 1 to 4
Sigmoid
**Production cutovers for all 15 packages with defined rollback criteria**
Phase 4 milestone 1
By the final cutover milestone
Sigmoid with the Customer
**Legacy decommission readiness validation and certificate**
Phase 4
By the final cutover milestone
Sigmoid issues
**Post-cutover hypercare, operations runbooks, knowledge transfer**
Phase 4 milestone 2
By the handover milestone
Sigmoid
**Decommissioning the legacy platform and removing objects**
Out of scope
After certificate
Customer
### What to watch
Severity
Watch-out
What to do
High
**Close and budget windows.** Cutovers that land in a finance close can be refused or rolled back.
Build the calendar with Finance in Phase 1 and treat it as a fixed constraint.
High
**Fifteen cutovers in a short window.** The final milestone needs all fifteen done by the final cutover milestone.
Cut over each package soon after its sign-off, and keep a buffer in the weeks before the final cutover milestone.
Medium
**Rollback criteria stay vague.** They are required, but nobody has written them.
Draft criteria with owners in Phase 1 and rehearse a rollback once.
After cutover there is one way for data to reach the consumers: from the SAP systems through the Customer's ingestion into Bronze, then through Sigmoid's Silver and Gold. The animation follows three changes made in SAP, an update, a delete and a new record, through each stage.
**1**Sources and the legacy platform feeds**2**Capture the changes**3**Land and load to Bronze**4**Silver merges the changes**5**Rules and quarantine**6**Gold refresh**7**Monitor lag and counts**8**Recover and reload
‹ BackPlayNext ›
SAP sourcesECC · BW · CRMand other sources
Qlik Replicatecaptures each changefull load, then CDC
S3 landingfiles asreceived
Bronzeraw changesCustomer-run
Silverincremental mergeby key, rules
Goldrefreshedafter Silver
ConsumersSQL, Qlik
Legacy feedsSLT, SDI, ODBCswitched off by the Customer
WorkflowsSilver first, then Gold, in order
Quarantineheld rows
Monitoringlag, counts
Recoveryrunbook
WHAT HAPPENS TO THE RECORDS (ILLUSTRATIVE)
Bronze ingestion is the Customer's and unchanged by the migration. How Bronze stores changes, and the freshness target, are not known, so those steps are assumptions to confirm in Phase 1.
### The steps in detail
1
### Sources and the legacy platform feeds
Customer
**What happens.** SAP stays the system of record, and changes are made there. Until cutover the same data also feeds the legacy platform through SLT, SDI and ODBC. After the decommission readiness certificate, the Customer switches those feeds off, leaving one path.
**Example.** Three changes in SAP: customer 100234 changes country, customer 100871 is deleted, and customer 100999 is created.
2
### Capture the changes
Customer ingestion
**What happens.** After the initial full load, Qlik Replicate captures each change with its operation type and a sequence number. This is the intended design, and the configuration has not been seen.
**Example.** Each change becomes one record with an operation and an order.
3
### Land and load to Bronze
Customer ingestion
**What happens.** Changes land on S3 and are loaded into Bronze as received. Whether Bronze keeps every change as a log or only the current row is not stated, and Silver depends on the answer.
**Example.** If Bronze is a log, all three records are kept. If it holds only the current state, the delete may already be gone.
4
### Silver merges the changes
Sigmoid
**What happens.** Each scheduled run reads only the changes since the last run and merges them by key. The latest change wins, using the sequence number, and a stated rule decides what a delete means, such as a flag or removal.
**Example.** 100234 is updated to FR. 100871 is flagged deleted under the agreed rule. 100999 is inserted.
5
### Rules and quarantine
Sigmoid, SAP owners fix
**What happens.** Rules run during the merge. A failing record is held in quarantine and is not dropped. The SAP owner corrects it at source, and the correction arrives as a new change.
**Example.** 100999 has no country, so it is held. After the owner fixes it in SAP, the next change flows through.
6
### Gold refresh
Sigmoid, Workflows
**What happens.** Workflows run Gold only after Silver has finished, so consumers never see a half-merged state. Consumers read Gold through the semantic layer and SQL warehouse.
**Example.** The customer dimension now shows FR for 100234. A report refreshed a minute earlier still shows DE.
7
### Monitor lag and counts
Sigmoid builds, the Customer runs
**What happens.** The dashboard tracks freshness against the agreed target, load lag, row counts against the source and the quarantine backlog. Alerts go out when a load is late or fails.
**Example.** Lag is 18 minutes against a target that has to be agreed. A nightly count check compares source and Bronze.
8
### Recover and reload
The Customer and Sigmoid
**What happens.** If a load fails or a rule changes, Silver and Gold can be rebuilt from Bronze, provided Bronze kept the history. A fresh Bronze reload is the Customer's job. The runbook is handed over in Phase 4.
**Example.** A late change arrives out of order and is replayed in sequence. A wrong rule is fixed and Silver is rebuilt for one table.
### Scope and owners
Item
Scope source
When
Owner
**Bronze ingestion, Qlik Replicate and landing zone**
Out of scope: established and maintained by the Customer
All phases
Customer
**Incremental Silver and Gold processing, merge rules and delete handling**
Silver and Gold engineering. The incremental design is a Phase 1 task.
Phases 1 to 3
Sigmoid
**Data quality monitoring and quarantine routing**
Phase 3, from Wave 1 onwards
From the first wave
Sigmoid builds, the Customer runs
**Switching off the feeds into the legacy platform**
Out of scope: The Customer decommissions the legacy platform
After the certificate
Customer
**Operations runbook and knowledge transfer**
Phase 4
Phase 4
Sigmoid hands over
### What to watch
Severity
Watch-out
What to do
High
**Deletes and late changes.** CDC can miss deletes and deliver changes out of order. Without a rule, Gold silently drifts from SAP.
Agree the delete rule and the ordering column per table in Phase 1, and test both in the pilot.
High
**Nobody owns a reload.** If Bronze is wrong, only the Customer's ingestion team can reload it, and no one has agreed how fast.
Ask for a reload process and a response time at kickoff.
Medium
**No freshness target.** Without one, job frequency, compute cost and the parity check all lack a basis.
Get a refresh time per package from the report owners.
Medium
**Silver cannot be rebuilt.** If Bronze keeps only the current row, history and deletes cannot be replayed.
Find out how Bronze stores changes before designing the merge.
Duplicates are a risk anywhere data is replayed: a retried CDC file, a job re-run, a fixed record coming back from quarantine, or a reload that overlaps live changes. The rule is simple: every step must give the same result if it runs twice. This tab shows how each step is made safe, using one customer record.
**1**How duplicates appear**2**Give every change an identity**3**Apply a change only if it is newer**4**Make re-runs safe**5**Fix from quarantine without doubling**6**Reload and backfill without overlap**7**Protect the Gold grain**8**Prove it and alert
‹ BackPlayNext ›
Reload / backfillCustomer ingestion
Bronze changeskey + sequenceop: I, U, D
Silver mergeone live row per keyapply if newer
Goldone row per grainrebuilt from Silver
Quarantineheld, keyed by key + rule
Batch watermarksaved with the write
CONTROLS IN PLACE SO FAR
1 Every change has an identity: key + sequence2 Silver applies a change only if it is newer3 Re-running a batch gives the same result4 A fixed record replaces the held one5 A reload can overlap the live changes safely6 Gold has one row per grain, tested each run7 Counts reconcile, duplicates raise an alert
THE SAME RECORDS, STEP BY STEP (ILLUSTRATIVE)
The design has to be tested in the pilot. It relies on a reliable sequence number from the source or Replicate, which has not been confirmed.
### The steps in detail
1
### How duplicates appear
Risk
**What happens.** Five common causes: a change file delivered twice, a failed job re-run on the same batch, a fixed quarantined record arriving while the held copy still exists, a reload that overlaps live changes, and a join in Gold that multiplies rows.
**Example.** Customer 100999 is held in quarantine for a missing country. The owner fixes it in SAP, a new change arrives, and the held copy is still waiting to be released.
2
### Give every change an identity
Customer ingestion, Sigmoid checks
**What happens.** Each change record needs the business key and a sequence number from the source or Replicate that only goes up. Exact repeats of the same key and sequence are then recognisable. Whether Bronze carries this is not known, so it is a kickoff question.
**Example.** Record 100999 op=U seq=7060. If the same file arrives twice, both copies share seq=7060.
3
### Apply a change only if it is newer
Sigmoid
**What happens.** Silver merges by key. A change is applied only if its sequence is higher than the one already stored. A replayed or stale change finds nothing to do, so merging twice equals merging once. Deletes are kept as markers with a sequence so a late update cannot bring the row back.
**Example.** seq=7060 is applied. A second copy of seq=7060 is ignored. An old seq=7014 arriving late is ignored.
4
### Make re-runs safe
Sigmoid
**What happens.** Each run reads a fixed range of changes. The merge is atomic, and the high-water mark is saved with the write. If the job fails halfway, the same range is read again and gives the same result. A batch is never half-applied.
**Example.** A run for seq 7011 to 7060 fails after the merge but before it finishes. The re-run reads the same range and changes nothing new.
5
### Fix from quarantine without doubling
Sigmoid, SAP owner fixes
**What happens.** A held record never enters Silver, so it cannot be duplicated there. The fix arrives from SAP as a new change with a higher sequence, and the merge applies it. The held item is then closed as released. If a newer change for that key was already applied, the held copy is stale and is dropped.
**Example.** The fixed 100999 arrives at seq=7060. Silver gets one row, and the quarantine entry for seq=7014 is closed.
6
### Reload and backfill without overlap
The Customer and Sigmoid
**What happens.** A full reload or backfill can overlap changes that are still arriving. Because the merge compares sequence numbers, the overlap does no harm. For a large rebuild, write to a new table and swap it in. CDC then resumes from the reload start position, not its end.
**Example.** A reload re-sends 100999 at its current state while a change at a higher sequence is already in. Silver keeps the newer one.
7
### Protect the Gold grain
Sigmoid
**What happens.** Gold is rebuilt from Silver in a repeatable way, not appended to. Each Gold table has a declared grain, and a uniqueness test runs on every refresh. Joins that multiply rows are caught by that test.
**Example.** gold.fact_billing has one row per billing document item. A join to a customer table with two live rows per customer would double the rows, and the test fails.
8
### Prove it and alert
Sigmoid builds, the Customer runs
**What happens.** At every layer, compare the number of rows with the number of distinct keys. Reconcile the counts: changes read equals inserted plus updated plus deleted plus ignored plus quarantined. Any duplicate key raises an alert and stops Gold from refreshing.
**Example.** Run 118: 640 changes read, 410 updated, 120 inserted, 35 deleted, 60 ignored as repeats, 15 quarantined. The total matches. (Illustrative.)
### Scope and owners
Item
Scope source
When
Owner
**Idempotent merge design, delete handling, quarantine release path**
Part of Silver and Gold engineering. Designed in Phase 1, proved in the pilot.
Phases 1 and 2
Sigmoid
**Automated test cases and the automated data quality rule engine**
Phase 2 milestone 1
By the Phase 2 kickoff milestone
Sigmoid
**A reliable sequence number and delivery guarantee on Bronze change records**
Bronze ingestion is the Customer's
Ask at kickoff
Customer ingestion
**Fixing quarantined records at source**
The Customer's responsibility. Source changes are out of scope
Ongoing
SAP data owners
**Duplicate and count monitoring in production**
Phase 3 data quality monitoring
From Wave 1
Sigmoid builds, the Customer runs
### What to watch
Severity
Watch-out
What to do
High
**No reliable sequence number.** If change records carry only a load time, a late change can overwrite a newer one, or a replay can look new.
Ask at kickoff what ordering and commit information Bronze holds, and decide the fallback before the pilot.
High
**Reload timing.** A reload run without a frozen start position can duplicate or miss changes.
Write the reload procedure with the Customer, and rehearse it once in the pilot.
Medium
**Fan-out in Gold.** Consolidating views can multiply rows through a join on a key that is not unique.
Declare the grain for every Gold table and test uniqueness on every refresh.
Medium
**Silent ignoring.** Dropping repeats and stale changes without counting them hides real problems.
Count ignored records per run and alert when the number jumps.
## AI proposes, people decide
Each HANA view goes through six steps. AI drafts; scripts and people check.
### Decide what to migrate first, then translate only what survives
Nothing is translated until a business owner has signed off its treatment.
**1**Extract
**2**Understand
**3**Propose
**4**Decide
**5**Translate
**6**Validate
**7**Publish
‹ Back
Play
Next ›
SAP legacy HANA · 500 models
sunset
SIGMOID · AI-ASSISTED
Reverse-Engineering Agent
extract · parse · map dependencies
Classification Engine
score · propose a treatment
Owner review · human gate
sign off, or overturn
Transpilation Agent
HANA SQL → Databricks SQL
Triage Agent + QA
parity checks · root cause
Documentation Agent
docs · lineage · rationalization log
Databricks · Silver / Gold
RATIONALIZATION LOG
CV_SD_BILLING_ITEMS_OLD · retire · no use
CV_FIN_YEAREND_ACCRUALS · keep · overturned
CV_SD_BILLING_ITEMS_EMEA/_NA · merge
**Migrate as-is**—**
**Redesign**—**
**Merge into another view**—**
**Retire**—**
Carried to Databricks**—**
Not translated**—**
Hollow dots are proposals from the AI. Filled dots are signed off by a business owner. Retired views and views folded into another view never reach the Translate step. A blue ring on a Databricks square means that view absorbed one or more merged views. The step numbers match the cards below.
### How the steps sequence against the programme calendar
PHASE 1 · TO MID-JANPHASE 2 · TO MID-APRPHASE 3 · TO MID-AUGPHASE 4
Proposeextract · understand ·score all 500 models
Decideowner sign-off perpackage, a wave ahead
Translatesurvivors only: as-is,merge, redesign
Validateparity checks, thenUAT sign-off, cutover
Legacy objectsstay live in parallel,readiness certificate
all 500 models
Week 8: inventory after rationalization
pilot
wave 1
wave 2
wave 3
treatment signed off, then build
pilot: The pilot package
wave 1
wave 2 (parallel)
wave 3
pilot UAT
waves parity, UAT
cutovers, hypercare
retired models stay live in legacy, so a wrong decision is reversible
Week 47: readiness certificate
KickoffM3M4M5M6M7M8M9M10M11M12M13
decision gates are proposed
Each diamond is the "treatment path signed off" gate, and a wave's build cannot start before it. The phase dates and the inventory and final cutover milestones are fixed. Wave 1 is Package A, Package B and Shared package. Wave 2 is Package C and Package D, run in parallel with a dedicated spike team of three senior engineers. Wave 3 is the nine low-complexity packages.
### Each HANA view goes through six steps
AI drafts while scripts and people check. Pick a step.
### Which tool for which job
Tools are compared in a pilot bake-off on the same 10 to 15 real views. Pick a job.
## Know what every column means
A data dictionary comes before the data model, and before any translation.
Three sample records. Pick a row to see how its confidence is resolved.
View · column
Source field
Derivation
Draft definition
Report use
Confidence
### How the dictionary is built
AI definitions stay drafts until an owner confirms them. Low-confidence columns never reach Gold.
## Design entities once, not one per report
The unit of work changes from “view” to “target entity”.
### Step 5 · The target data model
Legacy VIEWS TODAY · 7
TARGET ENTITIES IN GOLD · 4
COMPATIBILITY VIEWS · 7
FACT_BILLING_ITEMone row per billing itemfed by 2 views
FACT_AR_ITEMone row per open item
DIM_CUSTOMERconformed, shared by packagesfed by 3 views
DIM_MATERIALconformed, shared by packages
Seven views become four entities. The seven old names stay as views over them, each returning the rows it returned before. The parity gate compares these compatibility views with legacy output.
#### The big picture: how a SAP table becomes a Qlik number
SAP HANAsource tables(the Customer)
Bronzeraw copyno decisions
Silver1 clean table persource entity
Goldbusiness entities
dim_customer
fct_billing_item
Compat viewsold legacy names→ Qlik
Not our scopeThe Customer loadsWe clean: keys, types, dupsWe model: grain, history, ownersQlik needs no edits
Many legacy views collapse into a few Gold entities at the Gold step. The compatibility views then re-expose them under the old names, so the 100% match test can be run on the same reports.
### How we build it, step by step
#### Example 1: four legacy views become three Gold entities
Legacy views (before)Gold entities (after)
CV_BILLING_ITEM_EMEAone row per billing item, company codes 1xxx
CV_BILLING_ITEM_AMERone row per billing item, company codes 2xxx
CV_CUST_ACCOUNT_HEALTHone row per customer per month
CV_CUSTOMER_MASTERone row per customer
fct_billing_itemgrain: billing item. Adds a region column.
fct_account_healthgrain: customer and month
dim_customer (shared)one owner. Used by both facts.
Four views in, three entities out. The two regional views differ only by a filter, so they merge.
Test applied: same grain, same key, same owner, same measures, differ only by company code. All yes, so one entity with a region column. The old names stay alive as compatibility views.
#### One entity, from draft to proof
1 · Claude proposesdictionary entriesviews grouped by grainentity and table drafts
input: view analysis record,
data dictionary
2 · Architects decideSAP and Databricksarchitects review eachentity: accept or change
output: entity mapping matrix,
decision log
3 · Owners signbusiness owner confirmsdefinitions and measuressilence is not consent
output: signed entity mapping
matrix, modelling standards record
4 · Tests checkgrain is uniquebuilt table matchesthe signed modelparity gate
Changes needed: back to step 1 with comments
If a test in step 4 fails, the entity goes back to the architects in step 2, not on to translation.
### The bus matrix sets the build order
Business processes down the side, shared dimensions across the top. Pick a dimension. Illustrative.
### One entity or two?
Six tests, asked of every candidate pair.
### Reference: how SAP’s Customer Data Platform organises its data
#### SAP Customer Data Platform: data, metadata and model layers
SOURCESIDENTITYPROFILE (the data model)OUTPUTS
SAP ERP and CRM
Web and app sign-ups
Other front-end systems
Identity
lite account (email only)↓ later linked toregistered accountidentity resolution joinsrecords to one person(rules not read)
Unified profile record
profile attributes
preferences
custom data fields*
subscriptions*
account links and B2B / B2C views
stored in a fully indexed, dynamic-schema store
Segmentsattributes calculatedon an ongoing basis
Sync downstreamengagement systemsand SAP ERP
Consent layer, applied to every recordterms, privacy policy, third-party processing, marketing · text is versioned
Metadata and auditsearchable audit logs · governance logs · 7-year history in a secure vault
* from memory, notverified in this review
Solid boxes are from SAP's product page and the valantic overview. Dashed boxes are unverified against SAP's Profile Schema documentation. SAP Learning confirms the profile entity with a customer schema, identity resolution through matching and merging rules, and an application library of connectors. Custom data fields, subscriptions and the 7-year vault remain unconfirmed.
### Design choices settled in the standards session
Build
## Tables are rebuilt from the SAP source
Your ingestion lands SAP data in Bronze. We build Silver and Gold on top.
### The data route, from SAP to Qlik
SAP sources
ECC · BW · CRM
Salesforce · Redshift
tables per package
Existing ingestion
Already running,
Customer-managed
CUSTOMER
S3 landing
Raw files as
received
CUSTOMER
Bronze
Raw Delta tables
Full history
SCOPE: CONFIRM
Silver
Cleansed and
deduplicated
SIGMOID
Gold
Data products
and Qlik layer
SIGMOID
1
Source vs Bronze: row counts, key checksums
2
Bronze vs Silver: rule results, dedupe counts
3
Gold vs legacy output, by region and period
Quality failures in Silver go back to the SAP owners
and return through CDC once fixed at source.
Boxes marked Customer are already in place or owned by them. Bronze is existing and the Customer's to maintain, including its data quality validation. The numbered checkpoints separate a load problem from a logic problem.
### What we confirm about Bronze first
Each open item is a question for the kickoff call or the first two weeks.
## Replays can’t create duplicates
Every change carries a key and a sequence. Only newer changes apply.
Send changes for customer 100999 into Silver and watch what sticks.
New change
Same file again
Old change arrives late
Delete in SAP
Start again
Silver table
Key
Country
Sequence
Status
Merging twice equals merging once. Deletes stay as markers, so a late update can’t bring a row back.
### The eight protections
## Bad records are held, never hidden
Failures wait in quarantine and go back to SAP to be fixed at source.
**1**Set rules
**2**Run rules
**3**Quarantine
**4**Route to owners
**5**Fix and return
**6**Five checks
**7**Triage
**8**Sign-off
‹ Back
Play
Next ›
DQ QUARANTINE · RUNS ON EVERY SILVER LOAD
SAP sourcesreplicated byQlik Replicate
Bronzeraw, full history
SILVER · DQ RULES
completenessuniquenessreferentialbusiness rules
Goldclean recordsonly
DQ QUARANTINE TABLE
rule · severity · record keys · owner · age
SAP ownerscorrect the recordin the sourceCUSTOMER
DQ monitoring and alertsfail rate by rule · open itemsage of items · SLA breaches
pass
fail
ticket with keys
fix in SAP
PARITY GATE · PER PACKAGE, BEFORE CUTOVER
Gold + Qlik layerfrom Databricks
Legacy outputstill live
Triage Agentroot cause and suggested fix
mismatch
fix, re-run
Business UATsign-off
Go / cutoverper package
Rollback criteriaagreed before cutover
Dots are records. Red dots fail a rule and are held, amber dots are waiting for the SAP owner, green dots have passed. The gate compares whole packages, not single records. The step numbers match the cards below.
### The DQ quarantine, step by step
### Load defect or source defect?
A record fails a Silver rule. One question decides who fixes it.
Does Bronze match SAP for this key?
YesNo
### Rule types and starting severities
Starting values, agreed with the SAP owners. A rule feeding a regulated or safety view starts at High.
### The life of a quarantine item
Prove
## Nothing switches until **five checks pass**
Scripts compare Databricks with HANA. Flip a check to see the effect.
Go
### Freeze the as-of date
Fixes in SAP change HANA’s output too. Both sides are compared at one agreed date, so the target stops moving during a run.
### Test a period-end
At least one close per package, so year-end and audit views are covered before anyone signs.
### Quarantine is reported, not hidden
HANA still holds the bad records, so Gold differs by exactly the quarantined items. They’re listed as “explained by quarantine” until released or waived.
## Cut over with a way back
Package by package, away from close, with rollback rehearsed first.
Plan
## About eleven months, in four phases
The shape of a 500-model, 15-package programme.
Your own phases are sized in the first twelve weeks.
M1
M2
M3
M4
M5
M6
M7
M8
M9
M10
M11
M12
M13
1 · Assess & Designto week 12
2 · Pilot migrationto week 25
3 · Scaled migrationto week 42
4 · Cutover & transitionto week 51
WavesPhase 3 packages
W1W2W3
Legacy + Databricks in parallel from the pilot (both platforms paid for by the Customer)
1
2
3
4
5
6
7
8
9
10
11
12
Week 1
Replay the timeline
**Week 51:** hypercare closed, handover.
The wave labels in milestones 7 to 10 do not match section 4.3, see the table below. Extended support is optional and not included.
### What each phase does
HANA and Databricks run in parallel from the pilot until each package signs off
###
Phase gate
Milestones
### Phase 1: eight deliverables and a modelling baseline
Each has an owner, a signer and a done-when test. Phases 1 and 2 also run 42 workshops, package by package, with each package owner in the room.
## Ready to kick off?
Anything not ready on day one comes straight out of Phase 1.
Prerequisites in place
0 of 8
### If a prerequisite is late
Late inputs extend the schedule day for day. Each has a fallback.
## Estimate by complexity class
Change the inputs to see what drives effort and duration.
### Three passes, each narrower than the last
Before assessmentAfter inventory and classificationAfter the pilotEach wave
Rough order of magnitude: widest rangeBaseline: range narrows once objects are countedCalibrated: measured effort replaces assumptionsRe-forecast
The shape is illustrative. The width of each range depends on how much is known, not on a fixed percentage.
**0**total hours
**0**person-days
**0**months at this team size
**0%**of build hours from High and Very High objects
**Build, net of reuse
**Package overhead
**Fixed work
**Contingency
Illustrative starting values: 6, 14, 32 and 64 hours per Low, Medium, High and Very High object, 40 hours per package and 3,000 hours of fixed work. The pilot replaces them with measured values. AI savings count only after the pilot proves them.
## Every gate names its evidence
Anyone can replay a pass or fail from what’s stored.
## Judgement calls, made repeatable
Decision tables for the questions that come up on every package.
Control
## Keep the programme under control
Risks, traps, security, cost and change, in one place.
Run
## It ends when someone else can run it
Who runs each activity after go-live, and what they receive.
Handover criteria
Runbooks accepted, support owners named for every package, DQ ownership transferred, monitoring live, and the evidence pack archived.
Glossary
## The terms, in plain words
Pick a term to see what it means here.
## We’ve done this before
A packaged-foods company moved its SAP analytics to Databricks with Sigmoid, using parallel validation against the old platform and quarantine for bad records.
5×
lower total cost of ownership
99%
data trust
### Rebuild the logic, not the tables
The hard part lives in the calculation views: currency and unit conversion, hierarchies, fiscal calendars. We build a tested pattern library for these first.
### Check parity every day, not once
Reconciliation runs daily through the parallel period. Every exception gets an owner and a reason.
### Pick a pilot that hurts
A pilot on an easy package produces templates that break later. Ours includes the multi-currency, high-volume models.
### Understand consumption early
Downstream apps decide table shapes. Invest early in CI/CD and delivery templates, and validate with the business in every phase.
## About the Author
**Neehal Lobo** is Director of Delivery Management for Data and AI at Sigmoid. He has 26 years of experience in data and AI consulting, delivery governance, pre-sales and building technology practices, with deep expertise across Azure, AWS, GCP, Databricks and Snowflake. Over the course of his career, he has worked with organizations across healthcare and life sciences, manufacturing, insurance and other industries to build data and AI capabilities and scale high-performing teams. In his current role, Neehal leads Sigmoid’s data and AI initiatives and projects, driving the shift from traditional analytics and data engineering to agentic AI and GenAI-native delivery for its Fortune 500 clients and helping enterprises unlock greater value from data, AI and modern technology platforms.
## Planning a move off SAP HANA?
Tell us about your estate. We’ll show you how the sort, the gates and the parallel run would look for it.
[Talk to Sigmoid](https://www.sigmoid.com)
---
## Navigation
- [Company](/about-sigmoid)
- [Newsroom](/newsroom)
- [Life at Sigmoid](/careers)
- [Takshashila](/takshashila)
- [Contact Us](/contact-us)
- [AI Strategy Blueprint your AI advantage](/enterprise-ai-strategy/)
- [Generative AI Drive innovation with Generative AI](/generative-ai/)
- [Responsible AI Build trust with ethical AI practices](/responsible-ai-in-enterprise/)
- [Agentic AI Reshape business with scalable agentic systems](/agentic-ai-solutions/)
- [AI Managed Services Ensure reliable AI performance](/ai-managed-services/)
- [Advanced Analytics Transform your business with data-driven insights](/advanced-data-analytics-solutions/)
- [Download whitepaper](/ebooks-whitepapers/the-new-qsr-playbook-transforming-menu-strategy-into-a-growth-engine/)
- [Data Strategy Strong data foundations for scalable AI](/data-analytics-strategy/)
- [Data Management Leverage data as a strategic asset](/ai-data-management-services/)
- [DataOps Automate data for speed and quality](/data-devops/)
- [Forward Deployed Engineering Scale enterprise AI with FDEs & specialized agents](/forward-deployed-engineering/)
- [Data Engineering Deliver insights faster with scalable pipelines](/data-engineering/)
- [Cloud Transformation Modernize data to maximise efficiency](/cloud-migration/)
- [Download whitepaper](/ebooks-whitepapers/building-data-products-in-a-data-mesh-to-drive-business-value/)
- [Data Modeling Structure data for better decisions](/data-modeling-services/)
- [Data Visualization Transform data into actionable stories](/data-visualization-service/)
- [BI Migration Enhance decision making with modern BI tools](/bi-migration/)
- [Data Observability Build trust with healthy, accurate data](/data-observability/)
- [Automated Insights Make smarter decisions with auto-generated insights](/automated-insights/)
- [Download whitepaper](/ebooks-whitepapers/how-cognitive-bi-drives-an-intelligent-enterprise/)
- [CPG & Retail End-to-end analytics for planning, operations, and commercial excellence](/industries/cpg-analytics/)
- [Life Sciences Trusted intelligence across clinical, commercial, and operational workflows](/industries/life-sciences/)
- [Banking & Financial Services AI-powered analytics for risk, compliance and customer experience](/industries/banking-financial-analytics-services/)
- [Insurance Modernize core insurance platforms and unlock new revenue streams with AI](/industries/insurance-reimagined-with-ai/)
- [Download infographic](/wp-content/uploads/2026/03/Infographic_Maturity-Framework-for-Scaling-Agentic-AI.pdf)
- [Helios Turn marketing data into measurable business outcomes](/marketing-suite-helios/)
- [Stratos AI-powered platform for resilient supply chains](/supply-chain-stratos/)
- [DataGuard Agentic AI platform for data quality and governance](/data-quality-dataguard/)
- [MediaIQ Advanced platform for in-flight marketing measurement](/accelerators/sigmoid-mediaiq-multi-touch-attribution-tool/)
- [CampaignIQ AI-driven platform for optimized campaign budget allocation](/accelerators/sigmoid-campaigniq/)
- [AssistBot GenAI email assistant that automates human-like responses](/accelerators/sigmoid-assistbot-for-ai-email-assistant/)
- [CreativeBot GenAI tool for personalized and brand-aligned creative design](/accelerators/sigmoid-creativebot/)
- [SocialBot GenAI platform to analyze digital conversations and trends](/accelerators/#marketing|socialbot)
- [DemandIQ Predict trends accurately and optimize inventory management](/accelerators/sigmoid-demandiq/)
- [NetworkIQ Track and optimize logistics operations in real-time to quickly address disruptions](/accelerators/sigmoid-networkiq/)
- [SupplyIQ End-to-end platform to optimize supply chain operations](/accelerators/sigmoid-supplyiq/)
- [ProcurementIQ Automated procurement operations for maximum savings, compliance and efficiency](/accelerators/sigmoid-procurementiq/)
- [RapidML Accelerated deployment for machine learning models](/accelerators/sigmoid-rapidml/)
- [DataGuard Comprehensive platform for proactive data quality management](/accelerators/data-quality-tool-sigmoid-dataguard/)
- [CloudPulse Cloud cost optimization platform with multi-cloud management](/accelerators/sigmoid-cloudpulse/)
- [RAPID GenAI foundation with built-in governance and cost clarity](/accelerators/sigmoid-rapid/)
- [AnalyticsBot Conversational analytics for faster business decisions](/accelerators/sigmoid-analyticsbot/)
- [DataConnect Seamlessly ingest, integrate and harmonize data from diverse sources](/accelerators/sigmoid-dataconnect/)
- [Reconica AI-powered data harmonization and reconciliation engine](/accelerators/sigmoid-reconica/)
- [ConverseBot GenAI driven insights generation for automated insights from reports](/accelerators/#sales|conversebot)
- [iNRM Cross-lever revenue growth optimization platform](/accelerators/sigmoid-inrm/)
- [AssortmentIQ Optimize shelf layouts and assortment mix at scale with AI-based insights](/accelerators/sigmoid-assortmentiq/)
- [Read whitepaper](/ebooks-whitepapers/building-agentic-ai-chatbots-for-business-process-transformation/)
- [Listen Podcast](/events/podcast/how-jack-in-the-box-is-redefining-personalization-and-supply-chain-with-ai/)
- [Blogs](/blogs/)
- [White Papers](/ebooks-whitepapers/)
- [Case Studies](/case-studies/)
- [Podcast](/events/podcast/#Podcasts)
- [Download case study](/case-studies/ai-based-dynamic-targeting-and-personalized-hcp-engagement-for-pharma-marketing/)
- [ConverseBot](/accelerators/#sales|conversebot/)
---
## Footer Links
- [Talk to our AI experts](/contact-us/)
- [AI Strategy](/enterprise-ai-strategy/)
- [Agentic AI](/agentic-ai-solutions/)
- [Generative AI](/generative-ai/)
- [AI Managed Services](/ai-managed-services/)
- [Responsible AI](/responsible-ai-in-enterprise/)
- [Advanced Analytics](/advanced-data-analytics-solutions/)
- [Data Strategy](/data-analytics-strategy//)
- [Data Engineering](/data-engineering/)
- [Data Management](/ai-data-management-services/)
- [Cloud Transformation](/cloud-transformation/)
- [DataOps](/data-devops/)
- [Forward Deployed Engineering](/forward-deployed-engineering/)
- [Data Visualization](/data-visualization-service/)
- [Automated Insights](/automated-insights/)
- [BI Migration](/bi-migration/)
- [Data Modeling](/data-modeling-services/)
- [Data Observability](/data-observability/)
- [CPG & Retail](/industries/cpg-analytics/)
- [Life Sciences](/industries/life-sciences/)
- [Banking & Financial Services](/industries/banking-financial-analytics-services/)
- [Insurance](/industries/insurance-reimagined-with-ai/)
- [Case Studies](/case-studies/)
- [Thought Leadership](/ebooks-whitepapers/)
- [Blogs](/blogs/)
- [Helios](/marketing-suite-helios/)
- [Stratos](/supply-chain-stratos/)
- [DataGuard](/data-quality-dataguard/)
- [Company](/about-sigmoid/)
- [Newsroom](/newsroom/)
- [Accelerators](/accelerators/)
- [Careers](/careers/)
- [Privacy Policy |](/privacy-policy/)
- [Cookie Policy](/cookie-policy/)