Data and configuration in the Salesforce pipeline

Data and configuration that is not metadata slips through many Salesforce pipelines: Field Service scheduling policies and work rules, operating hours, custom settings records, products and price books. A metadata deployment does not carry them, so they end up as manual release steps. My rule: configuration data lives in the repository and runs through the same pipeline after the metadata deployment, with SFDMU or the data module of a commercial tool. Master data from SAP arrives through the integration, test data as a seed set. Customer data in partial and full sandboxes is masked after every refresh, for example with Salesforce Data Mask & Seed.

Which data counts as configuration, and which does not?

Chapter 6 covered how metadata reaches production with checks. A Salesforce org also depends on records that drive behaviour. I split them into four groups, because each one needs a different route:

Group	Manufacturing examples	Source of truth	Route into the orgs
Configuration data	scheduling policies, work rules, service objectives, operating hours, custom settings values, mapping tables in custom objects	repository	pipeline, after the metadata deployment
Master data	products, price books, material numbers, dealer master	SAP or another leading system	integration in production, seed set in sandboxes
Test data	30 dealers, 200 assets, edge cases from service	repository	seeded into scratch orgs and developer sandboxes
Customer data	cases, work orders, contracts, warranty claims	production	copied into partial and full, then masked

The Field Service examples are real. In the Trailhead module on scheduling policies (opens in a new tab), you create work rules from their own tab and pick a record type; the policy brings work rules and service objectives together for the scheduling engine. Operating hours, according to the object reference (opens in a new tab), are the OperatingHours object, which sets when a service territory, a resource or an account is available for work. If someone changes a work rule in the integration sandbox so that technicians only get assets that match their skills, that change is a record. No metadata deployment carries it forward.

The most important line runs between configuration data and master data. A price book, per the object reference (opens in a new tab), holds the list of products an org sells. If that list comes from SAP, a deployment from a sandbox to production would be a second, competing route. That is why I never let master data run through the release pipeline into production.

Custom metadata types, custom settings and where deployment stops

For new configuration, one simple decision saves a lot of data work. Trailhead’s module on custom metadata types (opens in a new tab) states the difference clearly: custom metadata type records can be packaged and deployed through the Metadata API or change sets. Custom settings records cannot be deployed by any of those routes.

For a manufacturer, that means:

• Custom metadata types for configuration that is versioned in the repository and rolled out with the release: mapping product groups to service queues, thresholds for warranty checks, switches for integration endpoints. They go through the normal pull request with review. In delta deployments with dependencies, still in beta according to its documentation (opens in a new tab), sfdx-hardis deploys all records of a custom metadata type as soon as one of them has changed.
• Custom settings only where the business case needs them, for example values per user or profile. Their records then need a data step in the pipeline.
• Custom and standard objects used as configuration, such as the Field Service objects or pricing rules in a configurator. Here you have no choice; they are data.

Configuration data runs through the same pipeline

External IDs instead of Salesforce IDs. A work rule has a different Salesforce ID in every org. A data step has to recognise the record by a business key, or it creates duplicates on the second run. I give every configuration object a field marked as external ID, holding a readable code such as WR_SKILL_HYDRAULICS.

The data lives in the repository. Then the same rules apply as for code: pull request, review, history. A changed threshold shows up in the diff, and nobody has to guess who changed it and when.

Metadata first, then data. A record can only be loaded once the field it writes to exists. So the data step runs after the metadata deployment, and only if that succeeded.

SFDMU and sfdx-hardis

SFDMU (opens in a new tab) is an open-source Salesforce CLI plugin for data migration, licensed under BSD-3 according to its npm entry (opens in a new tab). It copies records directly from org to org or from CSV files, supports insert, upsert, update and delete, handles several related objects in one run and is configured through a single export.json file. Field mapping, anonymisation and composite external ID keys come on top. Processing runs locally or on the pipeline runner, with no cloud service in between.

sfdx-hardis runs SFDMU through deployment actions (opens in a new tab). How deployment actions fit into the pull request flow is covered in chapter 5. In the free sfdx-hardis training, deployment actions for reference data are part of level 2; my notes are in what I learned from the free sfdx-hardis training, and an introduction is in sfdx-hardis: open-source Salesforce DevOps explained.

For developers

How do deployment actions of type data run?

An action of type data loads records from one of the project's SFDMU workspaces into the target org. Actions are stored in config/.sfdx-hardis.yml or in a file per pull request and run once per org by default.

They are skipped when the metadata deployment failed. A failed action runs again with the next deployment that carries its pull request.

For small seed sets the Salesforce CLI itself is often enough. Salesforce recommends the data tree commands (opens in a new tab) for small sets, such as fewer than 3,000 records, and the data bulk commands for large volumes.

What do Gearset, Copado and Flosum offer for data?

The commercial platforms have their own data modules with a user interface. According to the vendors’ documentation:

Tool	What it does with data	Keep in mind
Gearset data deployments (opens in a new tab)	dependencies up to 15 levels deep, matching on external ID, masking with fake values, flows, triggers and validation rules can be switched off during the run, templates (opens in a new tab), CPQ and Vlocity configuration (opens in a new tab)	part of the “Sandbox seeding & compliance” add-on, not the deployment licence; per Gearset (opens in a new tab) no rollback, no deletion, production disabled by default
Copado Data Deploy (opens in a new tab)	data templates for configuration records with complex relationships, ready-made templates from the DevOps Exchange, masking of sensitive values	in Copado Essentials, per its documentation (opens in a new tab), an Essentials Plus option with a per-user licence
Flosum Data Migrator (opens in a new tab)	templates with objects, fields and filters, relationships preserved, masking, triggers can be switched off	error report per failed record

What these tools do well: an admin sees before the run which related records will go along and does not have to write an export.json. For CPQ configuration with many nested objects, that is a real advantage. Which tool fits your programme overall is decided in chapter 4; for data the same rule applies as for metadata. The tool runs in your name, and the data templates sit with you, not in a partner’s account. The reasons are in why clients should own their DevOps tools.

One point applies to every tool:

How does customer data in sandboxes stay protected?

Partial and full sandboxes hold real customer data after every refresh. Which environment may carry which data was set in chapter 3; which fields are sensitive at a manufacturer and what coding agents may read is covered in sandbox customer data: what AI coding agents may read.

Salesforce sells Data Mask & Seed (opens in a new tab) for this. According to Trailhead, Data Mask (opens in a new tab) is a managed package you install in production that masks data in full and partial sandboxes; production stays unchanged, and masked data cannot be unmasked. Per field, the configuration module (opens in a new tab) lets you choose random characters or numbers, a replacement from a library such as names or addresses, or deletion. Checkbox, lookup and picklist fields are not supported. The configuration carries over on refresh, and you start the run in the sandbox.

The seed part, according to the product page, fills developer, developer pro, partial and full sandboxes with records filtered by seed templates. Mask and seed jobs can be scheduled daily, weekly or monthly, and seeding bypasses automations. The page lists no price. Gearset, Copado and Flosum also mask data in transit, and SFDMU can anonymise. The difference: Data Mask works on an existing sandbox copy, the others mask data while they move it.

My order after every refresh of a partial or full sandbox:

1. 01

   Start the mask

   Start the run and check the execution log.

2. 02

   Add new fields

   Add fields created since the last run to the mask; this check sits in the release checklist.

3. 03

   Mask email fields

   Make sure email fields on contacts are on the mask list, so no dealer receives a test notification.

4. 04

   Load configuration

   Load configuration data from the repository if the sandbox needs a newer state than production.

5. 05

   Open access

   Only then open access for partners and business users.

Coding agents get a stricter rule, described in chapter 8 of the AI masterclass: they work in scratch orgs and developer sandboxes with seed data, never in a full copy.

Worked example: Field Service for maintenance

Assume a machine builder introducing Field Service for maintenance and repairs at customer sites. Two partners work on the same org, and SAP supplies products, material numbers and prices. All numbers are assumptions.

Without a data step in the pipeline, go-live looks like this: the metadata deploys cleanly, then a consultant spends two days re-entering 6 scheduling policies, 25 work rules, 40 operating hours records and 15 service territories by hand in production, working from a spreadsheet taken from the UAT sandbox. Afterwards three rules differ from the tested state, which only shows when technicians get jobs outside their territory.

With a data step, the same go-live looks different:

• Every configuration object has a field marked as external ID, and the records sit as an SFDMU workspace in the manufacturer’s repository.
• Changes to work rules go through a pull request reviewed by the service process owner.
• After the metadata deployment, a deployment action loads the configuration into integration, UAT, staging and production, with the same state each time.
• In production, products and price books come from SAP; sandboxes get a seed set with 300 parts and two price books.
• The UAT sandbox, a partial copy, is masked after every refresh before the partner gets access.

On Gearset, Copado or Flosum the structure would be the same, with a data template instead of export.json. What counts is that no manual data step is left in the release plan.

These rules belong in the partner contract

I would write these into every contract:

1. Configuration data is never edited directly in a stage org, only in the repository.
2. Every configuration object has an external ID.
3. New configuration is built as a custom metadata type unless there is a business reason against it.
4. Master data from SAP never runs through the release pipeline into production.
5. No partner gets access to a sandbox before masking has run after the refresh.

When to schedule data steps in the release calendar, and how the three Salesforce releases a year fit in, is covered in chapter 8. Data steps and sandbox levels are part of my work in DevOps and release management.