AI org assessment: technical debt before a partner change

An AI org assessment is worth doing before a relaunch or partner change, as long as the model reads and the tools count. An LLM explains what each Apex class, flow and permission set does, which business process it serves and where partners built the same logic twice. In the worked example below, a batch run over 2,250 components costs about $59 to $119. Usage, rule violations and test coverage must come from deterministic tools: Code Analyzer, sfdx-hardis monitoring and an Apex test run. I run those first and give their results to the model.

A grown org carries the handwriting of every partner

A Salesforce org that has grown for ten years under several partners carries their handwriting. At a machine builder, one partner may have built the dealer portal, another the field service processes, a third the SAP integration for orders and spare parts. Each brought its own trigger framework, naming and test style. Before a new partner takes over or a relaunch starts, someone has to answer three questions: what is in the org, what of it is used, and what of it is risky to keep.

The two obvious candidates for that answer are poor ones. The outgoing partner defends its code. The incoming partner has an interest in finding a lot to rebuild. An assessment run by the client, with the client’s tools, is the neutral baseline, and it later feeds the object ownership matrix for the new set-up.

What can an AI org assessment do that scanners cannot?

A scanner checks rules. It finds a query inside a loop or a class without a sharing keyword. It cannot tell you that two classes from two partners both calculate warranty end dates, with different rules for dealers in different countries. That is a question of meaning, and here an LLM is useful. Per component, I would ask the model for:

• Purpose in two sentences, in business terms a head of service understands.
• Business capability, for example warranty, spare parts, service contracts, dealer orders or quoting.
• Objects touched, so components can later be grouped by object.
• Duplicated meaning, logic that does the same as another component even when the code looks different.
• Test quality, whether a test class checks business results or only raises coverage.
• A recommendation: keep, refactor or replace with standard functionality.

Every finding must point to a method name or line number. I would drop every AI finding that cannot show where it comes from, because a model describes wrong things as confidently as right ones.

Which findings must come from deterministic tools?

Anything that is a count, a usage figure or a rule violation. A model reading one class at a time cannot know whether that class is called anywhere, or prove that something is absent. These tools can:

• Salesforce Code Analyzer1 bundles eight engines, among them PMD for Apex, CPD, which “identifies blocks of duplication across files”, a Flow Scanner for security issues in flows and the Salesforce Graph Engine for deeper Apex analysis.
• ApexGuru is “available only in Production and Full Copy Sandbox orgs” and rates severity with runtime data, according to the Salesforce Developers Blog2. For an assessment against production that is exactly what you want: performance findings ranked by real CPU use.
• sfdx-hardis monitoring runs, according to its monitoring overview3, checks such as “Detect unused Apex classes in an org”, “Detect Apex classes and triggers with deprecated API version”, “Detect underused permission sets”, “Detect inactive metadata”, active users without logins in six months and the Salesforce Security Health Check.
• An Apex test run gives the real coverage per class, which no model should estimate.
• Dependencies come from the Tooling API object MetadataComponentDependency4, which Salesforce still labels Beta. Who built what comes from Git history or the LastModifiedBy fields, not from the model’s guess about coding style.

My rule: if a sentence in the report contains a number or the word “unused”, a tool produced it. I wrote more about these tools as pipeline gates in quality gates for AI code; for an assessment you run them once against the whole org.

Worked example: three partners, one org, one relaunch

Assume a machine builder with Sales Cloud, Service Cloud, Field Service, a dealer portal and SAP. Over twelve years three partners have worked on the org, and a fourth is about to take over. The org holds 1,400 Apex classes and triggers, 600 flows and 250 permission sets. All volumes and token sizes below are assumptions.

The order of work:

1. 01

   Retrieve metadata

   Retrieve all metadata into a repository the client owns.

2. 02

   Run the tools

   Run Code Analyzer, sfdx-hardis monitoring and a full Apex test run.

3. 03

   Clean the data

   Remove hard-coded personal data from test classes with a pattern search.

4. 04

   Let the model read

   Send each component to the model, together with the assessment questions and the tool findings for that component.

5. 05

   Condense per object

   Group the answers by object and run a second pass per object, because a trigger, two flows and a validation rule on Work Order only make sense together.

What the batch run costs

The Claude Batch API5 processes requests asynchronously, most batches within an hour, and charges 50 percent of standard prices. According to the pricing page6, that means $1 input and $5 output per million tokens for Sonnet 5.5, and $2 and $10 for Opus 5.5. A batch holds up to 100,000 requests or 256 MB, whichever comes first. The scenario’s input is roughly 150 MB of text (my estimate), so I would split it into two batches to stay clear of the limit.

• Code and metadata: 1,400 classes at 6,000 tokens, 600 flows at 10,000 tokens of flow XML, 250 permission sets at 4,000 tokens = 15.4 million tokens.
• Questions and tool findings: 12,000 tokens per request × 2,250 requests = 27 million tokens.
• Output: 1,500 tokens per component = 3.4 million tokens.

$59

Sonnet 5.5, first pass: 42.4 million × $1 + 3.4 million × $5, rounded

$119

Opus 5.5, first pass: 42.4 million × $2 + 3.4 million × $10, rounded

$10

Second pass for 40 objects at 100,000 tokens in and 5,000 out on Opus 5.5

Prompt caching stacks with the batch discount, but in batches cache hits are best effort; Anthropic names hit rates from 30 to 98 percent. I calculate without caching and treat any saving as a bonus. Thinking tokens are billed as output7 and cannot be turned off on the 5.5 models; if they triple the output, the Opus pass rises to about $187. The basics of this token math are in the masterclass chapter on costs and tokens.

Where the real cost sits

The token bill is small; the work sits with people. In the worked example I would plan about ten architect days: three or four to set up the tools and write the questions, the rest to check a sample of 10 percent of the findings and every finding marked high risk. Reading 2,250 components by hand at an assumed ten minutes each would take about 375 hours, or 47 days. The LLM does the first read, and people decide.

What about source code and data protection?

The org’s source code belongs to the client, and test classes often contain names, email addresses or record IDs. Anthropic’s data retention page8 lists batch processing as not eligible for zero data retention, because results are stored for 29 days. The same page says retained data is “never used for model training without your express permission”. Strip personal data before the run and agree the set-up with your data protection officer; the German angle is in AI coding tools, works council and GDPR.

If the zero-retention rule is strict, there are two alternatives. You pay full price without the batch, about $119 to $238 in the worked example, after checking which endpoints your contract covers. Or you use the AI documentation in sfdx-hardis, which according to its AI setup page9 works with Agentforce, OpenAI or, through LangChain.js, Anthropic, Google GenAI and Ollama for locally run models. The same page warns that the first documentation run “can make hundreds of API calls” and recommends committing the AI cache, and it caps calls per command at 10,000 by default.

Who should own the assessment?

The client. Run the scans in your pipeline and the model calls in your own API account, and keep the results in your repository, as I argue for DevOps tools in why clients should own their DevOps tools. If a partner does the work, its contract should name the tools and data rules, see partners coding with AI: what belongs in the contract.

The assessment is the starting point for the handover: which objects move to which partner, which components get deleted and which are rebuilt. Steering that is part of partner governance.

Retrieve all metadata into a repository you own

1. Retrieve the full metadata into a repository you own.
2. Turn on sfdx-hardis monitoring for two weeks to collect usage data.
3. Run Code Analyzer and a full Apex test run against production metadata.
4. Write ten assessment questions and test them on 50 components, twice; if the two runs disagree on many risk ratings, sharpen the questions.
5. Run the batch, then the second pass per object.
6. Check samples and every high-risk finding before anything goes into the handover plan.

Sources9

1. 1Salesforce Code Analyzer (opens in a new tab)developer.salesforce.com
2. 2Salesforce Developers Blog (opens in a new tab)developer.salesforce.com
3. 3monitoring overview (opens in a new tab)sfdx-hardis.cloudity.com
4. 4MetadataComponentDependency (opens in a new tab)developer.salesforce.com
5. 5Claude Batch API (opens in a new tab)platform.claude.com
6. 6pricing page (opens in a new tab)platform.claude.com
7. 7billed as output (opens in a new tab)code.claude.com
8. 8data retention page (opens in a new tab)platform.claude.com
9. 9AI setup page (opens in a new tab)sfdx-hardis.cloudity.com