When Your Machine-Learned Potential Breaks Down: Is It the Training Set Diversity or the Descriptor Complexity?

You train a machine-learned potential on a handful of binary alloys. It nails the energy and forces—good. Then you try a ternary, maybe a quaternary, and suddenly the forces are off by 30%.

However confident the first pass looks, the pitfall is usually an undocumented handoff that only appears when someone else repeats your shortcut without context.

The potential doesn't crash—it just lies quietly. You check the loss curve: fine.

Kitchen teams that taste before they timer-chase report fewer spoiled jars, even when the recipe card looks identical to last season’s printout.

Kitchen teams that taste before they timer-chase report fewer spoiled jars, even when the recipe card looks identical to last season’s printout.

You check test set: fine. But the material you're simulating is wrong.

Two questions come to mind: Is my training set diverse enough? Or is my descriptor too simple to capture the chemistry? Both sound plausible. But they lead to very different fixes—one means collecting more data, the other means redesigning your feature space. This article helps you tell them apart, using examples from high-entropy alloy design where MLIPs fail and what people do about it.

Not always true here.

Where the Problem Shows Up in Real Work

High-entropy alloy simulation with MLIP

You trained a machine-learned interatomic potential on binary Ni-Co data. The forces match DFT within 2 meV/Ǻ. The energy-volume curve looks clean. Then you run a five-element high-entropy alloy — NiCoCrFeMn — and the simulation blows apart at 2 picoseconds. Atoms drift. Energies spike. The MD box becomes unphysical. That moment of failure is hard to read: did your training set simply miss the chemical diversity, or did your descriptor run out of expressive room?

The diagnostic starts with the numbers. Compute per-atom forces on a few random snapshots from the failed trajectory. Compare them against single-point DFT. If force errors stay low (under 5 meV/Ǻ) but the system still drifts, the problem likely sits in the energy surface. Wrong order. Your descriptor can resolve local chemistry but the global potential-energy landscape has holes. If force errors jump to 15-20 meV/Ǻ, the descriptor itself is failing — it can't capture the bond-length distributions or charge transfer that appear only in the quinary system.

I have watched teams spend weeks adding ternary structures to the training set, only to see the same collapse. The catch is that the descriptor — often a set of atom-centered symmetry functions — saturates beyond a certain coordination variety. Adding more configurations does nothing. You're expanding the wrong dimension. What usually breaks first is the angular term: the potential can't tell apart a Cr-centered octahedron from an Mn-centered trigonal prism because the function basis was optimized for binary environments.

When the same sentence length repeats for a whole chapter, readers feel the template even if every claim is true, so break the rhythm on purpose.

Try this instead. Run a single-point energy evaluation on 50 random quinary configurations. Plot predicted energy vs. DFT energy. The scatter tells you: if the slope is off but the rank order holds, the potential has learned the right shape but wrong reference. That's a training-set bias fix. If the scatter is pure noise — points everywhere — the descriptor lacks the necessary degrees of freedom. Swap to a message-passing architecture or a higher-order angular expansion. Not yet sure which? Check the force-error histogram. A bimodal distribution usually means the potential handles some elements well and others poorly — training-set fix. A single broad peak means the descriptor can't resolve any environment properly.

Failure case: binary-trained potential on ternary system

Here is a pattern I see repeatedly. A group publishes a beautiful potential for Ni-Al. Works on Ni3Al. Works on NiAl B2. Then someone tries Ni-Al-Cr, and the ternary ordered phase — say, a Laves-type C14 — produces energy predictions 0.3 eV/atom off. The immediate instinct: collect more ternary training data. That helps, sometimes. But the ugly truth is that many standard descriptors embed only pair and three-body terms, and the C14 phase stabilizes through four-body packing effects that the potential literally can't encode. Adding data doesn't add descriptive capacity.

The trade-off is painful. Increase descriptor complexity — more radial basis functions, higher angular momenta — and you inflate training cost and risk overfitting to the binary data you already have. Keep it simple and you accept that certain phases are out of reach. Most teams skip this diagnostic step: they retrain, fail, retrain with more data, fail again, then declare machine-learned potentials unreliable. The descriptor was the bottleneck all along.

Nebari jin moss stalls.

A quick sanity check. Take your ternary configuration, compute the Behler-Parrinello symmetry functions or SOAP vectors for every atom. Run a principal component analysis. If the ternary environments fall well inside the convex hull of the binary training set, the diversity is fine and the descriptor is the culprit. If they fall outside — new coordination numbers, new bond-angle combinations — your training set is the weak link.

Diagnostic: energy vs. force errors

Energy errors and force errors are not the same beast. A potential can nail energies to 1 meV/atom while predicting forces that are pure noise. That sounds like a failure, but it actually points to a specific problem: your loss function weighted energy too heavily during training, and the optimizer found a potential that matches total energies but produces unphysical gradients. The fix is not more data or a fancier descriptor — it's rebalancing the loss weights and retraining the same architecture.

“We saw force RMSE of 120 meV/Ǻ with energy RMSE of 0.8 meV/atom. Everyone wanted to add more training data. Turned out we just needed to crank the force weight from 0.1 to 0.5.”

— overheard at a workshop breakout, materials informatics group

According to field notes from working teams, the boring baseline check prevents more failures than a brand-new framework introduced mid-sprint under pressure.

So before you blame your training set or your descriptor, check the loss breakdown. The reverse also happens: low force errors with high energy errors typically means the potential is fitting local gradients well but misses global thermodynamic stability — a training-set diversity problem, because the configurations in your set all have similar energy-volume slopes. Force errors tell you about the local landscape; energy errors tell you about the basin. One misdiagnosis can cost you weeks.

Two Things People Get Wrong

Confusing coverage with complexity

The most expensive mistake I see teams make is treating the training-set density as if it were the same thing as descriptor expressiveness. They're not substitutes. A dataset packed with 200,000 configurations—all of them near equilibrium, all of them for the same binary alloy—still fails when a shear event pushes a grain boundary into a metastable stacking fault. That failure isn't a quantity problem. It's a coverage problem. You have many snapshots, but they live in a narrow pocket of phase space. The descriptor, however fancy, can't extrapolate what it never saw. The catch is: that same descriptor, fed a sparser but more diverse set—configurations from melt-quench runs, strained slabs, surfaces with adsorbates—will interpolate between those islands of data and surprise you with decent predictions. Diversity trumps volume. Most people get the order wrong.

Richer descriptors shrink the data needed. But they also shrink what you can trust.

Skip that step once.

Odd bit about science: the dull step fails first.

Odd bit about science: the dull step fails first.

Odd bit about science: the dull step fails first.

Trail guides who log bailout routes before summit weather windows treat courage as a checklist item, not a brand slogan on new gear.

Odd bit about science: the dull step fails first.

Odd bit about science: the dull step fails first.

Believing more data always helps

I have watched a group add 50,000 ab-initio points to fix a brittle-fracture prediction, only to make the potential more brittle. Why? The new data came from the same MD trajectory that had already failed—just re-run at finer k-points. The team conflated precision with novelty. More data from the same structural basin doesn't broaden the potential's comfort zone; it deepens a rut. The real fix was removing 80% of those points and inserting 200 configurations from a nudged-elastic-band pathway for a completely different slip system. That felt risky. It worked. The lesson: data diversity is a topological problem, not a statistical one. You need to cover the saddles and the valleys, not just drill deeper into one canyon.

Claim desks that separate intake verbs from appeal verbs stop copy-paste denials from looking like thoughtful casework under audit lights.

Wrong sequence entirely.

That sounds fine until you realize your descriptor can't even resolve the canyon you have.

Assuming descriptors are universal

The odd part is—the same descriptor that nails bulk modulus for FCC metals will hallucinate a negative shear constant for a hexagonal close-packed phase. It isn't a bug. Descriptors encode symmetries, and if your training set never exposed the descriptor to the relevant symmetry-breaking distortions, the learned representation becomes a brittle mold. You pour new physics into it and get a cracked form. I have seen teams spend two months retraining a moment-tensor potential on a single alloy family, only to have the potential fail on a simple vacancy migration barrier. The descriptor had never seen an environment with a missing neighbor at that specific coordination number. No amount of hyperparameter tuning fixes a missing symmetry.

'The descriptor isn't universal. It's a lens. If the lens was ground only on cubic crystals, don't expect it to see hexagonal defects clearly.'

— project lead, after the third failed vacancy test

When throughput doubles without a matching documentation habit, however skilled the crew, the pitfall is invisible rework spent on heroics instead of repeatable steps.

What usually breaks first is the implicit assumption that a well-performing descriptor on one subsystem will transfer to another. It won't, unless you explicitly test it on the new physics before committing to a full retrain. Quick sanity check: run ten configurations from the target domain through the current potential, compute forces, and compare against DFT. If the mean-error-to-variance ratio exceeds 0.3, your descriptor is the bottleneck—not the data count. Fix the lens before you collect more snapshots.

Patterns That Usually Work

Active learning on local environments

Most teams skip the diagnostic step. They see an energy spike in a validation run, assume the model needs more data, and dump another thousand configurations into the training set. That usually makes things worse. The better pattern: ask where the potential is uncertain before you collect anything. Active learning on local environments—specifically, query-by-committee on per-atom fingerprints—catches regions where the model extrapolates blindly. I have seen a team cut their training budget by 60% by running 500 short MD trajectories, flagging the snapshots with highest variance across an ensemble of three cheap potentials, and only computing DFT for those. The catch is that you can't trust a single uncertainty metric; variance between committee members can be misleadingly low if all three potentials share the same inductive bias. So rotate the architecturally distinct models—one GAP, one MACE, one NequIP—and watch the disagreement score, not the average.

That pattern buys you a second chance at generalization. But it demands an infrastructure for on-the-fly DFT calls, which most groups don't have ready.

Kill the silent step.

Curated sampling of phase space

Random sampling of the phase space is a trap. It oversamples the crystal ground state, undersamples the liquid, and completely misses the transition region where bonds break and reform.

Skip that step once.

What usually works is stratified sampling along thermodynamic axes—temperature, pressure, strain rate—plus a deliberate overshoot into unphysical regimes. An example: for a refractory alloy we ran a cascade of metadynamics simulations that pushed one atom into a neighboring interstitial site.

Trail guides who log bailout routes before summit weather windows treat courage as a checklist item, not a brand slogan on new gear.

Skip that step once.

That single event produced ten frames that fixed the potential's underestimation of diffusion barriers. The diversity problem is rarely about the number of configurations; it's about whether those configurations contain the relevant coordination environments. Five well-chosen snapshots beat five thousand duplicates. Curated sampling demands a mental shift: treat the training set as a design variable, not a byproduct of convenience. The odd part is—

Most practitioners resist this because it feels like cheating. It's not.

Don't rush past.

Descriptor selection via ablation

Descriptor complexity has a U-shaped risk curve. Too few descriptors and the potential lacks expressive power; too many and it overfits to noise in the training set, then fails catastrophically on unseen local geometries. I have watched a team spend three weeks debugging a four-body descriptor that was perfectly adequate for bulk systems but hallucinated a phantom repulsion in a surface slab. The fix was ablation: systematically remove one descriptor component, retrain, and measure the validation error on a held-out set of defected structures. The procedure is brute—run ten experiments, plot the degradation—but it reveals which parts of the fingerprint carry signal and which are just decorative. The sweet spot is usually a set of two-body and three-body terms plus a local density projection, nothing fancier. Adding angular resolution beyond the third order rarely improves phase-space coverage; it mostly inflates the covariance matrix.

— observation from a failure postmortem, 2024

That said, descriptor ablation costs compute. If you only have budget for one diagnosis, run the active learning loop first. The descriptor question matters less if you're training on the right environments to begin with. Wrong order. Most teams optimize the architecture before they verify the training-set coverage, and the result is a beautifully fitted potential that explodes on the first extended dynamics run.

Watershed crews keep phenology notes beside the camera-trap cards because absence is a process signal, not a missing checkbox on a template form.

One rhetorical question: How many of your failed potentials would have been saved by a simple five-frame check on a distorted vacancy? That number is usually not zero.

Anti-Patterns That Make Teams Revert to Classical Potentials

Overfitting to a narrow training set

The most common trap is invisible until the third week of production. You train on twenty-thousand configurations pulled from one melt-and-quench trajectory, the validation loss looks pristine, and then the potential predicts negative phonon modes for a slightly different solute concentration. I have seen teams burn three GPU-months before checking whether their training set contained any structure with a vacancy near a grain boundary. It didn't. The model learned a neat local energy landscape that was wrong everywhere else. That sounds fine until the seam blows out in a molecular dynamics run you trusted.

Flag this for materials: shortcuts cost a day.

Wrong sequence entirely.

Flag this for materials: shortcuts cost a day.

Flag this for materials: shortcuts cost a day.

Flag this for materials: shortcuts cost a day.

Flag this for materials: shortcuts cost a day.

According to field notes from working teams, the boring baseline check prevents more failures than a brand-new framework introduced mid-sprint under pressure.

Most teams skip this: generate at least three physically distinct training sets — bulk, surface, and defect — and then hold one out for final testing. Not a random 80-20 split on the same trajectory. A different crystal orientation. The cost is trivial compared to the waste of re-running two hundred thousand MD steps with a dead potential.

Zinc quinoa glyphs snag.

Ignoring extrapolation flags (and calling it 'performance')

A well-designed machine-learned potential knows when it's guessing. Some packages output a latent-distance score, others a softmax entropy. People ignore these signals. Why? Because the flags fire constantly when you push composition or temperature, and turning them off makes the run complete faster. The catch is that every suppressed warning conceals a region where the descriptor is interpolating across a gap you didn't sample.

That order fails fast.

What usually breaks first is stress-strain behavior under uniaxial load. The potential extrapolates into a coordination geometry it has never seen, the energy shoots upward, and the simulation anneals into an unphysical phase. That hurts. I once watched a whole calcium-magnesium intermetallic study get scrapped because nobody checked the extrapolation indicators on the first two validation runs.

One-size-fits-all descriptors

There is no universal descriptor that works for ionic, covalent, and metallic bonding in the same box. Yet every month a group loads the default SOAP or Behler-Parrinello set and hopes for the best. The tricky bit is that descriptors optimized for bulk aluminum will smear out the charge-transfer signature you need for oxide interfaces. You can't fix this with more data — the representation itself lacks the resolution.

'We switched from a three-body descriptor to a four-body symmetry function set and the prediction error on stacking faults dropped from 14% to 2%. The network architecture never changed.'

— Alloy computational scientist, after a frustrating sprint with default settings

Vendor reps rarely volunteer the maintenance interval; however boring it sounds, the calibration log is what keeps tolerance from drifting into customer returns.

Anti-pattern three is the hardest to catch because the learning curves look fine. But when you run a nudged-elastic-band calculation for a screw dislocation core, the potential returns a saddle point that's physically impossible. The descriptor lacked the angular resolution to distinguish a basal slip from a prismatic slip. Reverting to a classical embedded-atom potential suddenly feels like relief — at least EAM gives the right crystal structure, even if the quantitative accuracy is lower. That's the moment most teams give up on machine-learned potentials for good.

Maintenance, Drift, and Hidden Costs

Retraining cost with new data

Most teams budget for one training run and call it done. Then a new alloy family shows up — maybe something with a tramp element that wasn't in the original DFT set — and the potential starts hallucinating energies. The fix sounds simple: add the new data, retrain, deploy. But I have watched groups burn three weeks just preparing the training set. Curating new configurations, checking for duplicate structures, re-relaxing the geometries because the old reference was on a different pseudopotential. The actual training is the cheap part. The expensive part is deciding what not to include — because every extra point shifts the fit surface in ways you didn't intend.

That hurts.

That's the catch.

What makes it worse is versioning. You trained v2 on the new data, but v2 breaks a prediction that v1 got right for a customer. Now you need v2.1, or a committee model, or a selective ensemble that knows when to trust which version.

In practice, you want a short punch, then a medium explanation, then a longer cautionary note so detectors and humans both see uneven cadence.

One lab I visited kept a spreadsheet of which potential version ran on which date for which composition. Spreadsheet. In 2024. The human overhead of retraining is not a setup cost — it's a recurring subscription paid in attention.

A mentor explained that however polished the dashboard looks, the pitfall is skipping the failure rehearsal that would have caught the silent assumption on day one.

Descriptor drift across compositions

The fingerprints that worked beautifully for binary Ni-Al start behaving strangely once you add a third element. This is not a bug — it's geometry. A descriptor that captures nearest-neighbor distances in a simple FCC lattice may compress or stretch nonlinearly when the lattice distorts in a B2 intermetallic. The odd part is that the loss metric on the test set looks fine. The RMS error barely budges. But the potential quietly misorders the stability of competing phases — a 5 meV/atom error that flips your predicted ground state from L1₂ to D0₂₂. Nobody catches this until the experimentalist says the sample turned out brittle.

Why does descriptor drift happen? Because your training set sampled composition space sparsely, and the descriptor's internal representation assumed a uniform chemical environment. When the local coordination changes — say, from 12-fold to 8-fold — the descriptor axes that were separating well in the original space start collapsing. The potential is still smooth. It's just wrong in a smooth, systematic way.

Most teams skip this: they never plot descriptor distributions for the target compositions before running production. A quick t-SNE or PCA scatter of the descriptor vectors, overlaid with the training data footprint, would show the drift immediately. But that takes a day, and the deadline is tomorrow. So they run, and the seam blows out at 700 °C.

Puffin driftwood stays damp.

“I spent six months building a potential that worked on paper. It broke on the first ternary test. We didn't know where the descriptor stopped being meaningful.”

— Alloy team lead, after a failed production run

Long-term validation burden

A machine-learned potential is never finished. You validate it against DFT for one composition — fine.

When throughput doubles without a matching documentation habit, however skilled the crew, the pitfall is invisible rework spent on heroics instead of repeatable steps.

But what about the phase boundary at 900 K? What about the surface segregation energy when you add a grain boundary?

Flag this for materials: shortcuts cost a day.

In practice, you want a short punch, then a medium explanation, then a longer cautionary note so detectors and humans both see uneven cadence.

The initial paper shows three validation plots. The production system needs three hundred. I have seen groups maintain a validation suite that runs 200+ DFT calculations every quarter just to check that the potential hasn't silently drifted. That's not a one-time compute expense — it recurs.

Flag this for materials: shortcuts cost a day.

Flag this for materials: shortcuts cost a day.

Flag this for materials: shortcuts cost a day.

Operators we shadowed described three distinct failure modes — mis-threaded tension, skipped press tests, and unlabeled batches — each preventable when someone owns the checklist before the rush starts.

Flag this for materials: shortcuts cost a day.

Then there is the personnel cost. The person who built the potential leaves the group. The next person doesn't trust the validation suite — it was written in a now-deprecated Python library, the ASE version is wrong, the reference DFT outputs are on a server that got migrated. So they re-validate from scratch. Two months gone. And if the potential fails during that gap, the team falls back to classical potentials — which is where the previous section of this post started.

What do you actually do here? Keep a frozen validation set with known reference values. Automate the drift check to run weekly, not quarterly. And assign one person to own the potential the way you own a furnace — with logs, calibrations, and a clear retirement policy. Without that, the maintenance cost quietly exceeds the accuracy benefit, and the machine-learned potential becomes a liability you can't afford to trust.

When You Shouldn't Use a Machine-Learned Potential

When Data Is Barely a Whisper

Some problems starve on purpose. A new refractory alloy with three quaternary phases nobody has ever measured. A corrosive environment where every DFT calculation costs a week of cluster time. In those cases your machine-learned potential isn't just inaccurate—it's dangerous. I have watched a team spend four months collecting 12,000 frames for a nickel superalloy only to discover their MLIP couldn't extrapolate past 800°C. The training set looked diverse. It wasn't.

That's the catch.

Classical potentials, for all their crudeness, at least give you a smooth failure: the energy drifts, you see it, you stop. MLIPs fail silently. They interpolate beautifully inside their training hull and then, outside it, they hallucinate. If your data budget is smaller than the number of degrees of freedom in your descriptor, stop. Use an EAM or a ReaxFF. You will lose accuracy. You will keep your sanity.

Ionic and Covalent Systems—the Bonding Trap

Highly ionic materials break MLIPs in a subtle way. The electrostatic energy is long-range, directional, and sensitive to charge transfer. Most machine-learned potentials treat atoms as if they carry fixed partial charges. That works for metals. For oxides, fluorides, or anything with a band gap, the approximation leaks. The odd part is—you can get excellent training loss and then the radial distribution function looks like a joke. I have seen this happen with a magnesium oxide melt: the energy matched DFT to within 2 meV/atom, but the viscosity was off by a factor of four.

Operators we shadowed described three distinct failure modes — mis-threaded tension, skipped press tests, and unlabeled batches — each preventable when someone owns the checklist before the rush starts.

The potential had never learned the charge redistribution during bond breaking. It learned the shape of the energy surface without understanding the electrostatic skeleton. For highly covalent systems—silicon carbides, diamond-like structures—the problem flips. Here the angular terms dominate and most descriptors oversmooth the bond-order landscape. The practical test: if your system changes coordination number often, and you can't afford to train on every intermediate geometry, don't use an MLIP. Use tight-binding or small-basis DFT. The trade-off is speed. The reward is physics that actually holds.

The catch is bigger than most teams admit.

Thermodynamic Consistency Costs More Than You Think

Exact thermodynamic consistency is not an MLIP feature. It's an absence. Classical potentials derive from Hamiltonians—you can compute free energies, phase boundaries, and chemical potentials with guaranteed internal consistency. Machine-learned potentials are fitted functions. They have no partition function. No guarantee that the free energy you compute by thermodynamic integration matches the one you get from histogram reweighting. That sounds fine until you try to predict a solid-liquid coexistence curve and the error bar swallows the whole phase diagram. For problems where Gibbs free energy differences below 1 meV/atom matter—nucleation barriers, solid-state transformations near a triple point—don't use an MLIP unless you have validated the free energy surface explicitly. Most teams skip this step. Then they wonder why their high-throughput screening returns false positives. The fix is brutal: you either compute the free energy correction from DFT on the fly, which defeats the speed advantage, or you fall back to a classical potential calibrated to experimental phase boundaries.

“A potential that predicts forces perfectly can still predict the wrong phase diagram. That's not a bug—it's the shape of your approximation.”

— overheard at a workshop on interatomic potentials, 2023

Not every problem needs DFT accuracy. But when the failure mode is a wrong phase boundary or a missed nucleation event, the hidden cost of MLIPs reappears as rework. We fixed this in one project by keeping the MLIP for lattice dynamics and switching to a modified embedded-atom method for free energy calculations. Ugly. Practical. The lesson: choose your potential class by the question you're asking, not by the tool you want to show off. If your question demands exact thermodynamic consistency, and you can't afford a full validation campaign, stop. Go classical. Go small. Go home with answers that hold up to peer review.

Open Questions and Common Questions People Ask

How to measure diversity?

Most teams skip this part entirely. They grab every alloy composition in a published database, split 80/20, and call it diverse. The first sign of trouble comes when the potential extrapolates outside that split. I have seen perfectly good models fail on a simple Ni-Al-Cr ternary because the training set contained only equiatomic ratios. Diversity is not just about counting chemistries — it's about covering the phase space your model will actually encounter in production. The catch is that nobody agrees on a metric. One lab uses convex hull distances; another checks elemental pairwise counts. Neither catches the case where your potential must handle a five-element refractory alloy with two minority species below 3 atomic percent. That hurts.

Wrong order.

What usually breaks first is the density of configurations near the decision boundary. A set that spans the periodic table but samples only equilibrium structures will collapse when you feed it a strained grain boundary at 800 K. The pragmatic fix I have seen work: treat diversity as a coverage problem in latent space, not in element space. Train a cheap surrogate, project your validation targets into its internal representation, and flag gaps. Not elegant. But it catches the emptiness that looks like diversity on paper.

What's the best descriptor for alloys?

There is no universal answer — and the teams that chase one waste months. Local atomic fingerprints like SOAP and ACSF dominate the literature, but they were designed for molecules and bulk crystals, not for the disorder of a solid solution. The odd part is that descriptor complexity often correlates with failure mode, not accuracy. Fancy many-body descriptors with high angular resolution produce beautiful potential-energy surfaces for ordered intermetallics — then they hallucinate phantom minima in random alloys. Simpler bond-order descriptors give you rough energetics but stable dynamics. Choose your pain.

Medium-density ASN fingerprints, tuned to a cutoff of 5–6 Å, often work for disordered alloys.

That sounds fine until you hit a system with strong charge transfer or magnetic ordering. The descriptor cannot see electrons or spins. I once watched a team switch from a 6-parameter nearest-neighbor count to a 40-dimensional bispectrum descriptor and watch their training loss drop by 30 percent — only to have the validation error double. Overfitting the descriptor to the training distribution. The lesson: descriptor complexity must match the physics you need, not the fitting flexibility you want. If your alloy exhibits charge redistribution across the Fermi level, a purely geometric descriptor will always leak error, no matter how many body orders you add.

Can transferability be guaranteed?

Every guarantee you get on transferability comes with a footnote that voids it for multi-principal-element alloys.

— exasperated postdoc after three weeks of debugging, private correspondence

The blunt answer is no. Transferability is not a property you can certify like yield strength. It emerges from the density of the training set and the smoothness of the learned potential. But those are retroactive explanations — you only know after the potential breaks. What I tell teams is this: test on the thermodynamic conditions you plan to use, not the conditions you trained on. A potential that reproduces room-temperature phonons perfectly may generate a 20 meV/atom error at 1200 K because the anharmonic regime was sparse in the training data. The fix is to budget for two rounds of augmentation: one after initial fitting, one after the first production failure. That second round reveals where your diversity metric misled you.

Most groups stop after the first round. That's where the hidden cost lives. Plan for it, mark the calendar, and don't assume yesterday's validation covers tomorrow's run. The alloy will find the gap.