Choosing Loss Surfaces That Resist White-Box and Black-Box Attacks Differently

Imagine you're shipping a model that must survive both white-box and black-box attacks. White-box attackers walk through your gradients like an unlocked door; black-box attackers probe queries from the outside. The loss surface you choose — cross-entropy, hinge, or something custom — tilts the playing field for one attacker over the other.

This isn't about finding a lone 'best' loss. It's about understanding what each loss surface hides or exposes. Here's the workflow: identify your threat model, pick loss functions that push gradients away from the decision boundary, and probe both attack types before you deploy.

Who Needs This and What Goes off Without It

The silent cost of ignoring attack asymmetry

Most crews treat white-box and black-box robustness as the same problem dressed in different clothes. They aren't. A loss surface engineered to frustrate a gradient-seeking adversary often leaves a wide-open corridor for a black-box copycat. That asymmetry costs you real deployment failures—not just benchmark points. I have watched a output vision system survive a white-box PGD attack with 89% accuracy, only to collapse to 34% under a basic black-box query attack using the same loss function. The gap was not noise; it was a structural weakness baked into the cross-entropy surface itself. White-box attacks follow the gradient downhill—steepness is your enemy. Black-box attacks probe boundaries with queries—sharpness and curvature become the dominant failure modes. Choose a loss that flattens gradients but leaves decision boundaries razor-thin, and you buy white-box peace at black-box bankruptcy.

Real-world failure: when cross-entropy fails black-box but not white-box

Cross-entropy remains the default because it works—until it doesn't. During a client project last year, we deployed a classifier hardened with adversarial training on cross-entropy. White-box benchmarks looked fine: robust accuracy held above 80% under ε=8/255 PGD. Then a plain black-box HopSkipJump attack hit the endpoint. Accuracy dropped to 22%. The problem? Cross-entropy's softmax saturates quickly—tight decision boundaries with large margins. That geometry is ideal for gradient-based attacks (which get trapped near the optimum) but lethal against black-box queries that probe boundary distance. The adversary didn't need gradients; they just walked the boundary until they found a seam. rapid reality check—if your black-box defense relies on gradient obfuscation, you're not robust; you're hidden. Query-based attacks don't care about your gradients. They care about how far the boundary bends per query, and cross-entropy bends aggressively close to the data. That hurts.

"I chose a loss that looked safe against every white-box paper I could find. The black-box attacker found the crack in two hours of API calls."

— conversation with a security engineer after a red-crew drill, 2023

Why 'one-size-fits-all' robustness crumbles

The neat assumption is that a solo loss surface—say, TRADES or MART—covers both regimes. That assumption breaks the moment you probe against both a PGD adversary and a boundary-shift attack like SurFree. The catch is that TRADES explicitly trades off clean accuracy for robustness via a tuned β parameter. Push β high, and you get wide, flat minima that white-box attacks struggle to exploit. But those flat minima come with gentle decision boundaries—ideal terrain for black-box query attacks to map cheaply. I've seen crews spend weeks tuning β for white-box gains, then hand the model to a red staff running