Background removal is a harder problem inside a design tool

At Config 2024, Figma announced a set of AI features built into the canvas. One of them was background removal, and it runs on the Jasper API.

The interesting part of that arrangement is not the announcement. It is the constraint. A model that lives inside somebody else's design tool has to meet a different standard than the same model running as a standalone utility, and the difference is not mostly about quality.

The workaround it replaced

Before this existed, removing a background in a design workflow meant leaving the design tool. Export the layer, open a separate application or a plugin, process the image, bring the result back, and hope the transparency survived the round trip.

Jasper's creative director Jess Rosenberg described the old state plainly: "For so long, designers have had to rely on various workarounds for removing a flat image's background. Now, with the Jasper API integration within Figma, it'll be easier than ever to keep this simple action right within your Figma workflow so you can stay in flow state longer, and avoid toggling between tools."

Figma's chief product officer Yuhki Yamashita framed the same point from the product side: "With Figma AI, designers can focus on more strategic work because time-consuming tasks like creating placeholder content or wiring up prototypes are now automated. This includes common tasks like background removal."

Both descriptions land on the same word, which is flow. That is the actual product being sold, and it sets the engineering requirements.

Embedded changes the latency budget

A standalone background removal tool is a destination. Someone navigated to it on purpose, they expect to wait, and a spinner is acceptable because waiting was already priced into the decision to go there.

A feature inside a canvas is not a destination. It sits next to operations that complete instantly, like moving a layer or changing a fill, and it inherits the expectations set by its neighbours. If it takes long enough to break concentration, it stops being a native capability and becomes a detour that happens to live in the same window. The user has to context-switch either way, which was the entire problem the feature was meant to solve.

This is why the model shipped alongside a body of work on inference speed. Earlier the same month, Jasper published Flash Diffusion, a distillation method for producing higher-quality image outputs faster and at lower cost. Speed work of that kind is what makes an embedded feature viable at all. Latency is not a nice-to-have that gets optimized in a later release. It determines whether the integration makes sense in the first place.

Embedded also changes what counts as an edge case

The second constraint is less obvious and probably harder.

When a model is a standalone tool, its failures are visible and recoverable. The user sees a bad cutout, shrugs, and tries something else. Nobody's document is damaged. When the same model is a menu item inside a design tool, a bad result lands directly in someone's working file, next to their real work, with the implicit endorsement of the application that offered it.

That raises the floor considerably. The distribution of images a design tool sees is also far wider than a product catalog. Designers paste in screenshots, illustrations, photographs, renders, scanned artwork, and images that have already been edited several times. There is no controlled input format and no opportunity to tell users to shoot differently.

Which puts the weight on the cases that are genuinely hard rather than the ones that demo well:

• Hair and fur, where the boundary is thousands of thin strands and the correct answer is partial transparency rather than a decision.
• Motion blur, where the edge is a gradient by definition.
• Transparent and reflective objects, where the background is legitimately visible through the subject.
• Fine structures like jewelry, wires, and foliage, which a slightly aggressive mask erases entirely.
• Low contrast between subject and background, where there is no strong signal to separate on.

The portrait at the top of this article is the canonical version of the first case. It is easy to produce a cutout of that image with a clean silhouette, and a clean silhouette is the wrong answer. The strands have to stay, which means the model has to output partial alpha values rather than a binary in-or-out decision for each pixel.

Why the API boundary is the right one

There is a reasonable question about why a company like Figma would call an external service for this rather than run something in the browser.

The answer is that image models are infrastructure with a maintenance cost. Keeping a model competitive means retraining as techniques improve, expanding coverage as new failure cases surface, and continuing to invest in inference speed. A product team that owns a model owns all of that indefinitely, and none of it is the thing their product is actually about.

Treating it as an API call means the capability improves without the integrating team doing anything. That is the same reason it is worth building these as single-purpose models with stable interfaces rather than as features of a larger application. The integration surface stays small, and the thing behind it can change.

The general shape

The Figma case is a specific instance of a pattern that keeps recurring. Editing capabilities that used to justify their own applications are becoming operations inside whatever tool someone is already using, and the winner in each category is not necessarily the most capable model. It is the one fast enough to feel native and reliable enough that a product team will put its own name on the output.