Product and Design in the Age of AI: We Are Designing Behaviour, Not Just Interfaces

Most software waits for an instruction and produces a predictable result. AI products infer, generate, recommend and sometimes act. The same input can produce different outputs. The system can be useful without being completely correct, confident without being right and technically successful while violating the user’s intent.

That changes the object of design.

We are no longer designing only screens and flows. We are designing behaviour under uncertainty.

The product team now has to decide what the system should attempt, what it should never do, how it communicates doubt, when it asks for permission, how a person can correct it and how the organisation will know whether it is getting better.

A chat box does not answer any of those questions.

The interface is the visible edge of a larger system

Traditional product design often assumes a relatively stable mapping between action and outcome. Tap “send,” and the message sends. Apply a filter, and the list changes according to known rules.

An AI feature introduces additional layers:

• the model interprets the user’s intent
• the system selects context and tools
• the model produces or proposes an output
• policies determine what can happen next
• evaluation estimates whether the behaviour is acceptable
• feedback changes the future experience

The interface may be simple while the behavioural system is complex. If product and design teams focus only on the visible layer, engineers end up making product decisions implicitly through prompts, thresholds and tool permissions.

Those are design decisions. They should be treated as such.

Start with the user’s decision, not the model’s capability

“What can we build with this model?” is an exciting exploration question. It is a poor product strategy.

The stronger starting point is: Which difficult decision or task should become meaningfully easier, and what must remain under human control?

For Neural Jobs, an AI system could summarise every vacancy, score candidate fit, recommend roles or draft applications. Each capability is possible. But the value depends on a more specific outcome: helping a person identify work that fits their evidence and intent without hiding important requirements or manufacturing confidence.

This framing prevents the common pattern of adding AI where a filter, template or deterministic workflow would be clearer and cheaper.

Five layers every AI product team must design

1. Mental model

Users need a useful understanding of what the system can and cannot do. That does not require a lesson on transformers. It requires clear expectations.

Is the output a verified answer, a draft, a prediction or a recommendation? What information did the system use? Will it remember this interaction? Can it take action or only suggest one?

Google’s People + AI Guidebook treats mental models, explanations, feedback and graceful failure as core design patterns. The principle is simple: trust depends on the user being able to predict the system well enough to use it safely.

2. Intent and permission

Inference is not consent.

An assistant may infer that a candidate values salary over location, but it should not silently exclude a role on that basis. An agent may draft an email, but sending it is a different level of permission. A recruiting tool may summarise a résumé, but inferring protected characteristics is another matter entirely.

Design permission as a ladder. Low-risk, reversible actions can be easier to automate. High-impact or externally visible actions require explicit confirmation, preview and a record of what happened.

3. Evidence and uncertainty

AI interfaces should not decorate uncertainty with confidence.

Product teams need to decide when to show sources, alternatives, ranges, missing information or an explicit “I do not know.” A single confidence score is rarely enough; it can look scientific while concealing a weak evaluation.

The right presentation depends on the decision. A creative suggestion may need little explanation. A claim about salary, eligibility or medical risk needs traceable evidence and careful boundaries.

4. Control and recovery

Every important AI action needs a path back.

Can the user edit the plan before execution? Undo the action? Correct the source data? Narrow the scope? Report a bad result? Return to a deterministic workflow?

Graceful failure is not a fallback screen designed at the end. It is part of the primary experience. The more autonomous the system becomes, the more visible control and recovery must become.

5. Feedback and evaluation

A thumbs-up button is not an evaluation strategy.

Teams need behavioural measures tied to the user outcome. Did the candidate discover more relevant jobs? Did a recruiter reach a sound decision faster? Did users accept, edit or reverse the system’s proposal? Which error types create harm even when the average metric improves?

Designers and product managers should help construct evaluation sets, label failure patterns and observe real use. Otherwise, model quality is defined only by what is easy to measure technically.

The new product specification is an evaluation contract

Traditional requirements describe what the software should do. AI requirements also need to describe what acceptable behaviour looks like across a range of cases.

For an AI feature, a useful product specification includes:

• the intended user and decision
• examples of successful behaviour
• examples of unacceptable behaviour
• known ambiguous and adversarial cases
• what evidence the output should expose
• when the system should ask, abstain or escalate
• what actions require confirmation
• quality, latency and cost targets
• who reviews failures after launch

This is not a static document. It becomes a shared evaluation set that product, design, engineering and domain experts can use to judge changes.

The evaluation contract is one of the most important new artefacts in AI product work because it makes taste, safety and user value discussable before a model ships.

Product managers need technical judgment, not model theatre

AI product managers do not all need to train models. They do need to understand enough to make honest tradeoffs.

They should be able to ask:

• What is the baseline without AI?
• Which failure modes did we test?
• Is the problem the model, the context, the tool or the interaction?
• How representative is the evaluation set?
• What happens when input distribution changes?
• What does one successful task cost at realistic usage?
• Which data are stored, and who can access them?
• What must a human approve?

Without this literacy, roadmaps become lists of capabilities copied from model announcements. With it, teams can select smaller, defensible problems and improve them systematically.

Designers need to prototype outcomes, not only flows

A static prototype shows the happy path. An AI prototype should show a distribution of paths.

Test the same task with clear, vague, incomplete, conflicting and malicious inputs. Include excellent outputs, mediocre outputs, refusals and wrong answers. Prototype the delay while tools run. Show what happens when a source disappears or an action only partially completes.

Then design the user’s response to each state.

This work may look less polished than a perfect demonstration. It is much closer to the experience people will actually have.

The strongest AI design portfolios will increasingly include:

• the behavioural problem definition
• sample inputs and output variation
• the evaluation framework
• permission and recovery decisions
• observed failure patterns
• evidence of iteration after real use

The screen remains important. The reasoning system around it is the differentiator.

The team boundary has to change

In conventional product development, design may hand an interaction to engineering and inspect the implementation later. AI makes that separation fragile because behaviour emerges from the model, data, prompts, tools, policies and interface together.

A better operating unit is a small cross-functional group that reviews behaviour continuously:

• product defines the user outcome and business constraint
• design defines interaction, control, explanation and recovery
• engineering defines system behaviour and operational limits
• data or research defines the experimental method
• domain experts define consequential errors
• legal, safety and security partners define non-negotiable boundaries

The group should review real examples, not only aggregate dashboards. Ten carefully chosen failures can teach more than one average score.

Do not automate away the source of trust

The most tempting AI roadmap removes effort. But some effort is how people exercise judgment, build skill or establish accountability.

If an AI hiring system removes the recruiter’s need to read evidence, it may also remove the moment when nuance enters the decision. If a design tool generates every option instantly, junior designers may lose the practice of forming and defending a point of view. If an agent performs a high-impact task invisibly, the user may be unable to notice when the context has changed.

The goal is not minimum human involvement. It is the right human involvement.

The durable advantage is a point of view about behaviour

Models will improve. Many capabilities that feel distinctive today will become standard infrastructure. A product cannot defend itself indefinitely by having access to the same model as everyone else.

The defensible work is deciding:

• which context makes the system useful
• which behaviour earns trust
• which feedback compounds into better performance
• which risks are unacceptable
• where human judgment changes the outcome

That is product strategy, interaction design and organisational design at once.

The age of AI does not make product and design less important. It expands their responsibility. We still design what people see—but now we must also design what the system believes it is allowed to do, how it behaves when reality is unclear and how humans remain capable of steering it.