Apple study details an AI-powered tool that helps developers build UI prototypes

Apple researchers have published details of SQUIRE, an experimental AI-powered tool that helped developers explore and refine interface ideas with more control than with typical AI coding tools. Here are the details.

An interesting approach to AI-powered interface prototyping

In a new paper titled SQUIRE: Interactive UI Authoring via Slot QUery Intermediate REpresentations, a groupof Apple developers proposes a novel way to approach AI-generated interfaces.

As they explain, natural language has been bringing more flexibility to multiple aspects of the development process, but it also brings two key challenges:

That’s where SQUIRE comes in. It is a visual interface that lets developers build and refine UI prototypes step by step, with clearer control over the results.

From the study:

In other words, natural language prompts were still how developers interacted with SQUIRE, but instead of affecting the entire interface at once, each prompt is tied to a specific part of the UI.

Based on observations with 11 frontend developers who used SQUIRE to develop interface prototypes, the researchers found that participants were able to explore and iterate on different UI designs with a strong sense of control, while also rating the system highly for usability and overall satisfaction.

Additionally, they note that this extra sense of control made developers more comfortable exploring paths they might not have tried otherwise, since changes were easy to make, predict, and undo.

From the study again:

SQUIRE under the hood

Rather than generating interface code directly from user prompts, SQUIRE first creates its own intermediate representation of the interface, called SquireIR, which models the UI as a tree of components with named slots that can be filled in over time, like in the example below:

That structure can also include placeholders for parts that have not yet been defined (e.g., a button label, an image, or a section of content), as well as multiple possible UI alternatives. For instance, it could represent the same content as either a list or a grid.

From there, SQUIRE translates that representation into code using HTML, CSS, and JavaScript, with Web Components handling the final UI structure.

Another key aspect of SQUIRE is how it handles changes.

If a developer asks to tweak a button or adjust a layout, only that part is updated, while everything else remains untouched.

According to the researchers, this helps avoid trial-and-error loops seen in many AI coding tools due to the overall unpredictability of LLMs, where the model may make changes beyond what the developer intended.

This structure is what also allows SQUIRE to suggest multiple options at each step, so developers can quickly compare different versions without losing their previous work.

Contrary to many technical papers, this study doesn’t go into detail about model training, architecture, or data. The researchers do note that SQUIRE is powered by OpenAI’s GPT-4o, but the paper’s focus is firmly on the system design and interaction model

SQUIRE is not generally available, and its use was limited to the 11 developers who participated in the study. However, it is not hard to imagine how something like this could be implemented in future versions of Xcode, or other Apple-built development tools.

To learn more about SQUIRE, follow this link.

Worth checking out on Amazon ...