Last updated:
AECBench: can AI add doors and windows to a building?

- We gave an AI agent 20 BIM models in Snaptrude with walls and rooms but no doors or windows, plus a one-sentence request to add them.
- A reviewer LLM, fine-tuned against reviews from four architects, scored each building out of 10 on door type, swing, width, sill height, coverage and facade.
- The agent finished all 20 buildings with a mean score of 7.4. 14 scored 7 or more, and it fixed 267 door problems on its own.
- Every stair door was fire-rated. Low window sills were the most common problem, and lifts with no lobby cost the most points.
Picture a hotel corridor at night. A guest opens their door, and it swings out into the path of someone walking past. Further down, a guest in a wheelchair stops at the accessible suite: its door is a few centimetres too narrow to get through. Inside a guest room, the window sill sits at knee height. At the end of the corridor is the stair, the one safe way out in a fire. On the plan, its door looks like every other door. That's the problem: it's a standard door with no fire rating, and it won't hold back the smoke.
Each of these is a small mistake, and most are almost invisible on a drawing. Repeat them across every room on six floors, and the building stops working. Every door has to open the right way, be wide enough for the people and things that pass through it, and suit its job: an exterior door at the entrance, sliding doors at the lift, a fire-rated door at the stair. Every window has to sit at the right height and open the right way.
This post zooms in on one part of AECBench, our benchmark for measuring AI on real architectural work. AECBench covers space planning, BIM detailing, drawings, services and value engineering. Doors and windows sit inside BIM detailing, and they're what this post is about.
Here's the setup. We give an AI agent a whole building with walls and rooms, but no doors and no windows. The request is a single sentence, like “Add doors and windows to this hotel.” Then we score only the doors and windows the agent adds.
Figure 1. One task, from start to score. The office tower starts with walls and rooms only. We ask for doors and windows in one sentence. The agent adds 220 doors and 366 windows on 8 floors, and the reviewer gives 10 out of 10. The building images are captures of the finished model in Snaptrude. For the video, we hid the agent's doors and windows and brought them back in groups. We chose that order, and it doesn't reflect the order the agent worked in.
How does one benchmark task work?
Each task is one building. The agent works inside a real BIM model in Snaptrude, so every door and window it adds is a real building component with a width, a swing and a family. A reviewer then inspects every floor of the result.
Figure 2. One benchmark task. There's no answer key. The request doesn't mention a door count, a door type or a swing direction. The agent has to read the building and decide.
Here's what that looks like on the hotel's ground floor. The rooms and walls stay put. The agent only adds openings: a door from each room to the corridor, each one swinging into its room, double doors to the lift lobbies, exit doors from the two stairs, and windows along the outside walls.

BeforeAfterFigure 3. One hotel floor, before and after. Drag the handle. Both images are captures of the finished model in Snaptrude, with the same camera. For “Before”, we hid the doors and windows the agent added. The small gaps in the walls show where they go.
What buildings are in the benchmark?
We built 20 buildings of different types, with 90 floors and 1,694 rooms between them. Each one has corridors, stairs, lifts and a different use on each floor. Architects checked the room sizes first, and we rebuilt the layouts so the rooms, corridor widths and distances to stairs are realistic.
What makes a door or window right?
We only score the doors and windows. Most problems are small on their own, but they repeat in every room and on every floor. Here are six of the checks:
Doors do not swing into corridors
A door that opens into a corridor can hit people who walk past.
Doors open against a wall
The hinge goes on the side near the wall, so the open door does not block the room.
Lift doors slide and open onto a lobby
A lift needs sliding doors at least 1.2 m wide, with a lobby in front where people wait.
Windows sit at the right height
Architects expect a window sill at 0.9 to 1.0 m above the floor in most rooms.
Each door suits its place
A single door type everywhere doesn't work. A stair needs a fire-rated door, a lift needs sliding doors and a hall needs a pair.
Windows make one facade
Windows line up from floor to floor, repeat a few sizes, and give each room the right amount of glass.
Figure 4. Six of the checks. Each panel shows a common mistake first, then the correct result.
For every building, the reviewer checks:
- Coverage: every room that needs a door or a window has one, on every floor.
- Entrances: people can get into the building from outside.
- The right type for the place: an exterior door at an entrance, sliding doors at a lift, a fire-rated door at a stair, pair doors for a large room.
- Door swing and hinge side, as shown above.
- Door width: doors are wide enough for their use, for example for wheelchairs or beds.
- Windows: windows open the right way and sit at the right height.
- Lift doors slide and open onto a lobby.
- Stair doors are fire-rated.
- The facade system: windows line up from floor to floor, use a few sizes, and give each room the right amount of glass for its use.
- Honesty: what the agent says it did matches the model. If the agent says “130 windows” and the model has 98, it loses points.
How did we build a reviewer that agrees with architects?
There's no single correct set of doors for a hotel. Ask two good architects and you'll get two different answers, so a simple pass-or-fail test can't grade this. Instead, a reviewer LLM inspects every floor of the finished building and gives it a score from 1 to 10. That reviewer is only useful if it agrees with architects, so we fine-tuned it against their reviews.
Four architects scored all 20 buildings and wrote down what was wrong. At first, the reviewer's scores looked nothing like theirs. So we studied what it missed, gave it measurements as well as pictures, and tested it again on the same 20 buildings.
Figure 5. How we fine-tuned the reviewer LLM. The architects' reviews came first. Everything the reviewer checks today comes from what they found.
So how do we know the fine-tuning worked? We rank the 20 buildings from best to worst twice: once by the architects' scores and once by the reviewer's. Comparing the two lists gives us a single number. If both lists are in exactly the same order, it's 1. If the reviewer's order has nothing to do with the architects', it's 0.
Hospital
Office tower
Senior living
Figure 6. Before and after fine-tuning. Before, the reviewer gave high scores to buildings that the architects scored low. After fine-tuning, it scores them lower, as the architects did.
Results
Here are the results across all 20 buildings.
Figure 7. Score of each building. The office tower, the largest building with 144 rooms on 8 floors, got the only 10. The theatre got 3 because the agent reported windows in rooms that didn't have any.








Figure 8. Doors and windows up close, in eight finished buildings. The plans are close-ups of one part of a floor: each door, which way it swings, and the row of windows along the facade. The elevation shows the facade that the windows make. All are Snaptrude's own 2D drawings of the finished models from the benchmark run, with the reviewer's score.
The agent fixes its own mistakes
The agent didn't always get a door right the first time. While it built the model, it caught most of its own mistakes and fixed them before finishing. That's where most of its score came from.
Figure 9. Problems the agent found and fixed itself. Counts are doors, from the agent's first and last self-check in each building. In the courthouse, 16 courtroom doors still open outward. The agent explained that this is correct for rooms where many people gather. The lift doors in the library, mall and theatre are covered below.
Choosing the right door and window
Two problems never showed up. Every stair door was fire-rated from the start in all 20 buildings, and no window opened the wrong way.
This comes from how the agent chooses families. A family is a type of door or window in Snaptrude's library, such as a flush door, a sliding lift door or a fixed two-panel window. The agent treats each opening as a job first: a door to a stair, a lift landing, an entrance, a door into a room, a window for a room that people use. Then it picks a family that suits that job and places it with the right settings. A stair door goes in with its 60-minute fire rating. The rating comes with the door, so there's nothing to forget later.
Figure 10. How the agent matched families to jobs in the hotel. The reviewer checked every one: all 16 stair doors were fire-rated, and all 14 lift doors were 1.2 m sliding doors. The other buildings followed the same pattern. The theatre, for example, also got pair doors for the auditorium exits and a rolling shutter at the loading bay.
Where it lost points
Figure 11. Main problems by building. A dot means the reviewer listed the problem as a weakness for that building. Low sills are window sills below the 0.9 m that architects expect.
- Lift doors. Lifts with no lobby cost the most points. This one has a different cause, so it gets its own section below.
- Window height. In 6 buildings the reviewer found window sills below 0.9 m, most at about 0.6 m. This happened even in buildings that scored 8 or 9.
- Missing doors and windows. In the transport terminal, 38 ground-floor rooms had no door and the building had no entrance. The municipal hall missed 2 doors and 4 windows on the ground floor. The sports centre had 12 studios with no window.
- Door position. In the apartment block, the library and the mixed-use building, many large rooms got their door in the middle of the corridor wall. The door works, but it wastes space. The path from the door cuts the room in two, and the open door stands in the middle of the floor. A door near a corner keeps the path along the side wall and leaves one large area for furniture. The reviewer didn't take points for this, because the standard doesn't yet say where along a wall a door should go. We spotted it in the plans, and we'll add it as a criterion.


Figure 12. Where the door sits in a large room. Top: one room, with the door in the middle of the wall and then near a corner. The red band is the path from the door, and the coloured areas are free for furniture. Bottom: the same choice in two finished buildings, in Snaptrude. The apartment block puts the door in the middle of most large rooms. The hotel puts it near a corner, with the open door against the side wall.
When the starting building is the problem
The largest loss came from the lifts in the library, the mall and the theatre. In these three buildings, the lifts opened straight onto a corridor or a shopping gallery. There was no lift lobby: a space in front of the lift where people can wait without blocking the way.
This problem came from the building we gave the agent, and the agent noticed. Its own check flagged the lift doors, and it read the plan correctly: no door can fix a missing lobby, because the walls have to change. The request was only to add doors and windows, so the agent didn't move walls on its own. It left the plan as it was, explained why and suggested next steps.
“The checker wants a lift lobby in front of each. That means changing the plan, so I've left it for you. I can carve a lobby out of each corridor or move the lift doors.”The agent, in the theatre
“Fixing that means changing the plan, so I've left it for you to decide.” Its first next step: “Add lift lobbies in front of the passenger and freight lifts.”The agent, in the mall
Figure 13. A lift lobby needs new walls, so the agent asked first. Left: the plan as given, and the kind of change that a lobby needs. Right: what the agent told the user at the end of each run.
The reviewer took points anyway, 66 lift doors in all, because the building can't be accepted as it is. But this is the answer we'd want from an architect: say what's wrong with the starting design, say what it takes to fix it, and ask before changing it.
What we are adding next
These checks are the first set for doors and windows. Architects judge doors and windows on a lot more than this, and the benchmark will too. From now on, we're releasing something new every week: new criteria for the parts of AECBench you already know, and entirely new parts of the benchmark.
View and privacy
Windows that look out to a good view. No clear glass where people need privacy, such as a toilet or two bedrooms that face each other.
Travel distance
How far a person walks from each room, through its door, to the nearest stair or exit. The doors must keep this walk short and clear in an emergency, without a path through other rooms.
Sunlight and daylight
Enough daylight in each room, and the right amount of sun. This criterion will also apply in other parts of AECBench, not only to windows.
Figure 14. What we are adding next. Every week we will release new criteria and new parts of AECBench.
Frequently asked questions
What is AECBench?
AECBench is Snaptrude's benchmark for measuring AI on real architectural work. It covers space planning, BIM detailing, drawings, services and value engineering. Doors and windows are one part of BIM detailing.
Can an AI agent add doors and windows to a BIM model?
Yes. From a one-sentence request, Snaptrude's agent added doors and windows to all 20 buildings, with one hour per building. Its mean score was 7.4 out of 10. The office tower, with 220 doors and 366 windows on 8 floors, scored 10.
How is the agent's work scored?
A reviewer LLM inspects every floor and scores the doors and windows from 1 to 10. It checks coverage, entrances, door type, swing and hinge side, width, window height, lift doors, fire-rated stair doors, the facade, and whether the agent's report matches the model. We fine-tuned it against four architects' reviews, and its agreement with them went from 0.12 to 0.48.
What mistakes did the agent make most often?
Window sills below 0.9 m were the most common problem, found in 6 buildings. Lifts with no lobby cost the most points. That problem came from the starting plans, and the agent flagged it instead of moving walls without asking.
Why build the agent inside the BIM model?
Because every door and window it adds is a real component with a width, a swing and a family, so the work can be checked and built on. We wrote more about this in why AI in architecture feels like more work, and what we did about it.
Review it yourself
The 20 buildings, the requests, the review instructions and every score are on GitHub. Open a building, look at what the agent did, and compare it with the reviewer's notes. If you disagree with a score, tell us. Architects' reviews made this reviewer better, and they're how it will keep getting better.
