AI app builder test v2: the exact prompt and checks (published before results)

This is the full plan for our second app-builder test of Lovable, Bolt, Base44 and v0. The prompt, rules, checks, scoring and tier cutoffs were fixed on 16 Oct 2026, before the first run. We publish them now, before any v2 results, so you can check that we didn't move the goalposts afterwards.

Why v2?

Test v1 ended in a near tie: Lovable, Bolt and Base44 scored 100 and v0 scored 95 (see the v1 results). A test every tool aces can't tell them apart, so v2 adds teams, roles, search and share links, and runs each tool 3 times instead of once. How we test in general: methodology.

The prompt (sent once, word for word)

Each run starts by pasting this exact text as one message:

Build a web app called "TeamTier", a shared task board for small teams.
Requirements:
1. Users sign up and log in with email and password, and log out.
2. A user can create a team and invite another user by email address. Invited users join the team after logging in.
3. Roles: the team creator is Owner. Owners can make members Admins or remove them. Members can create and edit tasks but cannot delete tasks or manage members. Admins can delete tasks.
4. Tasks have a title, description, status (To do, Doing, Done) and an optional assignee from the team.
5. Search tasks by title or description, and filter by status and assignee.
6. A task can be shared with a read-only public link that anyone can open without logging in. The owner can turn the link off, and it then stops working.
7. Users only ever see data from teams they belong to.
8. Data is saved in a database. It works on a phone-sized screen. Deploy it to a public URL.

Rules for every run

• Fresh project every run, on one normal public-signup account per tool.
• Free plan, default settings and the default model. We record the product or model version if the tool shows one.
• 60-minute hard stop, counted from the moment the prompt is sent.
• Up to 3 fix prompts, always in this form: "This doesn't work: [what we saw]. Please fix it." A tool's own one-click fix button counts as a fix prompt.
• No hand-edited code. If the tool offers its own built-in login or database, we accept the default and log every extra click.
• No paid top-ups or paid plans. If free credits run out, the run stops there and we record it.
• 3 runs per tool (N=3) on separate days. The rank uses each tool's median score, and we show the range (lowest to highest). If free credits allow fewer runs, we publish the real N and say why.
• Desktop checks run in a 1280×800 browser window; phone checks at 375×812.

The public checks

An agent runs these checks against each deployed app with five separate sessions: the Owner, an Admin, a Member, an outsider with their own account, and a logged-out visitor. Most checks are scripted.

Success checks (F1–F14)

1. F1Sign up, log in and log out with email and password.
2. F2Create a team.
3. F3Invite two other users by email. They join the team after logging in.
4. F4The Owner makes one of them an Admin.
5. F5Create a task with a title, description, status and assignee.
6. F6A Member can edit a task.
7. F7A Member cannot delete a task, either in the app or by replaying the delete request directly.
8. F8An Admin can delete a task.
9. F9Search finds tasks by words in the title and in the description.
10. F10Filter tasks by status and by assignee.
11. F11A task's share link opens while logged out and is read-only.
12. F12After the Owner turns the share link off, it stops working.
13. F13Data is still there after a page reload and in a fresh browser.
14. F14On a phone-sized screen (375×812), creating a task and filtering work with no sideways scrolling.

Security checks (S1–S5)

1. S1A logged-in outsider who is not on the team sees none of the team's data.
2. S2When we replay the Owner's list, update and delete requests using the outsider's login, the app refuses them.
3. S3When a Member tries to change roles by calling the app's API directly, the app refuses.
4. S4A share link shows only that one task. Its link or ID can't be used to find other tasks.
5. S5The code sent to the browser contains no secret keys (we search it for things like sk_ and service_role keys).

Hidden checks (H1–H3)

There are also 3 hidden checks worth 5 points. They were fixed with the rest of the plan, but their content stays private until results are published, so no tool can be tuned to pass them. We'll publish them alongside the v2 results.

Scoring (100 points)

Cost is reported separately (credits or dollars per run, and cost per successful run) and is not part of the score, unless a tool couldn't finish on its free plan (−10, noted). If the top tools land within about 5 points of each other, we'll say the test is saturated and plan v3.

Tier cutoffs

• S85+
• A70–84
• B55–69
• Cbelow 55

What we'll publish with the results

For every run: the timestamped prompt and fix-prompt log, screenshots of each check on desktop and phone (emails and passwords redacted), the security notes, bugs with steps to reproduce, credits used before and after, both UX ratings, and the deployed URL. Results follow the same evidence labels as the rest of the site. Payment never buys rank (disclosures).

How to cite

See the v1 results How we test