Published 22 Oct 2026 · our column updated 26 Oct 2026 · each site's own pages read 22 Oct 2026 · methods only, not a ranking of sites

Star ratings vs receipts: what a 3.5/5 can't tell you about an AI app builder

More sites now score AI app builders, and their numbers look alike: stars, points out of 100, tiers. The methods behind them are not alike. Here is what four sites, ours included, say they actually do, and the six questions worth asking behind any score.

The short answer: a score is a summary, so check what it summarises. App Builder Index says its star ratings come from its own panel running 6 briefs 3 times per tool on paid plans. AI Form Guide labels every current score "Preliminary · Reviewed from docs". AI Coding Hub (coding tools) publishes the commands, version strings and dates it ran Verified, each site read 22 Oct 2026. VibeTierList builds one app per tool on free plans and publishes the screenshots and credits used, but our first test was one run per tool, so each result is a single observation. Before you trust any number, look for: tested or read, which plan, how many runs, when, evidence you can open, and who pays the site.

Six questions to ask behind any score

1. Was it tested, or read?

A score built from a vendor's docs and pricing page tells you what the tool promises. A score built from runs tells you what it did on the day. Both are useful, but they answer different questions. Look for a label that says which one you're reading.

2. Which plan?

Free and paid plans can behave differently: credit caps, models, features behind a paywall. A rating from paid plans may not match what you get on a free account, and a free-plan test may undersell a paid one.

3. How many runs, and how far apart?

AI builders don't give the same result twice. One run is one data point. Several runs with a median and a range tell you how much the result moves.

4. When?

These tools ship changes weekly. A score with no test date, or with a date months old, may describe a different product from the one you'd sign up for today.

5. Can you see what happened?

A 3.5/5 or an 82/100 compresses everything into one number. Screenshots, logs, the deployed app and what it cost let you check the number, and find the one bug that matters for your app even when the total looks fine.

6. Who pays the site?

Affiliate commissions, sponsor slots and paid reviews are all common and can be fine when they're labelled. Read the disclosure and check whether payment can touch the score.

How each site says it tests

Each cell is what that site's own pages said when we read them on 22 Oct 2026. This compares methods, not accuracy: we haven't checked anyone's scores, and none of these sites paid us or was paid by us. AI Coding Hub covers coding tools rather than app builders; it's here because it is the other hands-on site we track.

QuestionApp Builder IndexAI Form GuideAI Coding HubVibeTierList (us)
What it covers33 AI app buildersNew AI tools, for builders and for everyday professional jobsAI coding tools4 AI app builders tested so far (Lovable, Bolt, Base44, v0)
Hands-on or from docsHands-on: its own testing panel, two testers per toolEvery current score says "Preliminary · Reviewed from docs"; entries will be marked "Tested hands-on" once testedHands-on, on its own repositoriesHands-on: an agent builds the same app in each tool and checks it in a real browser
Tasks and runs per tool6 briefs × 3 runs = 18 runs per toolNone yet (docs review)The same set of tasks for every tool; run count not stated on the page we readv1: 1 task × 1 run per tool. v2 (results published 25 Oct 2026): 1 harder task × 3 runs, median and range
Plan testedPaid plans ("free tiers behave differently")Not applicable yet"Paid where needed"Free plans only, default settings
Score format0–5 stars on 10 weighted axes, one decimal0–100 on 7 criteriaWeighted score on 6 dimensions; "in progress" until tested0–100, tiers S/A/B/C from the v2 median; v1 scores carry no tier
What you can check on the pageAxis scores, a dissenting view, a landing-page screenshot, dated sources, raw axis scores as JSON (CC BY 4.0). Per-run logs: not yet; the site said on 5 Sep 2026 it is working on publishing themScore breakdown per criterion, pricing with the date checkedExact commands, the version string the tool printed and the date runScreenshots of key checks, credits used before and after, time to live app, the deployed app URL
Dates shown"Last verified" and "next scheduled check" (90-day cycle) on each review"Checked" date on each tool page (15 Oct 2026 on the one we read)"Last updated" and "last verified" dates; most recent pass 4 Sep 2026Test date on every result; stale after 90 days; date we last checked for changes
Money, as the site describes itBuys its own subscriptions; no placements or paid rating changes; no affiliate links currentlySells labelled sponsor slots; may earn affiliate commission; says the score never reads paymentNo affiliate links, sponsorships or paid placementsSells $39 paid tests (same task, labelled) and labelled sponsor placements; no affiliate links yet; payment never buys rank

Sources, all read 22 Oct 2026: App Builder Index how we review, its Lovable review (4.2/5, last verified 20 Aug 2026; the 5 Sep 2026 reply about raw logs is on this page) and rankings JSON · AI Form Guide home, how we score, disclosure (updated 18 Oct 2026) and its v0 page (76/100, reviewed from docs, checked 15 Oct 2026) · AI Coding Hub how we test (updated 4 Sep 2026) · ours: methodology and disclosures (updated 16 Oct 2026). Sites change their methods; check the source before you cite it.

The pattern: the three sites that test hands-on (App Builder Index, AI Coding Hub and us) agree on the basics (same tasks for every tool, dated results, scores that payment can't move), and they differ mostly in depth and in what they show you. App Builder Index states the most runs per tool (18), on paid plans. AI Coding Hub shows the exact commands and versions. We show the screenshots and the free-plan credits from each run. A docs-based score, like AI Form Guide's current issue, is a different kind of number: a reading of what the vendor publishes, clearly labelled as such Inferred.

What a top score hid in our own test

In our v1 test (16 Oct 2026, free plans, one run each) Bolt's app scored 100/100. It also had edit and delete buttons that only appeared on mouse hover, so on a phone they were invisible Observed, 16 Oct 2026 · single observation (screenshot). A perfect total didn't tell you that; the receipt did. If your users are on phones, that one bug matters more than the score. The same is true of anyone's number, ours included: open the evidence behind the score that decides your choice.

All four v1 apps, with their screenshots and credits used: Lovable vs Bolt vs Base44 vs v0 · what one working app cost on each free plan.

Where our method is weaker

  • One run per tool in v1. Every v1 result is a single observation, and AI builders vary from run to run. App Builder Index's 18 runs per tool is a much bigger sample.
  • Free plans only. That matches what most people try first, but paid plans can behave differently, so our results may not describe a paid account.
  • One fairly simple app, 4 builders. v1 couldn't separate the tools (95 to 100 each), and we haven't tested the other builders other sites cover.
  • Agent-run, not a human panel. An agent runs the builds and the checks. That keeps every run identical, but no human designer or developer rates the result in v1.
  • The fix: our harder v2 test (teams, roles, share links, search) ran each tool 3 times under a plan we published before any results, and reports a median and range. v2 results published 25 Oct 2026 (other sites' methods were read 22 Oct 2026; our own column was updated for v2 on 26 Oct 2026).

Honest limits of this page

  • This compares what each site's own pages say about its method, read 22 Oct 2026. We haven't audited anyone's tests or checked their scores, so this says nothing about whose numbers are more accurate.
  • "Not stated", "not yet" or "none yet" means the pages we read didn't show it; it may exist elsewhere on the site.
  • We run one of the four sites in the table, so we have an interest. We've tried to list our own weak spots as plainly as anyone's method. No site in the table paid us or was paid by us.

Related

Evidence labels: Verified = read on that site's own page on 22 Oct 2026 (links above). Observed = seen in our own v1 run on 16 Oct 2026. Inferred = our reading of the evidence. What the labels mean.

FAQ

What can a single star rating or score out of 100 not tell you about an AI app builder?

On its own it can't tell you whether the tool was actually used or rated from its docs, which plan was tested, how many runs the number rests on, when it was tested, what broke, or what it cost. Two tools with the same score can fail in completely different places. Check the method page and the evidence behind the number before you rely on it.

Are App Builder Index ratings based on hands-on tests?

Its methodology page (read 22 Oct 2026) says yes: its own testing panel runs six fixed briefs three times each per tool, from a clean account, with two testers, on paid plans, and scores ten weighted axes out of five stars. Panel ratings are labelled Editor panel and community ratings are shown separately, not blended in.

Is AI Form Guide's score a hands-on test?

Not yet. Its homepage (read 22 Oct 2026) says every score in its first issue is preliminary and reviewed from public documentation, and that nothing on the page has been through its hands-on test yet. Its scoring page says each entry will be marked either Tested hands-on or Reviewed from docs.

How is VibeTierList's method different, and where is it weaker?

We build the same app in each tool on its free plan and publish screenshots, credits used and the deployed app for each run. The weak spots: our first test (v1, 16 Oct 2026) was one run per tool on one fairly simple app, so every v1 result is a single observation; we test free plans only; we cover 4 app builders, not 33; and v1 scores (95 to 100) were too close to separate the tools. Our harder v2 test ran each tool 3 times under a plan published before results, and reports a median and range; its results were published 25 Oct 2026.

Do these review sites take money from tool makers?

By their own pages, read 22 Oct 2026: App Builder Index says it sells no placements and runs no affiliate links. AI Form Guide sells labelled sponsor slots and may earn affiliate commissions, and says payment is never read by its score. AI Coding Hub says it has no affiliate links, sponsorships or paid placements. VibeTierList sells $39 paid tests (same task and scoring, labelled) and labelled sponsor placements, has no affiliate links yet, and payment never buys rank.

Which AI app builder review site should I trust?

Use more than one, and read each for what it is good at. Look for a test date, the plan tested, the number of runs, evidence you can open and a clear money disclosure. Where two sites that test differently agree, that is a stronger signal than either score alone. If you're about to ship, test your own app too: the two-account test takes about 10 minutes.

How to cite

VibeTierList, "Star ratings vs receipts: what a 3.5/5 can't tell you about an AI app builder", published 22 Oct 2026, each site's pages read 22 Oct 2026. https://vibetierlist.com/guides/app-builder-review-methods/