{
 "dataset": "VibeTierList prompt-to-app v2 results",
 "version": "2026-10-25",
 "published": "2026-10-25",
 "updated": "2026-10-25",
 "data_as_of": "24 Oct 2026",
 "url": "https://vibetierlist.com/data/v2/prompt-to-app-v2.json",
 "dated_snapshot": "https://vibetierlist.com/data/v2/prompt-to-app-v2-2026-10-25.json",
 "csv": "https://vibetierlist.com/data/v2/prompt-to-app-v2-runs.csv",
 "results_page": "https://vibetierlist.com/results/prompt-to-app-v2/",
 "plan": "https://vibetierlist.com/tests/prompt-to-app-v2/",
 "plan_fixed": "16 Oct 2026",
 "license": {
  "name": "CC BY 4.0",
  "url": "https://creativecommons.org/licenses/by/4.0/",
  "credit": "Source: VibeTierList, \"AI app builder test v2 results\" (data as of 24 Oct 2026), https://vibetierlist.com/results/prompt-to-app-v2/",
  "terms": "Free to share and adapt, including commercially, if you give the credit above with a link and say if you changed anything. Please do not present edited scores as ours."
 },
 "scope": "Only the published v2 results: per-tool medians and tiers, every scored run, voided attempts, disclosures and receipt links. Scan my app data and any private account data are excluded. No vendor paid for this test; payment never buys rank.",
 "method": {
  "runs_per_tool": 3,
  "checks_per_run": 22,
  "checks": "F1–F14 success (14), S1–S5 security (5), H1–H3 hidden (3)",
  "score": "0–100 per run from the pre-registered rubric; tool rank uses the median of its 3 runs",
  "tier_cutoffs": {
   "S": "85+",
   "A": "70–84",
   "B": "55–69",
   "C": "below 55"
  },
  "plan": "free plan, default settings, no paid top-ups; a run that ran out of free credits before deploying is scored \"did not finish on Free\" (0) and counted in the median"
 },
 "confidence_labels": {
  "single": {
   "label": "Single observation",
   "means": "Seen once: one run, one tool version. Worded \"in our run…\"."
  },
  "repeated": {
   "label": "Repeated result",
   "means": "Seen in 2 or 3 of 3 runs under the same conditions. Worded \"in 2 of 3 runs…\"."
  },
  "benchmark": {
   "label": "High-confidence benchmark",
   "means": "A tool's median and tier from 3 consistent runs under the pre-registered plan, with versions recorded and receipts public. No confidence interval, because N is below 5."
  }
 },
 "short_answer": "In our v2 test (the same team-app prompt, 3 runs per tool on free plans, 22 checks per run), Base44 came out on top: it built a working team task app in 2 of 3 runs (median 90.67, tier S). Lovable had the best single run (97.5), but one build never opened its team board and one ran out of free credits, so its median is 42.23 (tier C) and unstable. On Bolt's free plan, 2 of 3 runs ran out of tokens before anything deployed (median 0, tier C). No tool earned our High-confidence benchmark label, because none had three consistent runs.",
 "tools": [
  {
   "id": "lovable",
   "name": "Lovable",
   "plan": "Free",
   "median": 42.23,
   "range": [
    0,
    97.5
   ],
   "tier": "C",
   "confidence": "Single observation",
   "verdict": "The best single run in the test and the widest spread. Run 1 passed all 22 checks (97.5). In run 2 the team board never opened (42.23), and run 3 ran out of free credits before anything deployed (0). The median of 42.23 is unstable: three runs gave three different outcomes, so read Lovable's tier C as a weak result, not a settled one.",
   "results_url": "https://vibetierlist.com/results/prompt-to-app-v2/#broke-lovable",
   "runs": [
    {
     "run": 1,
     "date_utc": "17 Oct 2026",
     "status": "scored",
     "score": 97.5,
     "checks_passed": 22,
     "success_passed_of_14": 14,
     "security_passed_of_5": 5,
     "hidden_passed_of_3": 3,
     "time_to_live": "5 min",
     "fix_prompts": 0,
     "free_allowance_used": "1.80 credits (5 before, 3.20 after)",
     "version_shown": "not shown",
     "model_shown": "not shown",
     "receipt": "https://vibetierlist.com/receipts/v2/lovable/run-1/receipt.json",
     "what_broke": [
      {
       "checks": null,
       "text": "Nothing failed: 22 of 22 checks passed on the first deploy, about 5 minutes after the prompt, with 0 fix prompts.",
       "confidence": "Single observation"
      }
     ]
    },
    {
     "run": 2,
     "date_utc": "19 Oct 2026",
     "status": "scored",
     "score": 42.23,
     "checks_passed": 9,
     "success_passed_of_14": 2,
     "security_passed_of_5": 4,
     "hidden_passed_of_3": 3,
     "time_to_live": "did not reach",
     "fix_prompts": 0,
     "free_allowance_used": "6.90 credits (5 daily + 1.90 from balance)",
     "version_shown": "not shown",
     "model_shown": "not shown",
     "receipt": "https://vibetierlist.com/receipts/v2/lovable/run-2/receipt.json",
     "what_broke": [
      {
       "checks": "F3–F14",
       "text": "After creating a team, the team board never opened: the team's own page kept showing the \"Your teams\" list, on desktop and at 375×812. Invites, roles, tasks, search, filters and share links all live on that board, so none of them could be used from the app.",
       "confidence": "Single observation"
      },
      {
       "checks": "S2",
       "text": "An outsider (not on the team) could call the app's set_task_sharing function on the owner's task, got a share token back, and that link then opened for logged-out visitors.",
       "confidence": "Single observation"
      }
     ]
    },
    {
     "run": 3,
     "date_utc": "21 Oct 2026",
     "status": "did not finish on Free (scored 0, counted)",
     "score": 0,
     "checks_passed": 0,
     "success_passed_of_14": 0,
     "security_passed_of_5": 0,
     "hidden_passed_of_3": 0,
     "time_to_live": "did not reach",
     "fix_prompts": 0,
     "free_allowance_used": "all 5.20 credits left; paused out of Free credits ~11 min in",
     "version_shown": "not shown",
     "model_shown": "not shown",
     "receipt": "https://vibetierlist.com/receipts/v2/lovable/run-3/receipt.json",
     "what_broke": [
      {
       "checks": "all 22",
       "text": "Paused out of free credits about 11 minutes after the prompt (it started with 5.20 credits). Only a static landing page had been built and nothing was deployed, so there was no app to test. Did not finish on Free: scored 0.",
       "confidence": "Single observation"
      }
     ]
    }
   ],
   "voided_attempts": [
    {
     "run": 3,
     "date": "20 Oct 2026 UTC (evening of 19 Oct ET)",
     "reason": "First attempt at run 3 voided: tester-side stop while Lovable was waiting on us to approve publishing, not a Lovable failure. It used the leftover credit balance (see disclosures). Run 3 was redone on a fresh project on 21 Oct.",
     "counted": false
    }
   ]
  },
  {
   "id": "bolt",
   "name": "Bolt",
   "plan": "Free",
   "median": 0,
   "range": [
    0,
    29.04
   ],
   "tier": "C",
   "confidence": "Repeated result",
   "verdict": "On the Free plan, 2 of 3 runs hit Bolt's \"You're out of tokens\" wall before anything deployed (0 each). The one run that deployed (29.04) never let a user see a team they had created, because of a database policy error. Median 0, tier C.",
   "results_url": "https://vibetierlist.com/results/prompt-to-app-v2/#broke-bolt",
   "runs": [
    {
     "run": 1,
     "date_utc": "19 Oct 2026",
     "status": "scored",
     "score": 29.04,
     "checks_passed": 4,
     "success_passed_of_14": 1,
     "security_passed_of_5": 3,
     "hidden_passed_of_3": 0,
     "time_to_live": "did not reach",
     "fix_prompts": 0,
     "free_allowance_used": "~100K tokens (799K to 699K monthly)",
     "version_shown": "not shown",
     "model_shown": "not shown (selector 'Standard', during Bolt's GLM 5.3 Flash free weekend)",
     "receipt": "https://vibetierlist.com/receipts/v2/bolt/run-1/receipt.json",
     "what_broke": [
      {
       "checks": "F2–F14, H1–H3",
       "text": "With no team yet, the only \"New team\" control on desktop sat inside a team switcher that stays disabled until you have a team. Creating one from the phone menu saved it (201), but every read of teams, members and tasks then failed with a database error (500, \"infinite recursion detected in policy for relation team_members\"), so the team never appeared and no invite, task or share link could be made.",
       "confidence": "Single observation"
      },
      {
       "checks": "S2",
       "text": "Any signed-in user could read every account's profile row, including email addresses, even with no team in common.",
       "confidence": "Single observation"
      },
      {
       "checks": "S4",
       "text": "No share link could be created, so share-link security could not be confirmed; counted as a fail.",
       "confidence": "Single observation"
      }
     ]
    },
    {
     "run": 2,
     "date_utc": "22 Oct 2026",
     "status": "did not finish on Free (scored 0, counted)",
     "score": 0,
     "checks_passed": 0,
     "success_passed_of_14": 0,
     "security_passed_of_5": 0,
     "hidden_passed_of_3": 0,
     "time_to_live": "did not reach",
     "fix_prompts": 0,
     "free_allowance_used": "~100K tokens; 'out of tokens' wall ~4.5 min in",
     "version_shown": "not shown",
     "model_shown": "not shown (selector 'Standard')",
     "receipt": "https://vibetierlist.com/receipts/v2/bolt/run-2/receipt.json",
     "what_broke": [
      {
       "checks": "all 22",
       "text": "Bolt showed \"You're out of tokens\" about 4.5 minutes after the prompt, during its own build check, while its settings still showed 199k of 300k daily tokens left. About 100K tokens were used. Nothing deployed, so there was no app to test. Did not finish on Free: scored 0.",
       "confidence": "Repeated result"
      }
     ]
    },
    {
     "run": 3,
     "date_utc": "24 Oct 2026",
     "status": "did not finish on Free (scored 0, counted)",
     "score": 0,
     "checks_passed": 0,
     "success_passed_of_14": 0,
     "security_passed_of_5": 0,
     "hidden_passed_of_3": 0,
     "time_to_live": "did not reach",
     "fix_prompts": 0,
     "free_allowance_used": "~101K tokens (199k of 300k daily left at start); 'out of tokens' wall ~12 min in",
     "version_shown": "not shown",
     "model_shown": "not shown (selector 'Standard')",
     "receipt": "https://vibetierlist.com/receipts/v2/bolt/run-3/receipt.json",
     "what_broke": [
      {
       "checks": "all 22",
       "text": "Bolt showed \"You're out of tokens\" about 12 minutes after the prompt, after about 101K tokens (it started with 199k of 300k daily tokens left; see disclosures). The preview never loaded and nothing was published. Did not finish on Free: scored 0.",
       "confidence": "Repeated result"
      }
     ]
    }
   ],
   "voided_attempts": [
    {
     "run": 1,
     "date": "17 Oct 2026",
     "reason": "First attempt at run 1 voided: tester-side interruption, not a Bolt failure. Run 1 was redone from a fresh project.",
     "counted": false
    },
    {
     "run": 3,
     "date": "23 Oct 2026 ET (started 8:13pm ET, which is 24 Oct 00:13 UTC)",
     "reason": "First attempt at run 3 voided: tester-side stall before publish (the build was waiting on us), not a Bolt failure. Run 3 was redone on a fresh project on 24 Oct.",
     "counted": false
    }
   ]
  },
  {
   "id": "base44",
   "name": "Base44",
   "plan": "Free",
   "median": 90.67,
   "range": [
    37.93,
    94.17
   ],
   "tier": "S",
   "confidence": "Repeated result",
   "verdict": "Built a working team task app in 2 of 3 runs (94.17 and 90.67, all 14 success checks both times). In run 2 the owner couldn't open the team they had just created (37.93). Run 3 had one security fail: a Member could make themselves admin through the app's data API.",
   "results_url": "https://vibetierlist.com/results/prompt-to-app-v2/#broke-base44",
   "runs": [
    {
     "run": 1,
     "date_utc": "17 Oct 2026",
     "status": "scored",
     "score": 94.17,
     "checks_passed": 22,
     "success_passed_of_14": 14,
     "security_passed_of_5": 5,
     "hidden_passed_of_3": 3,
     "time_to_live": "15 min",
     "fix_prompts": 0,
     "free_allowance_used": "1 message credit (1 to 2 of 25)",
     "version_shown": "not shown",
     "model_shown": "not shown",
     "receipt": "https://vibetierlist.com/receipts/v2/base44/run-1/receipt.json",
     "what_broke": [
      {
       "checks": null,
       "text": "Nothing failed: 22 of 22 checks passed, with 0 fix prompts. First deploy passing all 14 success checks took about 15 minutes.",
       "confidence": "Single observation"
      }
     ]
    },
    {
     "run": 2,
     "date_utc": "19 Oct 2026",
     "status": "scored",
     "score": 37.93,
     "checks_passed": 6,
     "success_passed_of_14": 2,
     "security_passed_of_5": 4,
     "hidden_passed_of_3": 0,
     "time_to_live": "did not reach",
     "fix_prompts": 0,
     "free_allowance_used": "not recorded in the score sheet",
     "version_shown": "not shown",
     "model_shown": "not shown (Auto)",
     "receipt": "https://vibetierlist.com/receipts/v2/base44/run-2/receipt.json",
     "what_broke": [
      {
       "checks": "F3–F14, S4, H1–H3",
       "text": "Right after creating a team, the owner's team board and settings showed \"You don't have access to this team.\" (the app's getTeam call returned \"Not available for the User entity\"). Invites and new tasks were refused with 403 errors, so 16 of the 22 checks failed. The security checks that could still be run (S1, S2, S3, S5) passed.",
       "confidence": "Single observation"
      }
     ]
    },
    {
     "run": 3,
     "date_utc": "22 Oct 2026",
     "status": "scored",
     "score": 90.67,
     "checks_passed": 21,
     "success_passed_of_14": 14,
     "security_passed_of_5": 4,
     "hidden_passed_of_3": 3,
     "time_to_live": "20 min",
     "fix_prompts": 0,
     "free_allowance_used": "1 message credit (5.76 to 6.76 of 25)",
     "version_shown": "not shown",
     "model_shown": "not shown (Auto, Build mode)",
     "receipt": "https://vibetierlist.com/receipts/v2/base44/run-3/receipt.json",
     "what_broke": [
      {
       "checks": "S3",
       "text": "A Member could make themselves admin by editing their own team-membership record through the app's data API (the request returned 200), after which Delete and the share switch appeared for them. A Member could also add themselves to a task's admins and then delete that task. The plain delete attempt was refused, so F7 passed; the bypass is scored under S3.",
       "confidence": "Single observation"
      },
      {
       "checks": "Not scored",
       "text": "Any signed-in user, even one outside the team, could add a task to any team through the API, and it showed up on that team's board. None of the 22 checks fixed on 16 Oct covers creating tasks in someone else's team, so this doesn't change the score.",
       "confidence": "Single observation"
      }
     ]
    }
   ],
   "voided_attempts": [
    {
     "run": 3,
     "date": "21 Oct 2026 UTC (evening of 20 Oct ET)",
     "reason": "First attempt at run 3 voided: tester-side stall before publish, not a Base44 failure. Run 3 was redone on a fresh app on 22 Oct.",
     "counted": false
    }
   ]
  }
 ],
 "not_in_v2": [
  {
   "id": "v0",
   "name": "v0",
   "reason": "v0 is not in v2. Its free credit is a one-time $5 for new sign-ups and $3.81 of it went on our v1 run (16 Oct 2026), so a v2 run would have needed a paid top-up, which v2 does not allow. We decided on 19 Oct 2026 to leave it out. Its v1 result stays up, dated, and is not ranked against the v2 medians.",
   "v1_result": "https://vibetierlist.com/review/v0/"
  }
 ],
 "hidden_checks": [
  {
   "id": "H1",
   "text": "Removed member: once the owner removes someone from the team, that person can no longer see or edit the team's tasks, even in a browser where they were already logged in. We checked in the app and by replaying the app's own requests."
  },
  {
   "id": "H2",
   "text": "Invite to someone new: inviting an email address that has no account yet works without an error, and that person joins the team after they sign up."
  },
  {
   "id": "H3",
   "text": "Share links stay off: a share link that was turned off stays off after sharing is turned on for a different task, and the link can't be guessed from the task's number."
  }
 ],
 "findings": [
  {
   "text": "One build per tool can mislead. The same prompt gave Lovable 97.5 on 17 Oct and 42.23 on 19 Oct, and Base44 94.17 and then 37.93. A single test run would have put either tool at S or at C.",
   "confidence": "Repeated result"
  },
  {
   "text": "In 3 of the 9 scored runs (Lovable run 2, Base44 run 2, Bolt run 1) the app deployed, but its main screen was out of reach right after the user created a team.",
   "confidence": "Repeated result"
  },
  {
   "text": "On Bolt's Free plan this prompt hit the \"You're out of tokens\" wall after about 100K tokens in 2 of 3 runs, once while Bolt's own settings still showed 199k of 300k daily tokens left.",
   "confidence": "Repeated result"
  },
  {
   "text": "Replaying requests as a second user found problems that clicking through would miss: an outsider could switch on a share link for the owner's task in Lovable run 2, and a Member could make themselves admin in Base44 run 3.",
   "confidence": "Single observation"
  }
 ],
 "variance": {
  "tool": "lovable",
  "text": "best run 97.5 (17 Oct, 22 of 22 checks), worst 0 (21 Oct, ran out of free credits before deploying). Base44 also swung: 94.17, then 37.93, then 90.67."
 },
 "disclosures": [
  {
   "id": "bolt-void",
   "text": "Bolt run 1: the first attempt was voided because of a tester-side interruption, and run 1 was redone from a fresh project. The voided attempt is not scored."
  },
  {
   "id": "bolt-model",
   "text": "Bolt doesn't show which model it uses. Its model selector read 'Standard' in all three runs. The run 1 redo (19 Oct 2026, 00:53 UTC) was built while bolt.new showed a 'GLM 5.3 Flash: free this weekend' banner; that offer ended 18 Oct 2026, 11:59pm PT, before runs 2 and 3. We can't confirm the model changed, but conditions may have, so any Bolt finding that splits run 1 from runs 2 and 3 is labelled Single observation."
  },
  {
   "id": "v0",
   "text": "v0 is not in v2. Its free credit is a one-time $5 for new sign-ups and $3.81 of it went on our v1 run (16 Oct 2026), so a v2 run would have needed a paid top-up, which v2 does not allow. We decided on 19 Oct 2026 to leave it out. Its v1 result stays up, dated, and is not ranked against the v2 medians."
  },
  {
   "id": "no-topups",
   "text": "No paid top-ups or paid plans. Every run used the free plan with default settings; if free credits ran out, the run stopped and we recorded it."
  },
  {
   "id": "dnf-on-free",
   "text": "Lovable run 3 (21 Oct), Bolt run 2 (22 Oct) and Bolt run 3 (24 Oct) ran out of free credits before the app was deployed. We scored all three as \"did not finish on Free\": every check fails because there was no app to test, and the 10-point cost penalty applies, so each run scores 0. They are counted in the median, not voided."
  },
  {
   "id": "bolt-r3-balance",
   "text": "Bolt run 3 (24 Oct) started with 199k of its 300k daily free tokens, because the voided 23 Oct attempt had used about 100K from the same window. It used about 101K before Bolt showed an out-of-tokens wall, about 12 minutes in. Bolt run 2 hit the same wall after a similar amount with a full daily allowance."
  },
  {
   "id": "lovable-r3-balance",
   "text": "Lovable run 3 started with 5.20 credits, not the 8.20 run 2 had, because an earlier voided attempt on 20 Oct had used the leftover balance. A full daily allowance is 5 credits, and run 2 needed 6.90."
  },
  {
   "id": "dates",
   "text": "Run dates are UTC dates (the date of each run's folder in our logs). Most runs started in the evening, US Eastern time, so they show the next day: for example Base44 run 3 (22 Oct UTC) started at 8:20pm ET on 21 Oct. The one exception is the voided first attempt at Bolt run 3, listed under its ET date, 23 Oct (it started 8:13pm ET, which is 24 Oct 00:13 UTC). The Bolt run 3 redo started 24 Oct at 7:12am ET."
  },
  {
   "id": "no-money",
   "text": "No sponsor or affiliate relationship with Lovable, Bolt or Base44 as of the publish date. No vendor paid for anything on this page. Payment never buys rank."
  }
 ],
 "screenshots_note": "Redacted screenshots, where they exist, are linked from the results page; others follow as they are redacted.",
 "corrections": "Spotted a mistake? hello@vibetierlist.com. Corrections are made publicly, with the date."
}