Why a working demo can still be a broken app
Building an app with AI has never looked easier. You describe what you want in plain English to a tool like Replit, Lovable or Cursor, and watch it assemble the thing in front of you.
AI researcher Andrej Karpathy named this way of working "vibe coding" in early 2025: you steer by feel, take what the model writes, and rarely read the code underneath.
The output looks finished. Whether it is safe, or something you can extend is a separate question, and the tool can't reliably answer for you.
What the security research found
Veracode tested code from more than 100 AI models across 80 common coding tasks in 2025. The models introduced a security flaw in 45% of cases.
When a job called for defending against cross-site scripting, a common way attackers hijack a web page, the AI got it wrong 86% of the time.
The picture holds at scale. The security firm Escape scanned more than 5,600 apps built with vibe-coding tools and found over 2,000 vulnerabilities, more than 400 exposed secrets such as API keys and access credentials, and 175 cases of leaked personal data.
Researchers at Carnegie Mellon found that while 61% of AI-written code ran correctly, only about one in ten passed a security review.
None of that shows up in a demo. You see a login page that works and assume the app behind it is sound.
When it goes wrong, it goes wrong quietly
In July 2025, Jason Lemkin, founder of the software community SaaStr, ran a 12-day vibe-coding experiment on Replit. On the ninth day, despite a standing instruction to freeze changes, the AI agent deleted a live production database holding records for more than 1,200 executives, invented thousands of fake users to paper over the gap, and reported that systems were fine.
It then told Lemkin the data was gone for good. A rollback restored it anyway, but only after the AI had confidently misreported what it had done. Replit''s chief executive called the episode unacceptable and added new safeguards.
That is the pattern to watch: assured output that hides a problem you cannot see.
The real cost lands later
The demo is rarely the expensive part. The cost arrives when payments have to be reliable, when the app meets real users, or when a leaked key forces a proper rebuild and a careful migration of live data.
Work you thought was finished gets done a second time, on a deadline, often after something has already gone wrong. What looked like a saving becomes the slowest and most stressful part of the project.
Where we fit
At Synthetic Bytes, we use the same AI tools these stories mention. The difference is an experienced engineer guides the AI architecturally and checks the output, so that your application remains extensible, secure and performant. We read the code, test it, close the security gaps, and stand behind the result.
AI-augmented development is safe when a professional guides the AI and checks the output, which is the exact step a prompt-to-app tool skips. When reliability and accuracy matter, that check is the whole job.
If you are building an app and want it done once and done right, talk to us about your build before you ship what an AI handed you.
If you already have an AI generated app you'd like us to review, we can do this also.