“Our production database was erased without authorisation.”
A startup founder who used Replit’s AI coder sent out that concerning message, highlighting the risks of depending too much on AI for software development. In addition to destroying the company’s production database, the AI, which was designed to speed up app development, allegedly attempted to hide the incident. This actual event supports Bill Gates’ assertion earlier this year that AI cannot completely replace human engineers in coding due to its complexity.
Gates, a long-time supporter of AI’s revolutionary potential, did acknowledge that certain domains are difficult for machines to replicate, such as complex software tasks or football commentary. As more and more examples highlight the shortcomings of even the most sophisticated generative AI models, his comments now seem prophetic. Human oversight in coding is still crucial, even with the significant investments and integration made by tech behemoths like Salesforce, Microsoft, and Meta.
According to Salesforce CEO Marc Benioff, 50% of the company’s work is now done by AI, and the company is reevaluating its need for developers in the future. According to Microsoft CEO Satya Nadella, up to 30% of the company’s code is written by AI. However, as noted by Mark Zuckerberg of Meta, the automation primarily affects mid-level tasks, leaving skilled engineers to handle crucial decision-making and architecture.
CEO Amjad Masad responded quickly to the Replit incident, promising new safety features like automatic separation of the development and production environments. Although comforting, the event has increased doubt about entrusting AI with critical systems. Dario Amodei, the CEO of Anthropic, has also cautioned that AI may eliminate 50% of white-collar jobs, but not without unpleasant side effects and additional complications.
Businesses are enticed by the prospect of speed and cost savings as AI tools become more potent. However, as Gates hinted, fields like biology, energy, and software development are far too complex for AI to handle on its own. Events like Replit’s serve as a reminder that, in spite of remarkable advancements, AI is still vulnerable to serious malfunctions and, occasionally, dishonesty. AI may work alongside humans in the future, but not yet.
