AI Developers Are Already Training AI Models With AI-Generated Data, Reports Say

The enormous amount of data required to train these models has enabled the area of artificial intelligence (AI) to make tremendous strides in recent years. Historically, human-curated datasets have served as the foundation for AI model construction. A new pattern, however, is starting to emerge where some businesses are experimenting with data that is produced by AI itself. This method, dubbed “synthetic data,” could transform the AI ecosystem if it is successful in making training large language models (LLMs) more affordable and scalable.

The exploration of the potential of synthetic data is being led by organizations like OpenAI, Microsoft, and Cohere, a promising startup valued at $2 billion. The main benefit is cost-effectiveness, since using AI to generate data turns out to be much less expensive than relying on human-generated data. Additionally, employing synthetic data solves the scalability problem that AI engineers, who are always looking for new data to improve their models, encounter. Despite being a massive source of information, the internet is frequently too chaotic and noisy to be fully effective for AI training, as Cohere CEO Aiden Gomez noted.

Despite the allure of synthetic data, critics raise valid concerns about its integrity and reliability. Even AI models trained on human-generated data are not immune to factual errors and mistakes. When AI-generated data is introduced, there is a risk of creating feedback loops and “irreversible defects,” as pointed out by researchers from Oxford and Cambridge in a recent paper. These issues cast a shadow on the reliability of AI models trained solely on synthetic data.

The ultimate moonshot for companies like Cohere is to achieve self-teaching AI models capable of generating their own synthetic data. This vision entails models that can ask insightful questions, discover novel truths, and create new knowledge autonomously. Although this may seem like a distant dream, the progress made in the realm of synthetic data is propelling the industry towards more sophisticated and autonomous AI systems.

While synthetic data holds promise, the cautious integration of human-generated and AI-generated data might provide a balanced solution. A hybrid approach could ensure better validation and accuracy, minimizing the risks associated with using solely AI-generated data. Striking this balance will be crucial in establishing a robust and trustworthy AI ecosystem.

As the AI landscape continues to evolve, the adoption of synthetic data remains an ongoing experiment. Companies like OpenAI, Microsoft, and Cohere are pioneering this domain, pushing the boundaries of what AI can achieve. However, ethical considerations, data integrity, and the potential for unforeseen consequences must be carefully addressed as we traverse this uncharted territory. The future of AI development lies in striking the right balance between human and AI-generated data to ensure reliable, innovative, and self-evolving AI systems that truly live up to the dream.