Wikipedia has sent a clear message to AI developer companies: stop scraping its pages and start using its paid API service. The Wikimedia Foundation, which operates the free online encyclopedia, has urged AI developers to access its content responsibly through its paid Wikimedia Enterprise platform to ensure continued support for the website in the evolving digital age.
Wikipedia Sets Guidelines for AI Use
In a blog post released on Monday, the foundation said AI developers should properly credit Wikipedia’s contributors and obtain data through official channels rather than automated scraping. The Wikimedia Enterprise product allows large-scale use of Wikipedia’s content without overloading its servers. The foundation added that the revenue from the paid product supports its nonprofit operations, which help sustain the open knowledge platform that relies on volunteers and donations.
Wikipedia’s move comes after it discovered that AI bots had been scraping its website while disguising themselves as human users. After strengthening its bot detection systems, the organisation found that much of its increased traffic in May and June came from such bots attempting to bypass detection. During the same period, the site saw an 8% decline in visits from human readers, which raises concerns about reduced engagement from actual users.
Concerns Over Web Scraping and AI Bots
The foundation stated that it expects AI developers to give proper attribution to the human editors and contributors whose work forms the basis of AI-generated outputs. “For people to trust information shared online, platforms must make it clear where that information originates and highlight opportunities to visit and participate in those sources,” the blog noted.
Wikipedia warned that fewer visits could lead to fewer volunteers improving its content and fewer donors supporting its mission. Earlier this year, it also introduced an AI strategy aimed at assisting its editors with repetitive tasks, language translation, and workflow improvements, tools meant to support human editors rather than replace them.
Through these measures, the Wikimedia Foundation aims to maintain a fair balance between open access to knowledge and responsible use of its content in the fast-growing AI ecosystem.