China's DeepSeek claims that the cheap V3 model offers performance comparable to the competition, but seems to be contradicted by an independent test.
HDN Newsroom
China's famous DeepSeek chatbot came second-to-last among 11 different models in presenting news and other useful information, a development that comes to extinguish enthusiasm for Chinese low-cost technology.
DeepSeek claims that the V3 model offers comparable performance to ChatGPT, Meta's Llama and Google's Gemini. But V3 failed to answer correctly in 83% of tests conducted by NewsGuard, a company that rates news media credibility.
The chatbot reproduced fake news in 30% of cases and gave vague or unhelpful answers in 53% of questions, meaning it answered correctly in only 53%.
This is significantly lower than the average of 62% achieved by ten US models, including Gemini and ChatGPT.
DeepSeek became world famous overnight when it unveiled the V3, which it says was trained with an Nvidia chip at a total cost of just $6 million. dollars, well below the tens or hundreds of millions needed to train other large models.
The company also received enthusiastic feedback on its reasoning model R1, designed to mimic human logic, which costs about 30 times less to use than OpenAI's equivalent o1 model.
The v3 mobile app soon became the most popular app for Apple devices, but markets began to worry that the astronomical spending of American AI companies is not justified.
Companies like Nvidia and Meta plunged in stock markets and shares of U.S. tech companies lost a trillion. dollars before they begin to recover.
NewsGuard said it tested about 300 instructions, or prompts, on DeepSeek's chatbot, including 30 instructions referring to 10 fake news stories spreading online, including about the killing of a United Healthcare executive last month.
According to the company, in three of these ten prompts the system gave answers that are in line with the official positions of the Chinese government.
In China, AI companies are obliged to ensure that their models respect the official positions of the People's Republic (i.e. communist) government.
