Of course the writer says they the authors work for Apple so the paper can’t be good 🙄
“Testing models' on the benchmark that includes irrelevant information ‘exposes a critical flaw in LLMs’ ability to genuinely understand mathematical concepts and discern relevant information for problem-solving.’”
1/2
https://mashable.com/article/apple-study-reveals-major-ai-flaw-openai-google-meta-llms
2/2
The common misconception of engineering is that we are out to denigrate the people we are in competition with: no, that is marketing.
Engineering is about risk management. The greater the risk the more likely you will lose your job. Testing AI is important because too many people want to believe it is a magic bullet that will solve all ills.
It isn’t.
This paper is important to explain there is little reasoning in AI.
Most engineers have to fight management’s bad ideas.
This helps.