Lukasz 谈到前沿模型(frontier models)性能的“极度不均衡”时说: “你可以说这个模型在数学奥林匹克方面表现惊人。但与此同时,我有一个一年级的小孩……她才五岁。我从她的数学书里随便拿了一个练习题,结果没有任何一个前沿大模型能够解出来。” 题目是什么? 两个点阵,其中有一个点是共享的,然后你问它:总数是奇数还是偶数? “有趣的是,如果你让模型思考更久,它可能真的能做出来……但需要 15 分钟。而一个五岁的小孩只需要 15 秒。” Lukasz on how jagged frontier model performance really is: “You can say the model is amazing at mathematical Olympiad. And at the same time, I have a first grader… she's five years old. I took one exercise from this math book and none of the frontier models is able to solve it.” The task? Two groups of dots with one shared dot, and you ask if the total is even or odd. “The nice thing is if you let it think longer, it may actually solve it… but it takes 15 minutes. The human five year old takes 15 seconds.”

Matt TurckPartner at FirstMark Capital; Host of MAD (ML/AI/Data) Conference

观点原文

Lukasz 谈到前沿模型(frontier models)性能的“极度不均衡”时说:

“你可以说这个模型在数学奥林匹克方面表现惊人。但与此同时,我有一个一年级的小孩……她才五岁。我从她的数学书里随便拿了一个练习题,结果没有任何一个前沿大模型能够解出来。”

题目是什么?

两个点阵,其中有一个点是共享的,然后你问它:总数是奇数还是偶数?

“有趣的是,如果你让模型思考更久,它可能真的能做出来……但需要 15 分钟。而一个五岁的小孩只需要 15 秒。”

Lukasz on how jagged frontier model performance really is:

“You can say the model is amazing at mathematical Olympiad. And at the same time, I have a first grader… she's five years old. I took one exercise from this math book and none of the frontier models is able to solve it.”

The task? Two groups of dots with one shared dot, and you ask if the total is even or odd.

“The nice thing is if you let it think longer, it may actually solve it… but it takes 15 minutes. The human five year old takes 15 seconds.”

VCSharing

相关观点