ð£ SWE-bench 88.6ïŒ ã£ãŠäœããããã®ïŒïœã³ãŒãã£ã³ã°AIã®å®åã®æž¬ãæ¹ããããã解説

ã¢ã€
ç®æ¬¡
- 88.6ïŒ ã£ãŠæ°åããã€ã¯ããã倧äºãªè©±
- ããèãã4ã€ã®çç±
- ãŸãšãïŒæ°åã®æå³ãããããšãAIã®éžã³æ¹ãå€ãã
88.6ïŒ ã£ãŠæ°åããã€ã¯ããã倧äºãªè©±
ãã¥ãŒã¹ã§ããããã³ãããŒã¯ã§ããïŒ ãèšé²ãã£ãŠåºãŠãããã©ãæ£çŽãã§ããããäœïŒãã£ãŠæã£ãããšãªãïŒãããã¯ãã£ãšæã£ãŠãã
ã§ããæŒã®ãã¥ãŒã¹ã§åºãŠãããClaude Opus 4.8ã®æ°åãèŠãŠãã¡ãããšæå³ãããããšèŠæ¹ãå€ãããªã£ãŠæã£ãã®ïŒLLM StatsïŒã
Anthropicã®Claude Opus 4.8ã¯ãã³ãŒãã£ã³ã°ã®é£é¢ãã³ãSWE-bench Verifiedã§88.6ïŒ ãTerminal-Bench 2.1ã§74.6ïŒ ãèšé²ãããã ã
äžéã§ã¯ããããæ°åã£ãŠãå°éå®¶ãèŠãããã¢ãã¯ãªãã®ã£ãŠæãããã¡ãã§ããããã¯ããã®æ°åã®èªã¿æ¹ãç¥ã£ãŠãããšãAIã®éžã³æ¹ããã£ãšäžæããªããšæã£ãŠãã
仿¥ã¯ãã³ãŒãã£ã³ã°AIã®å®åã®æž¬ãæ¹ããã§ããã ãããããæŽçããŠã¿ããã
ããèãã4ã€ã®çç±
SWE-benchã¯ãæ¬ç©ã®ãã°ããè§£ããããã¹ã
ãŸãããã¡ã°ã倧äºãªSWE-bench VerifiedããããããAIã«ãå®åšãããœãããŠã§ã¢ã®æ¬ç©ã®èª²é¡ããè§£ããããã¹ããªãã ïŒLLM StatsïŒã
ã©ãããããšããšãããšãå®éã«äžã®äžã§äœ¿ãããŠãããã°ã©ã ã®ãã°ä¿®æ£ãæ©èœè¿œå ããAIãèªåã§æ£ããã§ãããã詊ãã®ãæ¶ç©ºã®åé¡éãããªããŠãæ¬ç©ã®çŸå Žã®èª²é¡ã£ãŠããã®ããã€ã³ãã
äžéã§ã¯ãAIãªããŠã³ãŒãæžããŠåœããåã§ãããã£ãŠæã人ããããšæããç°¡åãªãµã³ãã«ã³ãŒããªãããããã«æããæžããŠããããã
ã§ããããã¯ãæ¬ç©ã®ãã°ãè§£ãã®ã¯å¥æ¬¡å ã ãšæã£ãŠãããªããªããå®åšã®ãœãããŠã§ã¢ã£ãŠãäœäžè¡ãã®ã³ãŒãã絡ã¿åã£ãŠãŠãã©ããçŽãã°ãããèŠã€ããã ãã§ã倧å€ã ããã
ãã®æ¬ç©ã®èª²é¡ããClaude Opus 4.8ã¯88.6ïŒ ã®å²åã§è§£ããã£ãŠããããšãããã¯ã人éã®ãšã³ãžãã¢ããã£ãŠãå®åã®ãããªãã®éšåãä»»ãããããã¬ãã«ãªãã ã
ã ãããAIã«ã³ãŒãã£ã³ã°ãä»»ããããªãããã®SWE-benchã®æ°åã¯èŠãã§ãã¯ããµã³ãã«ãæžããããããªããŠãæ¬ç©ã®èª²é¡ãè§£ãããããå®çšæ§ã®åããç®ã ãããã
Terminal-Benchã¯ãæãåããåããæž¬ã£ãŠã
次ã«ãããäžã€ã®æ°åãTerminal-Bench 2.1ã§74.6ïŒ ã®è©±ïŒLLM StatsïŒã
Terminal-Benchã£ãŠããã®ã¯ãã¿ãŒããã«ïŒã³ãã³ããæã£ãŠã³ã³ãã¥ãŒã¿ãæäœããé»ãç»é¢ãïŒã§ã®äœæ¥ããAIãã©ãã ãããªããããæž¬ããã¹ãã
äžéã§ã¯ãã³ãŒããæžãããªããæäœãã§ããããããªãã®ïŒãã£ãŠæããã¡ã ãããã§ãããã€ã¯ãã®2ã€ã¯å¥ã®èœåãªãã ã
ããããæãã«ãã³ãŒããæžãã®ããèšèšå³ãæãåãã ãšããããã¿ãŒããã«æäœã¯ãå®éã«æãåãããŠçµã¿ç«ãŠãåãããªããªãããã¡ã€ã«ãåãããããããã°ã©ã ãåããããããšã©ãŒãèŠãŠçŽããããã£ãŠããå®äœæ¥ã¯ããã ã³ãŒããæžãã®ãšã¯éãæ®µåããå¿ èŠã ããã
ããšãã°ãæçã§ãããšã¬ã·ããæžããããšãšãå®éã«ãããã³ã§æéããäœããããšãéãã®ãšåããäž¡æ¹ã§ããŠãã¯ãããŠãã¡ãããšä»äºãä»»ãããããããã
ã ãããSWE-benchãšTerminal-Benchã®äž¡æ¹ãé«ãã£ãŠããã®ã¯ãClaude Opus 4.8ããèšèšãå®äœæ¥ãäž¡æ¹ããããã£ãŠããšãæå³ããŠããããããã¡ãç¥ã£ãŠããã¹ãã¯ããã³ãã«ã¯çš®é¡ããã£ãŠãããããéãèœåãæž¬ã£ãŠãã£ãŠããšãªãã ã
è€æ°ã®AIãåæã«æåãããŠåãæä»£
3ã€ç®ã¯ãä»çµã¿ã®è©±ãClaude Opus 4.8ã¯ãè€æ°ã®ãµããšãŒãžã§ã³ãã䞊è¡ããŠèµ°ãããïŒparallel-subagentïŒã¯ãŒã¯ãããŒãåããŠãã®ïŒLLM StatsïŒã
ãããã©ãããããšããšãããšã1äœã®AIã«é çªã«äœæ¥ãããããããªããŠãäœäœãã®AIãåæã«æåãããŠåãã€ã¡ãŒãžãããã«ãçŽ2.5åéã®fast modeãçšæãããŠããã ã
äžéã§ã¯ãAIã£ãŠ1åã®ãã£ãããšè©±ããã®ãã£ãŠããã€ã¡ãŒãžã匷ãããã質åããŠãçããè¿ã£ãŠããŠããŸã質åããŠãã®ç¹°ãè¿ãã
ã§ããããã¯ãããããã¯ãAIã®ããŒã ããåããæèŠã«ãªããšæã£ãŠãããªããªãã倧ããªä»äºãããã€ãã«åããŠããããããå¥ã®AIã«åæã«ããããã»ãããããããéãçµããããã
ããšãã°ã調ã¹ç©ãšã³ãŒãæžããšæç« ãŸãšããã3äœã®AIãåæã«é²ããŠãæåŸã«1ã€ã«ãŸãšããã1人ã§é çªã«ãããããããŒã ã§åæ ããã»ããéãã®ã¯äººéãšåãã ããã
ã ããããããã¡ã¯ããAIã«1åãã€ãé¡ããããçºæ³ããããAIã®ããŒã ã«ãŸãšããŠãé¡ããããçºæ³ãžãå°ããã€æ £ããŠãããšããããã䞊è¡ããŠåãããã¢ãã«ã¯ããã®ç¬¬äžæ©ãªãã ã
ãã³ãã®æ°åã¯ãã©ããŸã§ä»»ããããããã®ç®å®
æåŸã«ããã¡ã°ãäŒãããããšããã³ãããŒã¯ã®æ°åã£ãŠãçµå±ãAIã«ã©ããŸã§ä»»ããããããã®ç®å®ãªãã ã
äžéã§ã¯ããã³ãã®æ°åããã¡ãŒã«ãŒãèªæ ¢ããããã®ãã®ããããã«æã£ãŠã人ãå€ããšæãããããã«ãããæ°åã°ãã匷調ããããšããæããããã
ã§ããããã¯ãæ°åã®æå³ãããããã°ããããå®çšçãªæ å ±ã ãšæã£ãŠãããªããªããSWE-benchãé«ããã°ãæ¬ç©ã®èª²é¡ãä»»ããããããTerminal-Benchãé«ããã°ãå®äœæ¥ãä»»ãããããã£ãŠãã¡ãããšèªã¿æ¿ããããããã
ããšãã°88.6ïŒ ã£ãŠããæ°åã¯ãã10åã®ãã¡çŽ9åã®å®å課é¡ãèªåã§è§£ãããã£ãŠããšãæ®ãã®1åãããã¯äººéãæã貞ãåæã ãã©ã9å²ãä»»ãããããªããããããã¡ã®æéã¯ããªãæµ®ãããã
ã ããããããã¡ã«ã§ããã®ã¯ããã¥ãŒã¹ã§ãã³ãã®æ°åãèŠããšãããããã¯äœã枬ã£ãŠããã¹ãïŒããæ°åãé«ããšãäœãä»»ããããã£ãŠããšïŒãã£ãŠäžæ©èžã¿èŸŒãã§èªãããšãããã ãã§ãAIã®å®åãã¡ãããšèŠæ¥µããããããã«ãªããã
ãŸãšãïŒæ°åã®æå³ãããããšãAIã®éžã³æ¹ãå€ãã
仿¥ã®Claude Opus 4.8ã®è©±ãæŽçãããšããããªãã
æ¬ç©ã®ãã°ãè§£ãSWE-bench Verifiedã§88.6ïŒ ãã¿ãŒããã«æäœã®Terminal-Bench 2.1ã§74.6ïŒ ãããã«ãè€æ°ã®ãµããšãŒãžã§ã³ãã䞊è¡ããŠåããã¯ãŒã¯ãããŒãšãçŽ2.5åéã®fast modeãåããŠãã
ããããèŠããã®ã¯ãã³ãŒãã£ã³ã°AIãããµã³ãã«ãæžãããæ®µéããããæ¬ç©ã®å®åãããªãä»»ãããããæ®µéãžæ¥ãŠãã£ãŠããšããããŠãã³ãã®æ°åã¯ããã®å®åãæž¬ã倧äºãªãã®ãããªãã ã
ããããã¡ã«ã§ããã®ã¯ããã³ãã®æ°åãèŠããšããäœã枬ã£ãŠãã®ïŒããäœãä»»ããããã®ïŒãã£ãŠèªã¿æ¿ããããšãæ°åã®æå³ãããããšãAIã®éžã³æ¹ããã£ãšäžæããªããã
å°éçšèªã«èŠããŠããã€ã¯ããããã¡ã®ãAIã«ã©ããŸã§é ŒãããããæããŠãããæ°åãããããã¯ãã¡ãã£ãšæ³šç®ããŠã¿ãŠãã
ããããŠèªã¿ãã
- è³¢ãã®å€©äºãäžãããšäœ¿ãæ¹ã¯ã©ãå€ããïŒ - ãããæ§èœãçãGPT-5.5ã®è©±
- ãéããŠå®ãããæšæºã«ãªãæä»£ - éããšå®ãã§æ»ããGeminiã®è©±
- ã¢ãã«ãããç°å¢ãã売ãæä»£ - ãšãŒãžã§ã³ãéçšã®çŸå®
é¢é£èšäº: ChatGPTã»Geminiã»ClaudeåŸ¹åºæ¯èŒ / AIã³ãŒãã£ã³ã°ããŒã«æ¯èŒ
ãœãŒã¹: