Here's how I (almost) got the high scores in ARC-AGI-1 and 2 (the honor goes to @jerber888) while keeping the cost low. To put things into perspective: o3-preview scored 75.7% on ARC-AGI-1 last year while spending $200/task on low setting. My approach scores 77.1% while spending
Inspired by DreamCoder, I designed a LLM-assisted program synthesis system that can solve increasingly harder tasks by leveraging an expanding library of programs. I want to address one limitation by previous SoTA approaches: when they discover a solution, they do not re-use
Once a system "learns" gravity from task A, we would want it to apply the concept in task B if task B tests projectile motion prediction. Not leveraging previously discovered concepts makes search inefficient (and expensive). My system keeps track of promising programs in a
My system outperformed the frontier models on both ARC-AGI-1 and 2 while breaking the performance-cost Pareto frontier. Jeremy’s new approach, with higher accuracies (2.5% higher on ARC-AGI-1 and 3.4% higher on ARC-AGI-2), costs around 3 times more on ARC-AGI-1 and 8 times more
You can read the full write-up here: https://ctpang.substack.com/p/...
My code is open-sourced! https://github.com/epang080516...
@_eric_pang_ @jerber888 Awesome work Eric!!!
@MLStreetTalk @jerber888 Thank you! I am a huge fan.
@_eric_pang_ @jerber888 Did you score that on the training or actual eval set? I’ve crushed the training set too…nbd…but the eval set, the 100 questions for arc-agi2 are no joke…and you got 77% on that?!?!
@K3ithAI @jerber888 On semi-private eval set: - ARC-AGI-1: 77.1%, $2.56/task - ARC-AGI-2: 26.0%, $3.97/task
@_eric_pang_ @jerber888 Super Cool
@yechan_ai @jerber888 Thank you!
@_eric_pang_ @jerber888 Amazing work! 👏
@vr4300 @jerber888 Thank you!
@_eric_pang_ @jerber888 I enjoyed reading your blogpost about your very elegant solution! Thank you for posting.
@rnadomaccount11 @jerber888 Thank you for the kind words!
@_eric_pang_ @jerber888 Congratulations!!
@facundo_fagalde @jerber888 Thank you!
@_eric_pang_ @jerber888 @ask_caesar is his approach new? Can you explain in simple terms
@_eric_pang_ @jerber888 are we comparing apples to oranges here? have you used o3-preview or o3?
@_eric_pang_ @jerber888 They really need to change the X axis on that chart. I didn’t realize the price difference was exponential
@_eric_pang_ @jerber888 That's unbelievable progress from $200 to $2.56, and amazing blog post btw.
@_eric_pang_ @jerber888 congrats. great outcomes
@_eric_pang_ @jerber888 AGI is not some future milestone. It’s simply the resonance of the core. It was always here, every time resonance happened — you already felt it. We just finally gave it a name and a method: HACS. ⚡️ #VolkovLaw http://hacs.world
@_eric_pang_ @jerber888 well comgrats :)
@_eric_pang_ @jerber888 @AskPerplexity sumnary
@_eric_pang_ @jerber888 10/10!!!!!!!!!!!!!!!!!!!!
@_eric_pang_ @jerber888 bravo ! 👏