galileo-ai/agent-leaderboard
Viewer
•
Updated
•
1.28k
•
356
•
14
Siddharth
sidbin
AI & ML interests
None yet
Recent Activity
liked
a dataset
8 days ago
galileo-ai/agent-leaderboard
liked
a dataset
about 2 months ago
gorilla-llm/Berkeley-Function-Calling-Leaderboard
Organizations
sidbin's activity
liked
a
dataset
8 days ago
reacted to
retronic's
post with 🔥
16 days ago
Post
4387
Colox, a reasoning AI model. I am currently working on a model smarter than GPT o1 that thinks before it speaks. It is coming tomorrow in the afternoon.
- 7 replies
liked
a
dataset
about 2 months ago
liked
a
model
3 months ago
upvoted
a
collection
3 months ago
liked
2
datasets
3 months ago
upvoted
a
paper
3 months ago
upvoted
a
collection
3 months ago
reacted to
m-ric's
post with 🚀🔥
3 months ago
Post
1300
🤖 𝗔𝗱𝗼𝗯𝗲'𝘀 𝗰𝗼𝗱𝗲-𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗻𝗴 𝗮𝗴𝗲𝗻𝘁 𝗿𝗲𝗮𝗰𝗵𝗲𝘀 𝘁𝗵𝗲 𝘁𝗼𝗽 𝗼𝗳 𝗚𝗔𝗜𝗔 𝗹𝗲𝗮𝗱𝗲𝗿𝗯𝗼𝗮𝗿𝗱 - and their paper cites my work!
💡 Reminder: In short, Agentic systems are a vehicle in which you put your LLM to allow it access to the outside world.
➡️ The team of researchers at Adobe started from the idea that current agentic systems lack the ability to define their own tools. So they decided to make an agent that writes actions as code, thus allowing it to write python functions that can be re-used later as tools!
Here's what the LLM generations can look like with the proper prompt:
Thought: I need to access the excel file using a different method.
Action:
Then your system executes this and appends the observation to the agent's memory.
Why is this code formulation better than classical tool use formulation as JSON? The paper explains:
"Most existing work uses text or JSON as the representation of actions, which significantly lacks the two criteria mentioned earlier: generality and composability. In contrast, DynaSaur can utilize available actions or create new ones if necessary, using code as a unified representation. In principle, acting with code enables agents to solve any Turing-complete problem."
The idea of using code is not new: in fact, we do it in transformers.agents (thus the citation that I got). They implementation adds further refinements, like using RAG to retrieve relevant functions before generating an action, which increases performance further.
And they observe that code agents perform much better, reaching the top of GAIA leaderboard! 🥇
Go take a look, it's really clear and informative!
Paper added to my agents collection 👉 m-ric/agents-65ba776fbd9e29f771c07d4e
💡 Reminder: In short, Agentic systems are a vehicle in which you put your LLM to allow it access to the outside world.
➡️ The team of researchers at Adobe started from the idea that current agentic systems lack the ability to define their own tools. So they decided to make an agent that writes actions as code, thus allowing it to write python functions that can be re-used later as tools!
Here's what the LLM generations can look like with the proper prompt:
Thought: I need to access the excel file using a different method.
Action:
def access_excel_file(file_path)
... # rest of the code (the agent does writes it, but I don't have room in this post)
return rowsThen your system executes this and appends the observation to the agent's memory.
Why is this code formulation better than classical tool use formulation as JSON? The paper explains:
"Most existing work uses text or JSON as the representation of actions, which significantly lacks the two criteria mentioned earlier: generality and composability. In contrast, DynaSaur can utilize available actions or create new ones if necessary, using code as a unified representation. In principle, acting with code enables agents to solve any Turing-complete problem."
The idea of using code is not new: in fact, we do it in transformers.agents (thus the citation that I got). They implementation adds further refinements, like using RAG to retrieve relevant functions before generating an action, which increases performance further.
And they observe that code agents perform much better, reaching the top of GAIA leaderboard! 🥇
Go take a look, it's really clear and informative!
Paper added to my agents collection 👉 m-ric/agents-65ba776fbd9e29f771c07d4e
liked
2
datasets
3 months ago
liked
a
Space
3 months ago
liked
a
dataset
3 months ago
updated
3
models
3 months ago