AutoGen: A Multi-Agent Framework - Overview and Improvements

Video Description

AutoGen is a popular multi-agent framework by Microsoft that enables you to define your own agents by specifying the name and the description (as the system prompt) for each agent, and setting up a two-agent or group setting for conversation to happen. You can integrate various custom replies via the register_reply function, which specify how to reply to specific triggers (e.g. specific agents, specific keywords in messages). You can also integrate the agents with pre-defined tools/functions by specifying them in "llm-config" and registering the function via "register_function". See https://medium.com/@coldstart_coder/autogen-essentials-function-integration-for-smarter-agents-7c3b4a0fdc12 for details. All in all, AutoGen is a great first start for a multi-agent framework. I personally find the prompt very lengthy, and the conversation framework very verbose. It would be better if we have a more streamlined workflow (perhaps a templated list of actions tailored to a particular use case) with shorter LLM prompts for it to be more performant. At the last part of the presentation, Soham Sarkar (@Crit) demonstrated his working implementation of AutoGen for his own use case of adding a company into a database! ~~~~~ Github: https://github.com/microsoft/autogen Paper: https://arxiv.org/abs/2308.08155 My template AutoGen notebook: https://github.com/tanchongmin/TensorFlow-Implementations/blob/main/Paper_Reviews/AutoGen/AutoGen.ipynb My AutoGen Slides: https://github.com/tanchongmin/TensorFlow-Implementations/blob/main/Paper_Reviews/AutoGen/AutoGen%20Slides.pdf For those interested in the StrictJSON framework I talked about at 34:18, here it is: https://github.com/tanchongmin/strictjson ~~~~~ 0:00 Introduction and Code Walkthrough 23:54 Basic Agent Structure 32:04 Prompt for Agents 36:13 RAG on Demand Agent 39:08 Grounding Message Agent 46:31 Red-teaming and Blue-teaming Agents 48:26 Advanced Topics - Dynamic Conversation Flow 50:44 More Agents = Better? 53:27 Self-correcting code with a suite of agents 55:00 Pseudo Multi-Agent Conversation 57:45 Is conversation really necessary to do tasks? 59:02 Takeaways from AutoGen 1:03:34 Discussion 1:24:05 Demo by Sarkar! ~~~~~ AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator. Discord: https://discord.gg/bzp87AHJy5 LinkedIn: https://www.linkedin.com/in/chong-min-tan-94652288/ Online AI blog: https://delvingintotech.wordpress.com/ Twitter: https://twitter.com/johntanchongmin Try out my games here: https://simmer.io/@chongmin