Evalgpt

EvalGPT

What is EvalGPT

🧩 This project is still in the early stages of development, and we are actively working on it. If you have any questions or suggestions, please submit an issue or PR.

EvalGPT is an code interpreter framework, leveraging the power of large language models such as GPT-4, CodeLlama, and Claude 2. This powerful tool allows users to write tasks, and EvalGPT will assist in writing the code, executing it, and delivering the results.

EvalGPT's architecture draws inspiration from Google's Borg system. It includes a master node, known as EvalGPT, composed of three components: planning, scheduler, and memory.

When EvalGPT receives a request, it starts planning the task using a Large Language Model (LLM), dividing larger tasks into smaller, manageable ones. For each sub-task, EvalGPT will spawn a new node known as an EvalAgent.

Each EvalAgent is responsible for generating the code based on the assigned small task. Once the code is generated, the EvalAgent initiates a runtime to execute the code, even harnessing external tools when necessary. The results are then collected by the EvalAgent.

EvalAgent nodes can access the memory from the EvalGPT master node, allowing for efficient and effective communication. If an EvalAgent encounters any errors during the process, it reports the error to the EvalGPT master node, which then replans the task to avoid the error.

Finally, the EvalGPT master node collates all results from the EvalAgent nodes and generates the final answer for the request.

Benefits

1. Automated Code Writing: EvalGPT leverages advanced language models to auto-generate code, reducing manual effort and increasing productivity.
2. Efficient Task Execution: By breaking down complex tasks into manageable sub-tasks, EvalGPT ensures efficient…