Problem 1
Find the bug, write a fix, test it, and keep going until the failure is actually resolved.
Modern software development moves at breakneck speed. Developers write thousands of lines of code daily, but a significant portion of their time is spent manually tracing bugs, reading error logs, and writing repetitive tests. As codebases grow, maintaining code quality and squashing bugs becomes a massive bottleneck for engineering teams.
Current debugging tools and static analyzers only point out where an error might be or rely on rigid, pre-defined rules. They do not understand the context of the logic, nor can they actively write a fix, test it, and verify if the issue is resolved. Traditional AI coding assistants require constant human prompting and hand-holding. The challenge is to move from an "assistant" that suggests code to an "agent" that actively hunts, tests, and resolves bugs autonomously.
This problem is relevant to organizations and teams building software, including:
Build an Agentic AI system capable of autonomously analyzing a codebase, identifying bugs from user reports or error logs, proposing fixes, and verifying them.
The solution should adapt its approach based on the complexity of the bug. Participants are not required to support every programming language. Teams may choose a specific language (e.g., Python, JavaScript) and are encouraged to innovate beyond basic text generation. The objective is to build a system that acts as an independent software engineer.
Understand Context: Read multiple files and understand how they interact.
Tool Use: Execute code or run terminal commands to test if a bug exists.
Iterative Reasoning: If a proposed fix fails, the agent should read the new error log and try another approach.
The following levels are intended as guidance for increasing complexity. Participants may choose one or more levels based on their approach. Extreme extendability is encouraged!
Level 1 (Basic Agency): A conversational agent that takes a piece of buggy code and an error message as input, and outputs the corrected code with an explanation.
Level 2 (Tool Integration): An agent that is given access to a secure, sandboxed environment (or mocked terminal). It can independently run a script, read the resulting error log, and apply a fix without human prompting.
Level 3 (Multi-Agent Collaboration): A system with distinct agents. For example, a "Tester Agent" that writes unit tests to intentionally break the code, and a "Coder Agent" that writes fixes until the "Tester Agent" gives a passing grade.
Level 4 (Advanced/Autonomous Workflow): The agent integrates directly with GitHub (or a mocked version of it). It reads a natural language Issue ticket, clones the repo, fixes the code, runs the test suite, and automatically generates a Pull Request (PR).
A working demonstration where an agent takes a bug or codebase as input and produces a verified fix.
Participants are free to decide:
Which programming languages, frameworks (LangChain, CrewAI, AutoGen, LlamaIndex), or APIs (OpenAI, Gemini, HuggingFace) to use.
How to mock data, databases, and external tools.
The system architecture (single script, full-stack app, terminal-based).
1. Meaningful AI Usage
Do not just build a standard software app. The solution must rely on AI decision-making.
2. Autonomy over Hardcoding
The agent should figure out how to solve the problem, rather than relying entirely on hard-coded if/else statements.
3. Innovation Freedom
Participants are not restricted to specific tech stacks. Using smaller open-source models creatively can score just as high as using paid APIs.
What It Measures
Agentic Autonomy How well the system operates without human hand-holding
Whether the agent uses external tools correctly without crashing
Whether the AI self-corrects when it hits an obstacle
Scalability & Vision Whether the prototype could realistically be expanded into a real-world product
Ability to clearly demonstrate a working scenario within the 10-hour limit