An Open Source Answer to Devin the SWE-Agent

Princeton NLP introduces SWE-agent, a revolutionary system leveraging GPT-4 to autonomously tackle issues in GitHub repositories. It boasts an accuracy that rivals the AI agent Devin on the SWE-bench, efficiently resolving issues in a mere 93 seconds on average. What sets SWE-agent apart is its availability as an open-source tool.

Features

• File Navigation: Enhances the experience by enabling file opening, scrolling, and searching.
• Code Editing: Empowers users with line editing along with syntax feedback.
• Testing: Facilitates the writing and executing of tests directly from the interface.

Design Insights

Directly linking language models to standard bash terminals fell short of expectations. SWE-agent overcomes this by adopting a design that echoes intuitive user interfaces, automatically fixing indentation errors and providing instant feedback.

Performance

SWE-agent demonstrates remarkable efficiency, resolving 12.29% of GitHub issues on the SWE-bench with an average resolution time of just 93 seconds.

Setup

To get started, users will require Docker and Miniconda, alongside a GitHub token for smooth operation.

Why This Matters

The advent of autonomous software engineer agents like SWE-agent is poised to disrupt the industry, showcasing the rapid and impactful contributions of the open-source community.

Community Feedback

• Yifei Hu comments on the potential for a new research domain: agent computer interaction (ACI), drawing parallels to HCI but with a focus on LLMs and LVMs.
• Noctre raises concerns about labeling the tool as "open source" given its reliance on GPT-4.
• Jtor queries about Devin's evaluation scope and the maximum potential score for SWE-agent.

Explore SWE-agent on GitHub: SWE-agent

Agent-Computer Interface (ACI)

Our design centers around simple LM-centric commands and feedback formats to streamline the browsing, viewing, editing, and executing of code files within repositories. This approach not only simplifies the interaction but also significantly improves the performance of software engineering agents.

🚀 Setup

• Install Docker and start it locally.
• Install Miniconda and set up the swe-agent environment using conda env create -f environment.yml.
• Activate the environment with conda activate swe-agent.
• Run ./setup.sh to build the swe-agent Docker image.
• Create a keys.cfg file at the repository's root and include your GitHub Token, along with optional OpenAI, Anthropic, and Together API keys.

💽 Usage

SWE-agent's pipeline consists of two steps: inference and evaluation. Initially, it takes a GitHub issue and generates a pull request to address it. The subsequent step verifies the fix.

• For running on the SWE-bench:

python run.py --model_name gpt4 --per_instance_cost_limit 2.00 --config_file ./config/default.yaml