Agent-E

Description

Agent-E is an agent-based system designed for automating actions within a user's web browser. Built upon the AG2 agent framework, it offers a natural language interface for interacting with websites. The system focuses on enabling users to automate a wide range of web-based tasks, from simple form submissions to complex e-commerce product searches and media playback management.

At its core, Agent-E utilizes a skill-and-agent architecture. Skills represent atomic actions, such as clicking elements, entering text, or opening URLs. These skills are orchestrated by agents, primarily a User Proxy agent that executes skills and a Browser Navigation agent responsible for web interactions. This modular design allows Agent-E to flexibly assemble skills to create sophisticated web automation workflows. The system prioritizes using configured skills over allowing LLMs to generate arbitrary code, aiming for safer and more predictable outcomes.

Key capabilities include filling out web forms, searching and sorting products on e-commerce platforms like Amazon, locating specific content on websites, navigating and controlling web media like YouTube videos, performing comprehensive web searches, and managing tasks on project management platforms. Agent-E also offers advanced features through its managed web agent and orchestrator, including enterprise enhancements, advanced logging, role-based access, and cloud-hosted scalable infrastructure.

The target audience for Agent-E includes developers, researchers, and power users looking to automate repetitive or complex web-based tasks. Its value proposition lies in its ability to translate natural language commands into actionable browser interactions, saving time and reducing manual effort. The system also supports customization of LLM parameters and offers an API for programmatic integration, making it a versatile tool for various automation needs.

Agent-E provides a quick start guide with installation scripts for macOS/Linux and Windows, followed by environment variable configuration. Users can run Agent-E via a command-line interface or launch a FastAPI server for web endpoint access, enabling programmatic task execution. The project also emphasizes community contribution and provides resources for reporting issues and engaging in discussions.

Agent-E's Core Features

• Agent-driven web automation

• Natural language interface for browser interaction

• Form filling capabilities

• E-commerce product search and sorting

• Web content location and extraction

• Web media playback control

• Comprehensive web search functionality

• Task management on platforms like JIRA

• FastAPI wrapper for programmatic access

• Customizable LLM parameters

• Support for open-source LLMs via LiteLLM and Ollama

• DOM Distillation for efficient element identification

• Skills library for sensing and action capabilities

Getting Started with Agent-E

1. 1

   Clone the repository

2. 2

   Install dependencies using provided scripts (e.g., ./install.sh or .\win_install.ps1)

3. 3

   Configure environment variables in .env and agents_llm_config.json

4. 4

   Run Agent-E using 'python -m ae.main' or './run.sh'

5. 5

   Launch the FastAPI server for API access using 'uvicorn ae.server.api_routes:app --reload --loop asyncio'

6. 6

   Execute tasks via HTTP POST requests to the API endpoint

7. 7

   Customize LLM parameters via environment variables or JSON configuration files

Agent-E's Use Cases

• Web Scraping
• E-commerce Automation
• Form Filling
• Content Aggregation
• Task Management
• Personalized Assistance
• API Integration

FAQ from Agent-E

• What is Agent-E?

  Agent-E is an AI-driven automation tool that uses an agent-based framework to automate tasks within a web browser, enabling natural language interaction with websites.

• What kind of web tasks can Agent-E automate?

  Agent-E can automate tasks such as filling web forms, searching and sorting products on e-commerce sites, locating specific information on web pages, and managing web media playback.

• How do I install Agent-E?

  You can install Agent-E by cloning the GitHub repository and running the provided installation scripts for macOS/Linux or Windows, followed by dependency installation and environment configuration.

• Can Agent-E be used programmatically?

  Yes, Agent-E provides a FastAPI wrapper that allows you to send commands via HTTP requests and receive streaming results, enabling programmatic task automation.

• Does Agent-E support open-source LLMs?

  Yes, Agent-E supports the use of open-source models through LiteLLM and Ollama, allowing users to run language models locally on their machines.

• How does Agent-E handle complex web pages?

  Agent-E uses DOM Distillation to pare down the HTML DOM to only the elements pertinent to the user's task, delivering a more digestible JSON snapshot for LLMs.

• What are the main agents in Agent-E?

  The main agents are the User Proxy agent, which executes skills, and the Browser Navigation agent, which embodies all skills for interacting with the web browser.

• How can I contribute to Agent-E?

  You can contribute by forking the repository, creating a new branch for your changes, making improvements, running tests, and submitting a pull request. Community discussions are encouraged on Discord.

• What are the future plans for Agent-E?

  Future plans include improved dropdown handling, task caching, enhanced open-source and local LLM compatibility, multi-tab and bookmark handling, PDF text handling, and Google Suite compatibility.

• How can I report an issue or suggest a feature for Agent-E?

  You can report bugs or suggest features by opening an issue in the project's GitHub issue tracker. Detailed information is appreciated for effective resolution.