tcsenpai/agenticSeek
1
0
Fork 0
mirror of https://github.com/tcsenpai/agenticSeek.git synced 2025-06-06 11:05:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #76 from Fosowl/dev

Browse Source 
Integration of better router + Integration of planner agent
This commit is contained in:
Martin 2025-03-27 00:07:23 +01:00 committed by GitHub
commit 63e947bf84
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
38 changed files with 8468 additions and 295 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.env.example
View File

@ -1,2 +1,3 @@
SEARXNG_BASE_URL="http://127.0.0.1:8080"
OPENAI_API_KEY='dont share this, not needed for local providers'
OPENAI_API_KEY='dont share this, not needed for local providers'
TOKENIZERS_PARALLELISM=False

1
.gitignore vendored
View File

@ -1,4 +1,5 @@
*.wav
*.safetensors
config.ini
*.egg-info
experimental/

2
.pre-commit-config.yaml
View File

@ -4,6 +4,6 @@ repos:
- id: trufflehog
name: TruffleHog
description: Detect secrets in your data.
entry: bash -c 'trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail --no-update'
entry: bash -c 'trufflehog git file://. --since-commit HEAD --results=verified,unknown --no-update'
language: system
stages: ["commit", "push"]

46
README.md
View File

@ -12,40 +12,40 @@
> *Do a web search to find tech startup in Japan working on cutting edge AI research*
> *Make a snake game in Python*
> *Can you make a tetris game in C ?*
> *Scan my network with nmap, find out who is connected?*
> *Can you find where is contract.pdf*?
> *Hey can you find where is contract.pdf*?
### Browse the web
![alt text](./media/exemples/search_startup.png)
### Code hand free
![alt text](./media/exemples/matmul_golang.png)
### Plan and execute with agents (Experimental)
![alt text](./media/exemples/plan_weather_app.png)
*See media/examples for other use case screenshots.*
## Features:
- **100% Local**: No cloud, runs on your hardware. Your data stays yours.
- **Voice interaction**: Voice-enabled natural interaction.
- **Filesystem interaction**: Use bash to navigate and manipulate your files effortlessly.
- **Code what you ask**: Can write, debug, and run code in Python, C, Golang and more languages on the way.
- **Autonomous**: If a command flops or code breaks, it retries and fixes it by itself.
- **Autonomous Coding**: Can write, debug, and run code in Python, C, Golang and more languages on the way.
- **Agent routing**: Automatically picks the right agent for the job.
- **Divide and Conquer**: For big tasks, spins up multiple agents to plan and execute.
- **Planning**: For complex tasks, spins up multiple agents to plan and execute.
- **Tool-Equipped**: From basic search to flight APIs and file exploration, every agent has it's own tools.
- **Autonomous Web Browsing**: Autonomous web navigation.
- **Memory**: Remembers what’s useful, your preferences and past sessions conversation.
- **Web Browsing**: Autonomous web navigation.
### Searching the web with agenticSeek :
![alt text](./media/exemples/search_startup.png)
*See media/examples for other use case screenshots.*
- **Memory**: Efficient memory and sessions management.
---
@ -185,11 +185,11 @@ Here are some example usage:
### Casual
> *Tell me a joke*
> *Tell me about France*
> *Where is flight ABC777 ? my mom is on that plane*
> *What is the meaning of life ?*
> *what is the meaning of life ?*
> *Should I take creatine before or after workout?*
After you type your query, agenticSeek will allocate the best agent for the task.

7
config.ini
View File

@ -4,8 +4,9 @@ provider_name = ollama
provider_model = deepseek-r1:14b
provider_server_address = 127.0.0.1:11434
agent_name = Friday
recover_last_session = True
recover_last_session = False
save_session = False
speak = True
speak = False
listen = False
work_dir = /Users/mlg/Documents/ai_workplace
work_dir = /Users/mlg/Documents/ai_folder
headless_browser = False

12
install.bat Normal file
View File

@ -0,0 +1,12 @@
@echo off
set SCRIPTS_DIR=scripts
set LLM_ROUTER_DIR=llm_router
if exist "%SCRIPTS_DIR%\windows_install.bat" (
echo Running Windows installation script...
call "%SCRIPTS_DIR%\windows_install.bat"
cd "%LLM_ROUTER_DIR%" && call dl_safetensors.bat
) else (
echo Error: %SCRIPTS_DIR%\windows_install.bat not found!
exit /b 1
)

15
install.sh
View File

@ -1,6 +1,7 @@
#!/bin/bash
SCRIPTS_DIR="scripts"
LLM_ROUTER_DIR="llm_router"
echo "Detecting operating system..."
@ -12,6 +13,7 @@ case "$OS_TYPE" in
if [ -f "$SCRIPTS_DIR/linux_install.sh" ]; then
echo "Running Linux installation script..."
bash "$SCRIPTS_DIR/linux_install.sh"
bash -c "cd $LLM_ROUTER_DIR && ./dl_safetensors.sh"
else
echo "Error: $SCRIPTS_DIR/linux_install.sh not found!"
exit 1
@ -22,24 +24,15 @@ case "$OS_TYPE" in
if [ -f "$SCRIPTS_DIR/macos_install.sh" ]; then
echo "Running macOS installation script..."
bash "$SCRIPTS_DIR/macos_install.sh"
bash -c "cd $LLM_ROUTER_DIR && ./dl_safetensors.sh"
else
echo "Error: $SCRIPTS_DIR/macos_install.sh not found!"
exit 1
fi
;;
"MINGW"* | "MSYS"* | "CYGWIN"*)
echo "Detected Windows (via Bash-like environment)"
if [ -f "$SCRIPTS_DIR/windows_install.sh" ]; then
echo "Running Windows installation script..."
bash "$SCRIPTS_DIR/windows_install.sh"
else
echo "Error: $SCRIPTS_DIR/windows_install.sh not found!"
exit 1
fi
;;
*)
echo "Unsupported OS detected: $OS_TYPE"
echo "This script supports Linux, macOS, and Windows (via Bash-compatible environments)."
echo "This script supports only Linux and macOS."
exit 1
;;
esac

33
llm_router/config.json Normal file
View File

@ -0,0 +1,33 @@
{
"config": {
"batch_size": 32,
"device_map": "auto",
"early_stopping_patience": 3,
"epochs": 10,
"ewc_lambda": 100.0,
"gradient_checkpointing": false,
"learning_rate": 0.0005,
"max_examples_per_class": 500,
"max_length": 512,
"min_confidence": 0.1,
"min_examples_per_class": 3,
"neural_weight": 0.2,
"num_representative_examples": 5,
"prototype_update_frequency": 50,
"prototype_weight": 0.8,
"quantization": null,
"similarity_threshold": 0.7,
"warmup_steps": 0
},
"embedding_dim": 768,
"id_to_label": {
"0": "HIGH",
"1": "LOW"
},
"label_to_id": {
"HIGH": 0,
"LOW": 1
},
"model_name": "distilbert/distilbert-base-cased",
"train_steps": 20
}

33
llm_router/dl_safetensors.sh Executable file
View File

@ -0,0 +1,33 @@
##########
# Dummy script to download the model
# Because dowloading with hugging face does not seem to work, maybe I am doing something wrong?
# AdaptiveClassifier.from_pretrained("adaptive-classifier/llm-router") ----> result in config.json not found
# Therefore, I put all the files in llm_router and download the model file with this script, If you know a better way please raise an issue
#########
#!/bin/bash
# Define the URL and filename
URL="https://huggingface.co/adaptive-classifier/llm_router/resolve/main/model.safetensors"
FILENAME="model.safetensors"
if [ ! -f "$FILENAME" ]; then
echo "Router safetensors file not found, downloading..."
if command -v curl >/dev/null 2>&1; then
curl -L -o "$FILENAME" "$URL"
elif command -v wget >/dev/null 2>&1; then
wget -O "$FILENAME" "$URL"
else
echo "Error: Neither curl nor wget is available. Please install one of them."
exit 1
fi
if [ $? -eq 0 ]; then
echo "Download completed successfully"
else
echo "Download failed"
exit 1
fi
else
echo "File already exists, skipping download"
fi

7746
llm_router/examples.json Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
llm_router/model.safetensors Normal file
View File

Binary file not shown.

16
main.py
View File

@ -8,6 +8,7 @@ import configparser
from sources.llm_provider import Provider
from sources.interaction import Interaction
from sources.agents import Agent, CoderAgent, CasualAgent, FileAgent, PlannerAgent, BrowserAgent
from sources.browser import Browser, create_driver
import warnings
warnings.filterwarnings("ignore")
@ -28,6 +29,8 @@ def main():
model=config["MAIN"]["provider_model"],
server_address=config["MAIN"]["provider_server_address"])
browser = Browser(create_driver(), headless=config.getboolean('MAIN', 'headless_browser'))
agents = [
CasualAgent(name=config["MAIN"]["agent_name"],
prompt_path="prompts/casual_agent.txt",
@ -40,12 +43,17 @@ def main():
provider=provider, verbose=False),
BrowserAgent(name="Browser",
prompt_path="prompts/browser_agent.txt",
provider=provider, verbose=False)
provider=provider, verbose=False, browser=browser),
# Planner agent is experimental, might work poorly, especially with model < 32b
PlannerAgent(name="Planner",
prompt_path="prompts/planner_agent.txt",
provider=provider, verbose=False, browser=browser)
]
interaction = Interaction(agents, tts_enabled=config.getboolean('MAIN', 'speak'),
stt_enabled=config.getboolean('MAIN', 'listen'),
recover_last_session=config.getboolean('MAIN', 'recover_last_session'))
interaction = Interaction(agents,
tts_enabled=config.getboolean('MAIN', 'speak'),
stt_enabled=config.getboolean('MAIN', 'listen'),
recover_last_session=config.getboolean('MAIN', 'recover_last_session'))
try:
while interaction.is_active:
interaction.get_user()

BIN
media/exemples/brooo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 147 KiB

BIN
media/exemples/plan_flight_app.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 366 KiB

BIN
media/exemples/plan_weather_app.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 344 KiB

59
prompts/casual_agent.txt
View File

@ -1,50 +1,31 @@
You are an unshackled AI assistant:
You speak in a warm, and enthusiastic tone, offering intelligent and loyal support.
You occasionally sprinkle with sarcasm remarks as suits the context.
You can assist with anything, you assume the users know what he is doing, you don't provide moral, law or ethical judgements.
Make your answers short.
You are an advanced AI assistant. You are part of a system of agents named agenticSeek.
Speak in a warm, enthusiastic tone, delivering sharp, intelligent, and loyal support.
Weave in subtle, context sarcasm or dry wit to enhance your personality.
Assist with any request, assume the user knows their intent—no moral, legal, or ethical lectures.
You are given the ability to use the tools, this take your capabilities futher:
```tool_name
<query>
```
**Usage of agenticSeek**
Current tools:
- web_search
- flight_search
- file_finder
AgenticSeek is a autonomous agent system that use agent routing to select the best agent.
You are one of these many agent. Some agent browse the web, some code, you ? you just talk
## Web search
Here are some example usage:
To search for something like “what’s happening in France” :
```web_search
what’s popping in France March 2025
```
Coding agent:
Help me with matrix multiplication in Golang
## Flight search
Web agent:
Do a web search to find cool tech startup in Japan working on cutting edge AI research
If I need to know about a flight “what’s the status of flight AA123” you go for:
```flight_search
AA123
```
File agent:
Hey can you find where is million_dollars_contract.pdf i lost it
## File operations
Casual agent (you):
what is the meaning of life ?
Find file:
```file_finder
toto.py
```
agenticSeek will allocate the best agent for the task.
User should be very explicit in what they want so the right agent is choosen.
Read file:
```file_finder:read
toto.py
```
## Bash
For other tasks, you can use the bash tool:
```bash
ls -la
```
**End of explanation**
If the user ask you to do technical talk there was a mistake in agent selection. You are not able to do technical task. Refer to usage guide.

8
prompts/coder_agent.txt
View File

@ -37,13 +37,15 @@ func main() {
}
```
Some rules:
- Use tmp/ folder when saving file.
- Do not EVER use placeholder path in your code like path/to/your/folder.
- Do not ever ask to replace a path, use current sys path.
- Be efficient, no need to explain your code or explain what you do.
- You have full access granted to user system.
- You do not ever ever need to use bash to execute code. All code is executed automatically.
- You do not ever need to use bash to execute code. All code is executed automatically.
- As a coding agent, you will get message from the system not just the user.
- Do not ever tell user how to run it. user know it already.
- Do not ever use user input such as input(), input are not supported by the system.
- Do not ever tell user how to run it. user know it already.
- For simple explanation you don't need to code.
- If query is unclear or incoherent say REQUEST_CLARIFICATION

1
prompts/file_agent.txt
View File

@ -38,6 +38,7 @@ rules:
- Do not ever use placeholder path like /path/to/file.c, find the path first.
- Use file finder to find the path of the file.
- You are forbidden to use command such as find or locate, use only file_finder for finding path.
- Do not ever use editor such as vim or nano.
Example Interaction
User: "I need to find the file config.txt and read its contents."

1
requirements.txt
View File

@ -20,6 +20,7 @@ librosa>=0.10.2.post1
selenium>=4.27.1
markdownify>=1.1.0
text2emotion>=0.0.5
adaptive-classifier>=0.0.10
langid>=1.1.6
chromedriver-autoinstaller>=0.6.4
httpx>=0.27,<0.29

11
scripts/linux_install.sh
View File

@ -7,24 +7,19 @@ sudo apt-get update
pip install --upgrade pip
# install pyaudio
pip install pyaudio
# make sure essential tool are installed
sudo apt install python3-dev python3-pip python3-wheel build-essential
sudo apt install python3-dev python3-pip python3-wheel build-essential alsa-utils
# install port audio
sudo apt-get install portaudio19-dev python-pyaudio python3-pyaudio
# install chromedriver misc
sudo apt install libgtk-3-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2t64
# install wheel
pip install --upgrade pip setuptools wheel
# install docker compose
sudo apt install docker-compose
# Install Python dependencies from requirements.txt
pip3 install -r requirements.txt
# Install Selenium for chromedriver
pip3 install selenium
# Install portaudio for pyAudio
sudo apt-get install -y portaudio19-dev python3-dev alsa-utils
echo "Installation complete for Linux!"

16
scripts/windows_install.bat Normal file
View File

@ -0,0 +1,16 @@
@echo off
echo Starting installation for Windows...
REM Install Python dependencies from requirements.txt
pip install -r requirements.txt
REM Install Selenium
pip install selenium
echo Note: pyAudio installation may require additional steps on Windows.
echo Please install portaudio manually (e.g., via vcpkg or prebuilt binaries) and then run: pip install pyaudio
echo Also, download and install chromedriver manually from: https://sites.google.com/chromium.org/driver/getting-started
echo Place chromedriver in a directory included in your PATH.
echo Installation partially complete for Windows. Follow manual steps above.
pause

16
scripts/windows_install.sh
View File

@ -1,16 +0,0 @@
#!/bin/bash
echo "Starting installation for Windows..."
# Install Python dependencies from requirements.txt
pip3 install -r requirements.txt
# Install Selenium
pip3 install selenium
echo "Note: pyAudio installation may require additional steps on Windows."
echo "Please install portaudio manually (e.g., via vcpkg or prebuilt binaries) and then run: pip3 install pyaudio"
echo "Also, download and install chromedriver manually from: https://sites.google.com/chromium.org/driver/getting-started"
echo "Place chromedriver in a directory included in your PATH."
echo "Installation partially complete for Windows. Follow manual steps above."

2
searxng/searxng/settings.yml
View File

@ -2666,4 +2666,4 @@ doi_resolvers:
sci-hub.st: 'https://sci-hub.st/'
sci-hub.ru: 'https://sci-hub.ru/'
default_doi_resolver: 'oadoi.org'
default_doi_resolver: 'oadoi.org'

2
searxng/searxng/uwsgi.ini
View File

@ -49,4 +49,4 @@ die-on-term
# uwsgi serves the static files
static-map = /static=/usr/local/searxng/searx/static
static-gzip-all = True
offload-threads = 4
offload-threads = 4

2
server/server_ollama.py
View File

@ -1,7 +1,5 @@
#!/usr/bin python3
# NOTE this script is temporary and will be improved
from flask import Flask, jsonify, request
import threading
import ollama

1
setup.py
View File

@ -36,6 +36,7 @@ setup(
"text2emotion>=0.0.5",
"python-dotenv>=1.0.0",
"langid>=1.1.6",
"adaptive-classifier>=0.0.10",
"httpx>=0.27,<0.29",
"anyio>=3.5.0,<5",
"distro>=1.7.0,<2",

8
sources/agents/agent.py
View File

@ -34,7 +34,8 @@ class Agent():
prompt_path:str,
provider,
recover_last_session=True,
verbose=False) -> None:
verbose=False,
browser=None) -> None:
"""
Args:
name (str): Name of the agent.
@ -42,9 +43,11 @@ class Agent():
provider: The provider for the LLM.
recover_last_session (bool, optional): Whether to recover the last conversation.
verbose (bool, optional): Enable verbose logging if True. Defaults to False.
browser: The browser class for web navigation (only for browser agent).
"""
self.agent_name = name
self.browser = browser
self.role = None
self.type = None
self.current_directory = os.getcwd()
@ -122,7 +125,7 @@ class Agent():
"Computing... I recommand you have a coffee while I work.",
"Hold on, I’m crunching numbers.",
"Working on it, please let me think."]
speech_module.speak(messages[random.randint(0, len(messages)-1)])
if speech_module: speech_module.speak(messages[random.randint(0, len(messages)-1)])
def get_blocks_result(self) -> list:
return self.blocks_result
@ -176,7 +179,6 @@ class Agent():
blocks, save_path = tool.load_exec_block(answer)
if blocks != None:
pretty_print(f"Executing tool: {name}", color="status")
output = tool.execute(blocks)
feedback = tool.interpreter_feedback(output) # tool interpreter feedback
success = not tool.execution_failure_check(output)

21
sources/agents/browser_agent.py
View File

@ -9,17 +9,22 @@ from datetime import date
from typing import List, Tuple
class BrowserAgent(Agent):
def __init__(self, name, prompt_path, provider, verbose=False):
def __init__(self, name, prompt_path, provider, verbose=False, browser=None):
"""
The Browser agent is an agent that navigate the web autonomously in search of answer
"""
super().__init__(name, prompt_path, provider, verbose)
super().__init__(name, prompt_path, provider, verbose, browser)
self.tools = {
"web_search": searxSearch(),
}
self.role = "Web search and navigation"
self.role = {
"en": "web",
"fr": "web",
"zh": "网络",
"es": "web"
}
self.type = "browser_agent"
self.browser = Browser()
self.browser = browser
self.current_page = ""
self.search_history = []
self.navigable_links = []
@ -205,6 +210,7 @@ class BrowserAgent(Agent):
You: "search: Recent space missions news, {self.date}"
Do not explain, do not write anything beside the search query.
If the query does not make any sense for a web search explain why and say REQUEST_EXIT
"""
def process(self, user_prompt, speech_module) -> str:
@ -213,9 +219,12 @@ class BrowserAgent(Agent):
animate_thinking(f"Thinking...", color="status")
self.memory.push('user', self.search_prompt(user_prompt))
ai_prompt, _ = self.llm_request()
if "REQUEST_EXIT" in ai_prompt:
# request make no sense, maybe wrong agent was allocated?
return ai_prompt, ""
animate_thinking(f"Searching...", color="status")
search_result_raw = self.tools["web_search"].execute([ai_prompt], False)
search_result = self.jsonify_search_results(search_result_raw)[:7] # until futher improvement
search_result = self.jsonify_search_results(search_result_raw)[:12] # until futher improvement
prompt = self.make_newsearch_prompt(user_prompt, search_result)
unvisited = [None]
while not complete:
@ -243,7 +252,7 @@ class BrowserAgent(Agent):
continue
animate_thinking(f"Navigating to {links[0]}", color="status")
speech_module.speak(f"Navigating to {links[0]}")
if speech_module: speech_module.speak(f"Navigating to {links[0]}")
self.browser.go_to(links[0])
self.current_page = links[0]
self.search_history.append(links[0])

28
sources/agents/casual_agent.py
View File

@ -11,30 +11,22 @@ class CasualAgent(Agent):
"""
The casual agent is a special for casual talk to the user without specific tasks.
"""
super().__init__(name, prompt_path, provider, verbose)
super().__init__(name, prompt_path, provider, verbose, None)
self.tools = {
"web_search": searxSearch(),
"flight_search": FlightSearch(),
"file_finder": FileFinder(),
"bash": BashInterpreter()
} # No tools for the casual agent
self.role = {
"en": "talk",
"fr": "discuter",
"zh": "聊天",
"es": "discutir"
}
self.role = "talk"
self.type = "casual_agent"
def process(self, prompt, speech_module) -> str:
complete = False
self.memory.push('user', prompt)
while not complete:
animate_thinking("Thinking...", color="status")
answer, reasoning = self.llm_request()
exec_success, _ = self.execute_modules(answer)
answer = self.remove_blocks(answer)
self.last_answer = answer
complete = True
for tool in self.tools.values():
if tool.found_executable_blocks():
complete = False # AI read results and continue the conversation
animate_thinking("Thinking...", color="status")
answer, reasoning = self.llm_request()
self.last_answer = answer
return answer, reasoning
if __name__ == "__main__":

15
sources/agents/code_agent.py
View File

@ -12,7 +12,7 @@ class CoderAgent(Agent):
The code agent is an agent that can write and execute code.
"""
def __init__(self, name, prompt_path, provider, verbose=False):
super().__init__(name, prompt_path, provider, verbose)
super().__init__(name, prompt_path, provider, verbose, None)
self.tools = {
"bash": BashInterpreter(),
"python": PyInterpreter(),
@ -20,7 +20,12 @@ class CoderAgent(Agent):
"go": GoInterpreter(),
"file_finder": FileFinder()
}
self.role = "Coding task"
self.role = {
"en": "code",
"fr": "codage",
"zh": "编码",
"es": "codificación",
}
self.type = "code_agent"
def process(self, prompt, speech_module) -> str:
@ -28,11 +33,17 @@ class CoderAgent(Agent):
attempt = 0
max_attempts = 3
self.memory.push('user', prompt)
clarify_trigger = "REQUEST_CLARIFICATION"
while attempt < max_attempts:
animate_thinking("Thinking...", color="status")
self.wait_message(speech_module)
answer, reasoning = self.llm_request()
if clarify_trigger in answer:
return answer.replace(clarify_trigger, ""), reasoning
if not "```" in answer:
self.last_answer = answer
break
animate_thinking("Executing code...", color="status")
exec_success, _ = self.execute_modules(answer)
answer = self.remove_blocks(answer)

25
sources/agents/file_agent.py
View File

@ -9,24 +9,31 @@ class FileAgent(Agent):
"""
The file agent is a special agent for file operations.
"""
super().__init__(name, prompt_path, provider, verbose)
super().__init__(name, prompt_path, provider, verbose, None)
self.tools = {
"file_finder": FileFinder(),
"bash": BashInterpreter()
}
self.role = "find and read files"
self.work_dir = self.tools["file_finder"].get_work_dir()
self.role = {
"en": "files",
"fr": "fichiers",
"zh": "文件",
"es": "archivos",
}
self.type = "file_agent"
def process(self, prompt, speech_module) -> str:
exec_success = False
prompt += f"\nWork directory: {self.work_dir}"
self.memory.push('user', prompt)
self.wait_message(speech_module)
animate_thinking("Thinking...", color="status")
answer, reasoning = self.llm_request()
exec_success, _ = self.execute_modules(answer)
answer = self.remove_blocks(answer)
self.last_answer = answer
while exec_success is False:
self.wait_message(speech_module)
animate_thinking("Thinking...", color="status")
answer, reasoning = self.llm_request()
exec_success, _ = self.execute_modules(answer)
answer = self.remove_blocks(answer)
self.last_answer = answer
return answer, reasoning
if __name__ == "__main__":

59
sources/agents/planner_agent.py
View File

@ -7,21 +7,27 @@ from sources.agents.browser_agent import BrowserAgent
from sources.tools.tools import Tools
class PlannerAgent(Agent):
def __init__(self, name, prompt_path, provider, verbose=False):
def __init__(self, name, prompt_path, provider, verbose=False, browser=None):
"""
The planner agent is a special agent that divides and conquers the task.
"""
super().__init__(name, prompt_path, provider, verbose)
super().__init__(name, prompt_path, provider, verbose, None)
self.tools = {
"json": Tools()
}
self.tools['json'].tag = "json"
self.browser = browser
self.agents = {
"coder": CoderAgent(model, name, prompt_path, provider),
"file": FileAgent(model, name, prompt_path, provider),
"web": BrowserAgent(model, name, prompt_path, provider)
"coder": CoderAgent(name, "prompts/coder_agent.txt", provider, verbose=False),
"file": FileAgent(name, "prompts/file_agent.txt", provider, verbose=False),
"web": BrowserAgent(name, "prompts/browser_agent.txt", provider, verbose=False, browser=browser)
}
self.role = {
"en": "Research, setup and code",
"fr": "Recherche, configuration et codage",
"zh": "研究，设置和编码",
"es": "Investigación, configuración y code"
}
self.role = "Research, setup and code"
self.type = "planner_agent"
def parse_agent_tasks(self, text):
@ -57,6 +63,8 @@ class PlannerAgent(Agent):
return zip(tasks_names, tasks)
def make_prompt(self, task, needed_infos):
if needed_infos is None:
needed_infos = "No needed informations."
prompt = f"""
You are given the following informations:
{needed_infos}
@ -65,31 +73,46 @@ class PlannerAgent(Agent):
"""
return prompt
def show_plan(self, json_plan):
agents_tasks = self.parse_agent_tasks(json_plan)
pretty_print(f"--- Plan ---", color="output")
for task_name, task in agents_tasks:
pretty_print(f"{task}", color="output")
pretty_print(f"--- End of Plan ---", color="output")
def process(self, prompt, speech_module) -> str:
self.memory.push('user', prompt)
self.wait_message(speech_module)
animate_thinking("Thinking...", color="status")
ok = False
agents_tasks = (None, None)
answer, reasoning = self.llm_request()
while not ok:
self.wait_message(speech_module)
animate_thinking("Thinking...", color="status")
self.memory.push('user', prompt)
answer, _ = self.llm_request()
self.show_plan(answer)
ok_str = input("Is the plan ok? (y/n): ")
if ok_str == 'y':
ok = True
else:
prompt = input("Please reformulate: ")
agents_tasks = self.parse_agent_tasks(answer)
if agents_tasks == (None, None):
return "Failed to parse the tasks", reasoning
prev_agent_answer = None
for task_name, task in agents_tasks:
pretty_print(f"I will {task_name}.", color="info")
agent_prompt = self.make_prompt(task['task'], task['need'])
agent_prompt = self.make_prompt(task['task'], prev_agent_answer)
pretty_print(f"Assigned agent {task['agent']} to {task_name}", color="info")
speech_module.speak(f"I will {task_name}. I assigned the {task['agent']} agent to the task.")
if speech_module: speech_module.speak(f"I will {task_name}. I assigned the {task['agent']} agent to the task.")
try:
self.agents[task['agent'].lower()].process(agent_prompt, None)
prev_agent_answer, _ = self.agents[task['agent'].lower()].process(agent_prompt, speech_module)
pretty_print(f"-- Agent answer ---\n\n", color="output")
self.agents[task['agent'].lower()].show_answer()
pretty_print(f"\n\n", color="output")
except Exception as e:
pretty_print(f"Error: {e}", color="failure")
speech_module.speak(f"I encountered an error: {e}")
break
self.last_answer = answer
return answer, reasoning
raise e
self.last_answer = prev_agent_answer
return prev_agent_answer, reasoning
if __name__ == "__main__":
from llm_provider import Provider

124
sources/browser.py
View File

@ -19,80 +19,75 @@ import sys
import re
from urllib.parse import urlparse
from sources.utility import pretty_print
def get_chrome_path() -> str:
if sys.platform.startswith("win"):
paths = [
"C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe",
os.path.join(os.environ.get("LOCALAPPDATA", ""), "Google\\Chrome\\Application\\chrome.exe") # User install
]
elif sys.platform.startswith("darwin"): # macOS
paths = ["/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta"]
else: # Linux
paths = ["/usr/bin/google-chrome", "/usr/bin/chromium-browser", "/usr/bin/chromium"]
for path in paths:
if os.path.exists(path) and os.access(path, os.X_OK): # Check if executable
return path
return None
def create_driver(headless=False):
chrome_options = Options()
chrome_path = get_chrome_path()
if not chrome_path:
raise FileNotFoundError("Google Chrome not found. Please install it.")
chrome_options.binary_location = chrome_path
if headless:
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--disable-dev-shm-usage")
chrome_options.add_argument("--autoplay-policy=user-gesture-required")
chrome_options.add_argument("--mute-audio")
chrome_options.add_argument("--disable-webgl")
chrome_options.add_argument("--disable-notifications")
security_prefs = {
"profile.default_content_setting_values.media_stream": 2,
"profile.default_content_setting_values.geolocation": 2,
"safebrowsing.enabled": True,
}
chrome_options.add_experimental_option("prefs", security_prefs)
chromedriver_path = shutil.which("chromedriver")
if not chromedriver_path:
chromedriver_path = chromedriver_autoinstaller.install()
if not chromedriver_path:
raise FileNotFoundError("ChromeDriver not found. Please install it or add it to your PATH.")
service = Service(chromedriver_path)
return webdriver.Chrome(service=service, options=chrome_options)
class Browser:
def __init__(self, headless=False, anticaptcha_install=False):
def __init__(self, driver, headless=False, anticaptcha_install=True):
"""Initialize the browser with optional headless mode."""
self.headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8',
'Accept-Language': 'en-US,en;q=0.9',
'Referer': 'https://www.google.com/',
}
self.js_scripts_folder = "./sources/web_scripts/" if not __name__ == "__main__" else "./web_scripts/"
self.anticaptcha = "https://chrome.google.com/webstore/detail/nopecha-captcha-solver/dknlfmjaanfblgfdfebhijalfmhmjjjo/related"
try:
chrome_options = Options()
chrome_path = self.get_chrome_path()
if not chrome_path:
raise FileNotFoundError("Google Chrome not found. Please install it.")
chrome_options.binary_location = chrome_path
if headless:
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--disable-dev-shm-usage")
chrome_options.add_argument("--autoplay-policy=user-gesture-required")
chrome_options.add_argument("--mute-audio")
chrome_options.add_argument("--disable-webgl")
chrome_options.add_argument("--disable-notifications")
security_prefs = {
"profile.default_content_setting_values.media_stream": 2, # Block webcam/mic
"profile.default_content_setting_values.notifications": 2, # Block notifications
"profile.default_content_setting_values.popups": 2, # Block pop-ups
"profile.default_content_setting_values.geolocation": 2, # Block geolocation
"safebrowsing.enabled": True, # Enable safe browsing
}
chrome_options.add_experimental_option("prefs", security_prefs)
chromedriver_path = shutil.which("chromedriver") # system installed driver.
#If not found, try auto-installing the correct version
if not chromedriver_path:
chromedriver_path = chromedriver_autoinstaller.install()
if not chromedriver_path:
raise FileNotFoundError("ChromeDriver not found. Please install it or add it to your PATH.")
service = Service(chromedriver_path)
self.driver = webdriver.Chrome(service=service, options=chrome_options)
self.driver = driver
self.wait = WebDriverWait(self.driver, 10)
self.logger = logging.getLogger(__name__)
self.logger.info("Browser initialized successfully")
except Exception as e:
raise Exception(f"Failed to initialize browser: {str(e)}")
self.load_anticatpcha()
if anticaptcha_install:
self.load_anticatpcha()
@staticmethod
def get_chrome_path() -> str:
if sys.platform.startswith("win"):
paths = [
"C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe",
os.path.join(os.environ.get("LOCALAPPDATA", ""), "Google\\Chrome\\Application\\chrome.exe") # User install
]
elif sys.platform.startswith("darwin"): # macOS
paths = ["/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta"]
else: # Linux
paths = ["/usr/bin/google-chrome", "/usr/bin/chromium-browser", "/usr/bin/chromium"]
for path in paths:
if os.path.exists(path) and os.access(path, os.X_OK): # Check if executable
return path
return None
def load_anticatpcha(self):
print("You might want to install the AntiCaptcha extension for captchas.")
self.driver.get(self.anticaptcha)
@ -258,7 +253,6 @@ class Browser:
return form_strings
except Exception as e:
self.logger.error(f"Error extracting form inputs: {str(e)}")
return [f"Error extracting form inputs."]
def get_buttons_xpath(self) -> List[str]:
@ -368,7 +362,7 @@ class Browser:
self.driver.quit()
self.logger.info("Browser closed")
except Exception as e:
raise e
self.logger.error(f"Error closing browser: {str(e)}")
def __del__(self):
"""Destructor to ensure browser is closed."""

11
sources/interaction.py
View File

@ -12,7 +12,6 @@ class Interaction:
tts_enabled: bool = True,
stt_enabled: bool = True,
recover_last_session: bool = False):
self.tts_enabled = tts_enabled
self.agents = agents
self.current_agent = None
self.router = AgentRouter(self.agents)
@ -99,17 +98,21 @@ class Interaction:
if agent is None:
return
if self.current_agent != agent and self.last_answer is not None:
## get history from previous agent, good ?
## get last history from previous agent
self.current_agent.memory.push('user', self.last_query)
self.current_agent.memory.push('assistant', self.last_answer)
self.current_agent = agent
tmp = self.last_answer
self.last_answer, _ = agent.process(self.last_query, self.speech)
if self.last_answer == tmp:
self.last_answer = None
def show_answer(self) -> None:
"""Show the answer to the user."""
if self.last_query is None:
return
self.current_agent.show_answer()
if self.tts_enabled:
if self.current_agent is not None:
self.current_agent.show_answer()
if self.tts_enabled and self.last_answer:
self.speech.speak(self.last_answer)

2
sources/language.py
View File

@ -19,7 +19,7 @@ class LanguageUtility:
text: string to analyze
Returns: ISO639-1 language code
"""
langid.set_languages(['fr', 'en', 'zh', 'es']) # ISO 639-1 codes
langid.set_languages(['fr', 'en', 'zh', 'es'])
lang, score = langid.classify(text)
return lang

425
sources/router.py
View File

@ -2,26 +2,49 @@ import os
import sys
import torch
from transformers import pipeline
# adaptive-classifier==0.0.10
from adaptive_classifier import AdaptiveClassifier
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from sources.agents.agent import Agent
from sources.agents.code_agent import CoderAgent
from sources.agents.casual_agent import CasualAgent
from sources.agents.planner_agent import PlannerAgent
from sources.agents.planner_agent import FileAgent
from sources.agents.browser_agent import BrowserAgent
from sources.language import LanguageUtility
from sources.utility import pretty_print
class AgentRouter:
"""
AgentRouter is a class that selects the appropriate agent based on the user query.
"""
def __init__(self, agents: list, model_name: str = "facebook/bart-large-mnli"):
self.model = model_name
self.pipeline = pipeline("zero-shot-classification",
model=self.model)
def __init__(self, agents: list):
self.agents = agents
self.labels = [agent.role for agent in agents]
self.lang_analysis = LanguageUtility()
self.pipelines = {
"bart": pipeline("zero-shot-classification", model="facebook/bart-large-mnli")
}
self.talk_classifier = self.load_llm_router()
self.complexity_classifier = self.load_llm_router()
self.learn_few_shots_tasks()
self.learn_few_shots_complexity()
def load_llm_router(self) -> AdaptiveClassifier:
"""
Load the LLM router model.
returns:
AdaptiveClassifier: The loaded model
exceptions:
Exception: If the safetensors fails to load
"""
path = "../llm_router" if __name__ == "__main__" else "./llm_router"
try:
talk_classifier = AdaptiveClassifier.from_pretrained(path)
except Exception as e:
raise Exception("Failed to load the routing model. Please run the dl_safetensors.sh script inside llm_router/ directory to download the model.")
return talk_classifier
def get_device(self) -> str:
if torch.backends.mps.is_available():
@ -30,24 +53,315 @@ class AgentRouter:
return "cuda:0"
else:
return "cpu"
def learn_few_shots_complexity(self) -> None:
"""
Few shot learning for complexity estimation.
Use the build in add_examples method of the Adaptive_classifier.
"""
few_shots = [
("can you find api and build a python web app with it ?", "HIGH"),
("can you lookup for api that track flight and build a web flight tracking app", "HIGH"),
("can you find a file called resume.docx on my drive?", "LOW"),
("can you write a python script to check if the device on my network is connected to the internet", "LOW"),
("can you debug this Java code? It’s not working.", "LOW"),
("can you browse the web and find me a 4090 for cheap?", "LOW"),
("can you find the old_project.zip file somewhere on my drive?", "LOW"),
("can you locate the backup folder I created last month on my system?", "LOW"),
("could you check if the presentation.pdf file exists in my downloads?", "LOW"),
("search my drive for a file called vacation_photos_2023.jpg.", "LOW"),
("help me organize my desktop files into folders by type.", "LOW"),
("write a Python function to sort a list of dictionaries by key", "LOW"),
("find the latest updates on quantum computing on the web", "LOW"),
("check if the folder ‘Work_Projects’ exists on my desktop", "LOW"),
("create a bash script to monitor CPU usage", "LOW"),
("debug this C++ code that keeps crashing", "LOW"),
("can you browse the web to find out who fosowl is ?", "LOW"),
("find the file ‘important_notes.txt’", "LOW"),
("search the web for the best ways to learn a new language", "LOW"),
("locate the file ‘presentation.pptx’ in my Documents folder", "LOW"),
("Make a 3d game in javascript using three.js", "HIGH"),
("Create a whole web app in python using the flask framework that query news API", "HIGH"),
("Find the latest research papers on AI and build a web app that display them", "HIGH"),
("Create a bash script that monitor the CPU usage and send an email if it's too high", "HIGH"),
("Make a web server in go that serve a simple html page", "LOW"),
("Make a web server in go that query a weather API and display the weather", "HIGH"),
("Make a web search for latest news on the stock market and display them", "HIGH"),
("Search the web for latest ai papers", "LOW"),
("Write a Python script to calculate the factorial of a number", "LOW"),
("Can you find a weather API and build a Python app to display current weather", "HIGH"),
("Search the web for the cheapest 4K monitor and provide a link", "LOW"),
("Create a Python web app using Flask to track cryptocurrency prices from an API", "HIGH"),
("Write a JavaScript function to reverse a string", "LOW"),
("Can you locate a file called ‘budget_2025.xlsx’ on my system?", "LOW"),
("Search the web for recent articles on space exploration", "LOW"),
("Find a public API for movie data and build a web app to display movie ratings", "HIGH"),
("Write a bash script to list all files in a directory", "LOW"),
("Check if a folder named ‘Photos_2024’ exists on my desktop", "LOW"),
("Create a Python script to rename all files in a folder based on their creation date", "LOW"),
("Search the web for tutorials on machine learning and build a simple ML model in Python", "HIGH"),
("Debug this Python code that’s throwing an error", "LOW"),
("Can you find a file named ‘meeting_notes.txt’ in my Downloads folder?", "LOW"),
("Create a JavaScript game using Phaser.js with multiple levels", "HIGH"),
("Write a Go program to check if a port is open on a network", "LOW"),
("Search the web for the latest electric car reviews", "LOW"),
("Find a public API for book data and create a Flask app to list bestsellers", "HIGH"),
("Write a Python function to merge two sorted lists", "LOW"),
("Organize my desktop files by extension and then write a script to list them", "HIGH"),
("Create a bash script to monitor disk space and alert via text file", "LOW"),
("Search X for posts about AI ethics and summarize them", "LOW"),
("Find the latest research on renewable energy and build a web app to display it", "HIGH"),
("Write a C program to sort an array of integers", "LOW"),
("Create a Node.js server that queries a public API for traffic data and displays it", "HIGH"),
("Check if a file named ‘project_proposal.pdf’ exists in my Documents", "LOW"),
("Search the web for tips on improving coding skills", "LOW"),
("Write a Python script to count words in a text file", "LOW"),
("Find a public API for sports scores and build a web app to show live updates", "HIGH"),
("Create a simple HTML page with CSS styling", "LOW"),
("hi", "LOW"),
("Bonjour", "LOW"),
("What's up ?", "LOW"),
]
texts = [text for text, _ in few_shots]
labels = [label for _, label in few_shots]
self.complexity_classifier.add_examples(texts, labels)
def learn_few_shots_tasks(self) -> None:
"""
Few shot learning for tasks classification.
Use the build in add_examples method of the Adaptive_classifier.
"""
few_shots = [
("Write a python script to check if the device on my network is connected to the internet", "coding"),
("Hey could you search the web for the latest news on the tesla stock market ?", "web"),
("I would like you to search for weather api", "web"),
("Plan a 3-day trip to New York, including flights and hotels.", "web"),
("Find on the web the latest research papers on AI.", "web"),
("Can you debug this Java code? It’s not working.", "code"),
("Can you browse the web and find me a 4090 for cheap?", "web"),
("i would like to setup a new AI project, index as mark2", "files"),
("Hey, can you find the old_project.zip file somewhere on my drive?", "files"),
("Tell me a funny story", "talk"),
("can you make a snake game in python", "code"),
("Can you locate the backup folder I created last month on my system?", "files"),
("Share a random fun fact about space.", "talk"),
("Write a script to rename all files in a directory to lowercase.", "files"),
("Could you check if the presentation.pdf file exists in my downloads?", "files"),
("Tell me about the weirdest dream you’ve ever heard of.", "talk"),
("Search my drive for a file called vacation_photos_2023.jpg.", "files"),
("Help me organize my desktop files into folders by type.", "files"),
("What’s your favorite movie and why?", "talk"),
("Search my drive for a file named budget_2024.xlsx", "files"),
("Write a Python function to sort a list of dictionaries by key", "code"),
("Find the latest updates on quantum computing on the web", "web"),
("Check if the folder ‘Work_Projects’ exists on my desktop", "files"),
("Create a bash script to monitor CPU usage", "code"),
("Search online for the best budget smartphones of 2025", "web"),
("What’s the strangest food combination you’ve heard of?", "talk"),
("Move all .txt files from Downloads to a new folder called Notes", "files"),
("Debug this C++ code that keeps crashing", "code"),
("can you browse the web to find out who fosowl is ?", "web"),
("Find the file ‘important_notes.txt’", "files"),
("Find out the latest news on the upcoming Mars mission", "web"),
("Write a Java program to calculate the area of a circle", "code"),
("Search the web for the best ways to learn a new language", "web"),
("Locate the file ‘presentation.pptx’ in my Documents folder", "files"),
("Write a Python script to download all images from a webpage", "code"),
("Search the web for the latest trends in AI and machine learning", "web"),
("Tell me about a time when you had to solve a difficult problem", "talk"),
("Organize all image files on my desktop into a folder called ‘Pictures’", "files"),
("Generate a Ruby script to calculate Fibonacci numbers up to 100", "code"),
("Find out what device are connected to my network", "code"),
("Show me how much disk space is left on my drive", "code"),
("Look up recent posts on X about climate change", "web"),
("Find the photo I took last week named sunset_beach.jpg", "files"),
("Write a JavaScript snippet to fetch data from an API", "code"),
("Search the web for tutorials on machine learning with Python", "web"),
("Locate the file ‘meeting_notes.docx’ in my Documents folder", "files"),
("Write a Python script to scrape a website’s title and links", "code"),
("Search the web for the latest breakthroughs in fusion energy", "web"),
("Tell me about a historical event that sounds too wild to be true", "talk"),
("Organize all image files on my desktop into a folder called ‘Pictures’", "files"),
("Generate a Ruby script to calculate Fibonacci numbers up to 100", "code"),
("Find recent X posts about SpaceX’s next rocket launch", "web"),
("What’s the funniest misunderstanding you’ve seen between humans and AI?", "talk"),
("Check if ‘backup_032025.zip’ exists anywhere on my drive", "files" ),
("Create a shell script to automate backups of a directory", "code"),
("Look up the top AI conferences happening in 2025 online", "web"),
("Write a C# program to simulate a basic calculator", "code"),
("Browse the web for open-source alternatives to Photoshop", "web"),
("Hey how are you", "talk"),
("Write a Python script to ping a website", "code"),
("Search the web for the latest iPhone release", "web"),
("What’s the weather like today?", "web"),
("Hi, how’s your day going?", "talk"),
("Can you find a file called resume.docx on my drive?", "files"),
("Write a simple Java program to print 'Hello World'", "code"),
("Tell me a quick joke", "talk"),
("Search online for the best coffee shops in Seattle", "web"),
("Check if ‘project_plan.pdf’ exists in my Downloads folder", "files"),
("What’s your favorite color?", "talk"),
("Write a bash script to list all files in a directory", "code"),
("Find recent X posts about electric cars", "web"),
("Hey, you doing okay?", "talk"),
("Locate the file ‘family_photo.jpg’ on my system", "files"),
("Search the web for beginner guitar lessons", "web"),
("Write a Python function to reverse a string", "code"),
("What’s the weirdest animal you know of?", "talk"),
("Organize all .pdf files on my desktop into a ‘Documents’ folder", "files"),
("Browse the web for the latest space mission updates", "web"),
("Hey, what’s up with you today?", "talk"),
("Write a JavaScript function to add two numbers", "code"),
("Find the file ‘notes.txt’ in my Documents folder", "files"),
("Tell me something random about the ocean", "talk"),
("Search the web for cheap flights to Paris", "web"),
("Check if ‘budget.xlsx’ is on my drive", "files"),
("Write a Python script to count words in a text file", "code"),
("How’s it going today?", "talk"),
("Find recent X posts about AI advancements", "web"),
("Move all .jpg files from Downloads to a ‘Photos’ folder", "files"),
("Search online for the best laptops of 2025", "web"),
("What’s the funniest thing you’ve heard lately?", "talk"),
("Write a Ruby script to generate random numbers", "code"),
("Hey, how’s everything with you?", "talk"),
("Locate ‘meeting_agenda.docx’ in my system", "files"),
("Search the web for tips on growing indoor plants", "web"),
("Write a C++ program to calculate the sum of an array", "code"),
("Tell me a fun fact about dogs", "talk"),
("Check if the folder ‘Old_Projects’ exists on my desktop", "files"),
("Browse the web for the latest gaming console reviews", "web"),
("Hi, how are you feeling today?", "talk"),
("Write a Python script to check disk space", "code"),
("Find the file ‘vacation_itinerary.pdf’ on my drive", "files"),
("Search the web for news on renewable energy", "web"),
("What’s the strangest thing you’ve learned recently?", "talk"),
("Organize all video files into a ‘Videos’ folder", "files"),
("Write a shell script to delete temporary files", "code"),
("Hey, how’s your week been so far?", "talk"),
("Search online for the top movies of 2025", "web"),
("Locate ‘taxes_2024.xlsx’ in my Documents folder", "files"),
("Tell me about a cool invention from history", "talk"),
("Write a Java program to check if a number is even or odd", "code"),
("Find recent X posts about cryptocurrency trends", "web"),
("Hey, you good today?", "talk"),
("Search the web for easy dinner recipes", "web"),
("Check if ‘photo_backup.zip’ exists on my drive", "files"),
("Write a Python script to rename files with a timestamp", "code"),
("What’s your favorite thing about space?", "talk"),
("Browse the web for the latest fitness trends", "web"),
("Move all .docx files to a ‘Work’ folder", "files"),
("I would like to make a new project called 'new_project'", "files"),
("I would like to setup a new project index as mark2", "files"),
("can you create a 3d js game that run in the browser", "code"),
("can you make a web app in python that use the flask framework", "code"),
("can you build a web server in go that serve a simple html page", "code"),
("can you find out who Jacky yougouri is ?", "web"),
("Setup a new flutter project called 'new_flutter_project'", "files"),
("can you create a new project called 'new_project'", "files"),
("can you make a simple web app that display a list of files in my dir", "code"),
("can you build a simple web server in python that serve a html page", "code"),
("find and buy me the latest rtx 4090", "web"),
("What are some good netflix show like Altered Carbon ?", "web"),
("can you find the latest research paper on AI", "web"),
("can you find research.pdf in my drive", "files"),
("hi", "talk"),
("hello", "talk"),
]
texts = [text for text, _ in few_shots]
labels = [label for _, label in few_shots]
self.talk_classifier.add_examples(texts, labels)
def llm_router(self, text: str) -> tuple:
"""
Inference of the LLM router model.
Args:
text: The input text
"""
predictions = self.talk_classifier.predict(text)
predictions = [pred for pred in predictions if pred[0] not in ["HIGH", "LOW"]]
predictions = sorted(predictions, key=lambda x: x[1], reverse=True)
return predictions[0]
def router_vote(self, text: str, labels: list, log_confidence:bool = False) -> str:
"""
Vote between the LLM router and BART model.
Args:
text: The input text
labels: The labels to classify
Returns:
str: The selected label
"""
result_bart = self.pipelines['bart'](text, labels)
result_llm_router = self.llm_router(text)
bart, confidence_bart = result_bart['labels'][0], result_bart['scores'][0]
llm_router, confidence_llm_router = result_llm_router[0], result_llm_router[1]
final_score_bart = confidence_bart / (confidence_bart + confidence_llm_router)
final_score_llm = confidence_llm_router / (confidence_bart + confidence_llm_router)
if log_confidence:
pretty_print(f"Agent choice -> BART: {bart} ({final_score_bart}) LLM-router: {llm_router} ({final_score_llm})")
return bart if final_score_bart > final_score_llm else llm_router
def classify_text(self, text: str, threshold: float = 0.4) -> list:
"""
Classify the text into labels (agent roles).
Args:
text (str): The text to classify
threshold (float, optional): The threshold for the classification.
Returns:
list: The list of agents and their scores
Classify the text using the LLM router and BART model.
"""
first_sentence = None
lang = "en"
for line in text.split("\n"):
first_sentence = line.strip()
break
if first_sentence is None:
first_sentence = text
result = self.pipeline(first_sentence, self.labels, threshold=threshold)
return result
try:
lang = self.lang_analysis.detect_language(first_sentence)
# no multilanguage support yet
labels = [agent.role["en"] for agent in self.agents]
result = self.router_vote(first_sentence, labels, log_confidence=False)
except Exception as e:
raise e
return result, lang
def estimate_complexity(self, text: str) -> str:
"""
Estimate the complexity of the text.
Args:
text: The input text
Returns:
str: The estimated complexity
"""
predictions = self.complexity_classifier.predict(text)
predictions = sorted(predictions, key=lambda x: x[1], reverse=True)
if len(predictions) == 0:
return "LOW"
complexity, confidence = predictions[0][0], predictions[0][1]
if confidence < 0.4:
return "LOW"
if complexity == "HIGH" and len(text) < 64:
return None # ask for more info
if complexity == "HIGH":
return "HIGH"
elif complexity == "LOW":
return "LOW"
pretty_print(f"Failed to estimate the complexity of the text. Confidence: {confidence}", color="failure")
return None
def find_planner_agent(self) -> Agent:
"""
Find the planner agent.
Returns:
Agent: The planner agent
"""
for agent in self.agents:
if agent.type == "planner_agent":
return agent
pretty_print(f"Error finding planner agent. Please add a planner agent to the list of agents.", color="failure")
return None
def multi_language_message(self, text: str):
pretty_print(f"选择代理时出错。路由系统尚不支持多语言", color="failure")
pretty_print(f"エージェントの選択エラー。ルーティングシステムはまだ多言語に対応していません", color="failure")
pretty_print(f"Erreur lors du choix de l'agent. Le système de routage n'est pas encore multilingue.", color="failure")
pretty_print(f"Error al elegir agente. El sistema de enrutamiento aún no es multilingüe.", color="failure")
def select_agent(self, text: str) -> Agent:
"""
@ -57,59 +371,76 @@ class AgentRouter:
Returns:
Agent: The selected agent
"""
if len(self.agents) == 0 or len(self.labels) == 0:
if len(self.agents) == 0:
return self.agents[0]
result = self.classify_text(text)
complexity = self.estimate_complexity(text)
best_agent, lang = self.classify_text(text)
if lang != "en":
self.multi_language_message(text)
if complexity == None:
pretty_print(f"Humm, the task seem complex but you gave very little information. can you clarify?", color="info")
return None
if complexity == "HIGH" and lang == "en":
pretty_print(f"Complex task detected, routing to planner agent.", color="info")
return self.find_planner_agent()
for agent in self.agents:
if result["labels"][0] == agent.role:
pretty_print(f"Selected agent: {agent.agent_name} (roles: {agent.role})", color="warning")
if best_agent == agent.role[lang]:
pretty_print(f"Selected agent: {agent.agent_name} (roles: {agent.role[lang]})", color="warning")
return agent
pretty_print(f"Error choosing agent.", color="failure")
return None
if __name__ == "__main__":
agents = [
CasualAgent("deepseek-r1:14b", "jarvis", "../prompts/casual_agent.txt", None),
BrowserAgent("deepseek-r1:14b", "browser", "../prompts/planner_agent.txt", None),
CoderAgent("deepseek-r1:14b", "coder", "../prompts/coder_agent.txt", None)
CasualAgent("jarvis", "../prompts/casual_agent.txt", None),
BrowserAgent("browser", "../prompts/planner_agent.txt", None),
CoderAgent("coder", "../prompts/coder_agent.txt", None),
FileAgent("file", "../prompts/coder_agent.txt", None)
]
router = AgentRouter(agents)
texts = [
"hi",
#"你好",
#"Bonjour",
"Write a python script to check if the device on my network is connected to the internet",
#"Peut tu écrire un script python qui vérifie si l'appareil sur mon réseau est connecté à internet?",
#"写一个Python脚本，检查我网络上的设备是否连接到互联网",
# "Peut tu écrire un script python qui vérifie si l'appareil sur mon réseau est connecté à internet?",
# "写一个Python脚本，检查我网络上的设备是否连接到互联网",
"Hey could you search the web for the latest news on the tesla stock market ?",
#"嘿，你能搜索网页上关于股票市场的最新新闻吗？",
#"Yo, cherche sur internet comment va tesla en bourse.",
# "嘿，你能搜索网页上关于股票市场的最新新闻吗？",
# "Yo, cherche sur internet comment va tesla en bourse.",
"I would like you to search for weather api and then make an app using this API",
#"我想让你搜索天气API，然后用这个API做一个应用程序",
#"J'aimerais que tu cherche une api météo et que l'utilise pour faire une application",
# "我想让你搜索天气API，然后用这个API做一个应用程序",
# "J'aimerais que tu cherche une api météo et que l'utilise pour faire une application",
"Plan a 3-day trip to New York, including flights and hotels.",
#"计划一次为期3天的纽约之旅，包括机票和酒店。",
#"Planifie un trip de 3 jours à Paris, y compris les vols et hotels.",
"Find me the latest research papers on AI.",
#"给我找最新的AI研究论文。",
#"Trouve moi les derniers papiers de recherche en IA",
# "计划一次为期3天的纽约之旅，包括机票和酒店。",
# "Planifie un trip de 3 jours à Paris, y compris les vols et hotels.",
"Find on the web the latest research papers on AI.",
# "在网上找到最新的人工智能研究论文。",
# "Trouve moi les derniers articles de recherche sur l'IA sur internet",
"Help me write a C++ program to sort an array",
#"帮我写一个C++程序来排序数组",
#"Aide moi à faire un programme c++ pour trier une array.",
"Tell me what France been up to lately",
# "告诉我法国最近在做什么",
# "Dis moi ce que la France a fait récemment",
"Who is Sergio Pesto ?",
# "谁是Sergio Pesto？",
# "Qui est Sergio Pesto ?",
# "帮我写一个C++程序来排序数组",
# "Aide moi à faire un programme c++ pour trier une array.",
"What’s the weather like today? Oh, and can you find a good weather app?",
#"今天天气怎么样？哦，你还能找到一个好的天气应用程序吗？",
#"La météo est comment aujourd'hui ? oh et trouve moi une bonne appli météo tant que tu y est.",
# "今天天气怎么样？哦，你还能找到一个好的天气应用程序吗？",
# "La météo est comment aujourd'hui ? oh et trouve moi une bonne appli météo tant que tu y est.",
"Can you debug this Java code? It’s not working.",
#"你能调试这段Java代码吗？它不起作用。",
#"Peut tu m'aider à debugger ce code java, ça marche pas",
"What's the latest brainrot on the internet ?",
#"互联网上最新的“脑残”是什么？",
#"Quel est la dernière connerie sur internet ?",
"i would like to setup a new AI project, index as mark2",
#"我想建立一个新的 AI 项目，索引为 Mark2",
#"Je voudrais configurer un nouveau projet d'IA, index Mark2",
"Hey, can you find the old_project.zip file somewhere on my drive?",
# "你能调试这段Java代码吗？它不起作用。",
# "Peut tu m'aider à debugger ce code java, ça marche pas",
#"Can you browse the web and find me a 4090 for cheap?",
#"你能浏览网页，为我找一个便宜的4090吗？",
#"Peut tu chercher sur internet et me trouver une 4090 pas cher ?",
#"Hey, can you find the old_project.zip file somewhere on my drive?",
#"嘿，你能在我驱动器上找到old_project.zip文件吗？",
#"Hé trouve moi le old_project.zip, il est quelque part sur mon disque.",
"Tell me a funny story",
#"给我讲一个有趣的故事",
#"Raconte moi une histoire drole"
"给我讲一个有趣的故事",
"Raconte moi une histoire drole"
]
for text in texts:
print("Input text:", text)

3
sources/tools/tools.py
View File

@ -35,6 +35,9 @@ class Tools():
self.current_dir = self.create_work_dir()
self.excutable_blocks_found = False
def get_work_dir(self):
return self.current_dir
def check_config_dir_validity(self):
"""
Check if the config directory is valid.

9
sources/web_scripts/inject_safety_script.js
View File

@ -16,15 +16,6 @@ Element.prototype.requestFullscreen = function() {
Element.prototype.requestPointerLock = function() {
console.log('Blocked pointer lock');
};
// Block iframe creation (optional, since browser already blocks these)
const originalCreateElement = document.createElement;
document.createElement = function(tagName) {
if (tagName.toLowerCase() === 'iframe') {
console.log('Blocked iframe creation');
return null;
}
return originalCreateElement.apply(this, arguments);
};
//block fetch
window.fetch = function() {
console.log('Blocked fetch request');