Screenshot Assistant

A Wayland screenshot assistant with LLM integration. Capture screenshots and analyze them using Ollama's LLaVA model.

Installation

If using uv, skip the pip install step or use uv pip install -r requirements.txt.

git clone https://github.com/tcsenpai/screenshot-assistant.git
cd screenshot-assistant
pip install -r requirements.txt

Requirements

Wayland compositor (Sway, Hyprland, or other wlroots-based)
grim (screenshot utility)
slurp (area selection)
imv (image viewer)
Ollama with LLaVA model installed

You can install the requirements (except for Ollama) using the provided install script:

./ubuntu_deps.sh

Pulling Ollama models

ollama pull llava-llama3 # or whatever model you want to use

Usage

You can run the assistant using the provided run script:

./run

Configuration

Create a .env file with the following options:

OLLAMA_URL=http://localhost:11434/api/generate
OLLAMA_MODEL=llava
WINDOW_TITLE=Screenshot Assistant
SCREENSHOT_MODE=active  # Options: all, active, select