Skip to content
zaryif
mouseclaw
Public

About

Select text anywhere → Floating panel appears → Pick an action → AI responds in real-time → Or delegate to OpenClaw agent for autonomous work

Resources

Readme
MIT license

Contributing

Contributing
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository
BranchesTags

Repository files navigation

🐾 MouseClaw — AI Text Companion with OpenClaw Integration

Your cursor's AI companion. Select text anywhere → get AI instantly → delegate to your autonomous agent.

Swift macOS License OpenClaw Version

What is MouseClaw?

MouseClaw is a system-level AI text companion for macOS that lives at the intersection of two worlds:

  1. Instant AI Actions — Select any text on your screen and get a floating Liquid Glass panel with AI-powered actions (translate, summarize, polish, explain, search, etc.)
  2. OpenClaw Delegation — Send selected text and context directly to your locally-running OpenClaw agent for autonomous task execution.

How It Works

Select text anywhere → Floating panel appears → Pick an action → AI responds in real-time
                                               → Or delegate to OpenClaw agent for autonomous work

Key Features

  • 🖱️ System-wide text selection detection — Works in every app: browsers, editors, PDFs, terminals
  • ⚡ Streaming AI responses — Real-time token-by-token results via Gemini, Claude, GPT-4o, or Ollama
  • 🐾 OpenClaw integration — Delegate complex tasks to your autonomous AI agent
  • 🎨 Liquid Glass UI — Beautiful floating panel with smooth animations and native macOS feel
  • 🎙️ Voice AI companion — Push-to-talk voice commands with screen-aware context (Clicky)
  • ⚙️ Fully configurable — Choose actions, AI providers, languages, triggers, and custom task templates
  • 🔒 Privacy-first — All processing through your own API keys or local Ollama models
  • 📱 Menu bar app — Runs silently in background, no Dock clutter
  • 🔄 Automatic retry with exponential backoff — Handles API rate limits gracefully
  • ⏱️ Customizable panel delay — Set how long to wait after selection (0.1s to 10s)
  • 🎯 Smart panel positioning — Automatically adjusts for screen edges and multi-monitor setups
  • ⌨️ Keyboard shortcuts — Trigger panel with ⌘⇧M from anywhere
  • 📊 Multiple workflow modes — Interactive, Direct, or Silent background processing

Architecture

┌──────────────────────────────────────────────────────────────────┐
│                        MouseClaw.app                             │
│                                                                  │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐              │
│  │  Layer 1    │  │  Layer 2    │  │  Layer 3    │              │
│  │  Event      │─▶│  Text       │─▶│  Floating   │              │
│  │  Monitor    │  │  Extractor  │  │  Panel      │              │
│  │(CGEventTap) │  │(AXUIElement)│  │  (NSPanel)  │              │
│  └─────────────┘  └─────────────┘  └──────┬──────┘              │
│                                           │                      │
│                              ┌────────────┴────────────┐         │
│                     ┌────────▼────────┐  ┌─────────────▼──────┐  │
│                     │    Layer 4      │  │     Layer 5        │  │
│                     │    AI Router    │  │  OpenClaw Bridge   │  │
│                     │  (Direct LLM)  │  │  (WebSocket Client)│  │
│                     └────────────────┘  └────────────────────┘  │
│                              │                                   │
│                     ┌────────▼────────┐                          │
│                     │    Layer 6      │                          │
│                     │  Voice / TTS    │                          │
│                     │  (Clicky AI)    │                          │
│                     └────────────────┘                          │
└──────────────────────────────────────────────────────────────────┘

See idea.md for the full architecture document.

Project Structure

MouseClaw/
├── MouseClaw/
│   ├── App/              ← Entry point, lifecycle, menu bar
│   ├── Core/             ← Event monitoring, text extraction, selection context
│   ├── AI/               ← AI provider protocol, router, Gemini/Claude/GPT/Ollama/Puter
│   ├── Actions/          ← Action definitions, system prompts, executor
│   ├── OpenClaw/         ← WebSocket bridge to OpenClaw Gateway
│   ├── Panel/            ← Floating NSPanel, SwiftUI views, animations
│   ├── Overlay/          ← Screen overlay system
│   ├── Settings/         ← Settings manager, settings UI tabs
│   ├── MenuBar/          ← Menu bar dropdown UI
│   ├── Onboarding/       ← First-launch setup wizard
│   ├── Notifications/    ← macOS notification manager
│   ├── DesignSystem/     ← Shared design tokens, Liquid Glass styles
│   ├── Voice/            ← Push-to-talk voice input, Clicky AI companion
│   ├── TTS/              ← Text-to-speech engine
│   ├── Recording/        ← Audio recording utilities
│   ├── ScreenCapture/    ← Screen capture for vision-aware context
│   └── Resources/        ← Assets, Info.plist, entitlements
├── worker/               ← Background worker utilities
├── run.sh                ← Build & run script (CLI)
├── project.yml           ← XcodeGen project definition
├── idea.md               ← Full architecture & design document
├── README.md             ← This file
├── USER_GUIDE.md         ← Comprehensive user guide
├── USERGUIDE.md          ← Quick-start guide
├── FEATURES.md           ← Detailed feature documentation
├── CHANGELOG.md          ← Version history and changes
├── CONTRIBUTING.md       ← Contribution guidelines
├── LICENSE               ← MIT License
└── .gitignore            ← Git ignore rules

Requirements

  • macOS 12.0+ (Monterey or later)
  • Xcode 16+
  • Swift 6.0+
  • Accessibility Permission (System Settings → Privacy & Security → Accessibility)
  • At least one AI provider API key (Gemini, Claude, OpenAI) or local Ollama
  • (Optional) OpenClaw Gateway running locally for agent delegation

Getting Started

1. Clone & Build

git clone https://github.com/zaryif/mouseclaw.git
cd mouseclaw

Option A — Xcode

open MouseClaw.xcodeproj
# Select MouseClaw scheme → ⌘R

Option B — Command Line

chmod +x run.sh
./run.sh

The run.sh script will build the project, kill any existing instance, and launch MouseClaw from DerivedData.

2. Grant Permissions

  • Accessibility — System Settings → Privacy & Security → Accessibility → Enable MouseClaw
  • Microphone (optional, for voice) — Grant when prompted

3. Configure

  • Click the 🐾 icon in the menu bar
  • Open Settings (⌘,)
  • Add your AI provider API key(s)
  • (Optional) Configure OpenClaw Gateway connection

AI Providers

Provider Model Best For Rate Limiting
Google Gemini 2.5 Flash Speed, cost, search grounding ✅ Automatic retry
Anthropic Claude Sonnet 4 Quality rewrites, nuance ✅ Automatic retry
OpenAI GPT-4o General fallback ✅ Automatic retry
Ollama Any local model Offline, privacy N/A (local)
OpenRouter Custom models Access to many models ✅ Automatic retry
Puter Gemini-based Free tier option ✅ Automatic retry

OpenClaw Integration

MouseClaw connects to OpenClaw's Gateway via WebSocket (ws://localhost:18789) to delegate complex tasks:

  • Ask Agent — Free-form question with selection context
  • Research This — Agent does deep web research and saves notes
  • Remember This — Save to agent's persistent memory
  • Create Task — Extract deadlines, add to calendar
  • Draft Message — Agent writes a reply/email
  • Run Command — Agent executes terminal commands
  • Custom Task — Configurable YAML task templates

Features

Trigger Methods

  • Auto (after selection) — Panel appears after configurable delay (0.1s to 10s)
  • Auto (immediate) — Panel appears instantly on selection
  • Keyboard shortcut only — Use ⌘⇧M to trigger manually
  • Trackpad gesture only — Use trackpad gestures to trigger
  • Manual only (menu bar) — Trigger from menu bar dropdown

Monitoring Modes

  • Active (Enhanced) — Most reliable selection detection (uses defaultTap)
  • Passive (Compatible) — Avoids keyboard shortcut conflicts (uses listenOnly)

Workflow Modes

  • Interactive (Action Grid) — Select text → choose action → see result
  • Direct (Auto-execute) — Select text → configured action runs instantly
  • Silent (Replace without UI) — Select text → action runs silently → text replaced/copied

Voice AI (Clicky Companion)

  • Push-to-talk — Hold a key to speak commands
  • Screen-aware context — Clicky sees what's on your screen
  • Natural language actions — "Summarize this", "Translate to French", etc.
  • Text-to-speech responses — Hear answers spoken aloud

Panel Customization

  • Panel Width — 250px to 600px
  • Corner Radius — 6px to 30px
  • Panel Opacity — 0.0 to 1.0
  • Enable Animations — Toggle smooth panel transitions
  • Compact Mode — Smaller panel for minimal UI

Output Options

  • Show in panel — Display result in floating panel
  • Replace selected text — Replace text in the original app
  • Copy to clipboard — Copy result to clipboard
  • Show + copy to clipboard — Both display and copy

Translation

  • Target language — Choose from 25+ languages
  • Supported languages — Bangla, English, Spanish, French, German, Japanese, Chinese, Hindi, Arabic, and more

Configuration

Settings Sections

General

  • Workflow mode selection
  • Trigger method and delay
  • Monitoring mode (Active/Passive)
  • Minimum selection length
  • Panel appearance settings
  • Accessibility permission status

Actions

  • Enable/disable individual actions
  • Reorder action priority
  • Show/hide OpenClaw actions

AI Providers

  • Add API keys for each provider
  • Switch between providers
  • Configure Ollama host and model
  • Configure OpenRouter model selection

Translation

  • Set target language
  • View supported languages list

OpenClaw

  • Enable/disable OpenClaw integration
  • Configure Gateway URL and auth token
  • Auto-start Gateway option
  • Configure default session
  • Show tool progress
  • Notify on task completion

Keyboard Shortcuts

Shortcut Action
⌘⇧M Trigger MouseClaw panel
⌘, Open Settings

Troubleshooting

Panel not appearing after selection

  1. Check Accessibility permission is granted in System Settings
  2. Verify trigger mode is not set to "Keyboard shortcut only"
  3. Try switching monitoring mode between Active and Passive
  4. Check if the app is blacklisted in settings

Rate limit errors

The app automatically retries with exponential backoff (1s, 2s, 4s, 8s, 16s). If you still see rate limit errors:

  • Switch to a different AI provider
  • Reduce the frequency of requests
  • Use a local Ollama model for unlimited requests

OpenClaw connection issues

  1. Ensure OpenClaw Gateway is running on port 18789
  2. Check the Gateway URL in settings
  3. Verify auth token if required
  4. Check Gateway logs for connection errors

Development

Building from Source

git clone https://github.com/zaryif/mouseclaw.git
cd mouseclaw

# Option 1: Xcode
open MouseClaw.xcodeproj

# Option 2: CLI
./run.sh

Adding a New AI Provider

  1. Create a new struct conforming to AIProvider in MouseClaw/AI/
  2. Implement the stream method with SSE streaming
  3. Add the provider to AIRouter.swift refresh logic
  4. Add API key field to SettingsManager.swift
  5. Add settings UI in AISettingsView.swift

Adding a New Action

  1. Add case to MouseClawAction enum in MouseClaw/Actions/MouseClawAction.swift
  2. Add system prompt to systemPrompt property
  3. Add icon to icon property
  4. Action will automatically appear in settings

Version Control

MouseClaw uses semantic versioning (MAJOR.MINOR.PATCH). See CHANGELOG.md for version history.

# List available tags
git tag

# Checkout a specific version
git checkout v1.1.0

# Or build from a specific commit
git checkout <commit-hash>

Contributing

Contributions are welcome! Please read CONTRIBUTING.md for guidelines.

License

MIT License — see LICENSE for details.

Acknowledgments

  • Built with Swift 6.0 and SwiftUI
  • Uses CGEventTap for system-wide event monitoring
  • Uses AXUIElement for text extraction
  • Integrates with OpenClaw for autonomous agent capabilities
  • AI providers: Google Gemini, Anthropic Claude, OpenAI GPT-4o, Ollama, OpenRouter, Puter

Built with 🐾 by Zaryif — April 2026

About

Select text anywhere → Floating panel appears → Pick an action → AI responds in real-time → Or delegate to OpenClaw agent for autonomous work

Resources

Readme
MIT license

Contributing

Contributing
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages