agentleFS
espressifespressif/esp-claw-skills-lab/skills/gimbal_color_detect/SKILL.md

gimbal_color_detect

Skill37 starsChanged 33 days agoMITView on GitHub

Use this skill when the user wants esp-claw to use the camera as a visual servo target tracker. If the user asks to track "this object", "the object in front", "the thing in front", "这个物体", "前面这个东西", or any other visible object without naming a specific color, you MUST NOT start color tracking directly and MUST NOT fall back to the default green preset. First use the take_picture skill to capture the object, inspect the saved photo with AI vision, identify…

The file

Show its rules and text

We split this skill into the rules it gives an agent

13 rules. About 0 are common, 3 are distinctive, and 10 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈3

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 10

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Use this skill when the user wants esp-claw to use the camera as a visual servo target tracker.

    line 20
  2. Distinctive:

    If the user asks to track "this object", "the object in front", "the thing in front", "这个物体", "前面这个东西", or any other visible object without …

    line 25
  3. Very specific:

    If the recognized color is one of the built-in presets, pass only target_color_name.

    line 33
  4. Distinctive:

    Start tracking a built-in blue preset:

    line 186
  5. Distinctive:

    Start tracking a custom AI-returned color:

    line 201
  6. Very specific:

    When the user says "track this object" / "追踪这个物体", activate the take_picture skill and follow its own script instructions.

    line 222
  7. Very specific:

    After this call returns the saved JPEG path, inspect that image with AI vision.

    line 227
  8. Very specific:

    If the user refers to a visible object instead of naming a color, run the take_picture skill first.

    line 244
  9. Very specific:

    If the recognized color is one of red, orange, yellow, green, cyan, blue, or purple, register it by passing only target_color_name.

    line 247
  10. Very specific:

    If the recognized color is not one of those presets, ask AI vision for a practical HSV range and register it with target_color_name plus …

    line 248
  11. Very specific:

    If the user gives no color and does not refer to a specific visible object, run the tracker with no color args.

    line 249
  12. Very specific:

    Run the tracking script with lua_run_script_async.

    line 250
  13. Very specific:

    Use timeout_ms: 0, name: "gimbal_color_detect", exclusive: "gimbal_color_detect", and replace: false.

    line 251

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Gimbal Color Detect

Use this skill when the user wants esp-claw to use the camera as a visual servo target tracker.

Visible Object Rule

If the user asks to track "this object", "the object in front", "the thing in front", "这个物体", "前面这个东西", or any other visible object without naming a specific color, you MUST NOT start color tracking directly and MUST NOT fall back to the default green preset. First use the take_picture skill to capture the object, inspect the saved photo with AI vision, identify the intended foreground object's dominant visible color, tell the user the recognized color, and only then start this color tracking script with the recognized color.

If the recognized color is one of the built-in presets, pass only target_color_name. If it is not a preset, ask AI vision to return an HSV range and pass target_h_min, target_h_max, target_s_min, target_s_max, target_v_min, and target_v_max.

The built-in green preset is used only when the user gives no color and also does not refer to a specific visible object. HSV thresholds match ESP-DL: hue uses 0..180, saturation uses 0..255, and value uses 0..255. If target_h_min > target_h_max, the script treats the hue range as wrapping through 0, which is useful for red targets.

Preview display follows the esp-hello-new/examples/gimbal_base camera path by default: crop a centered 240x240 square from the board-default camera stream, and draw it centered on the 284x240 LCD. Detection runs on that same centered crop so off-screen colors do not affect the tracked box. Each display frame uses screen:begin(), draws the image and overlay, then submits with screen:present({ full = true }). Camera discovery uses camera.list_devices(); the built-in screen is opened with display.open().

Default Hardware

  • Camera device: from camera.list_devices()
  • Camera stream: board default mode from camera.open(dev_path)
  • LCD panel: from display.open()
  • X servo GPIO: 4
  • Y servo GPIO: 5
  • PWM driver: ledc, 50 Hz, 500 us .. 2500 us
  • X angle range: 0 .. 180, initial 90
  • Y angle range: 10 .. 70, initial 50
  • Runtime shape: a single async Lua job owns camera, LCD, color detection, and X/Y LEDC servos.

Start Script Args Schema

{
  "type": "object",
  "properties": {
    "run_seconds": {
      "type": "integer",
      "default": 0,
      "description": "0 means run until the async Lua job is stopped."
    },
    "frame_interval_ms": {
      "type": "integer",
      "default": 0,
      "description": "Extra delay after each frame. Keep 0 for maximum camera/display throughput."
    },
    "display_every_n": {
      "type": "integer",
      "default": 1
    },
    "perf_log_every_n": {
      "type": "integer",
      "default": 10,
      "description": "Print average get/convert/detect/display/loop timing every N frames. Set 0 to disable."
    },
    "display_crop_size": {
      "type": "integer",
      "default": 240,
      "description": "Centered square crop size before drawing to the LCD."
    },
    "detect_stride": {
      "type": "integer",
      "default": 4,
      "description": "Kept for compatibility. ESP-DL color detection evaluates the HSV mask for the selected source crop."
    },
    "detect_min_pixels": {
      "type": "integer",
      "default": 250,
      "description": "Minimum connected-component size, matching the ESP-DL gimbal_base detector scale."
    },
    "detect_max_blob_percent": {
      "type": "integer",
      "default": 35,
      "description": "Reject connected components larger than this percentage of the display crop."
    },
    "target_color_name": {
      "type": "string",
      "default": "green",
      "description": "Registered target color name. Built-in presets: red, orange, yellow, green, cyan, blue, purple."
    },
    "target_h_min": {
      "type": "integer",
      "description": "Custom registered target hue lower bound, 0..180. Required only when target_color_name is not a built-in preset."
    },
    "target_h_max": {
      "type": "integer",
      "description": "Custom registered target hue upper bound, 0..180. Values lower than target_h_min wrap through 0."
    },
    "target_s_min": {
      "type": "integer",
      "description": "Custom registered target saturation lower bound, 0..255."
    },
    "target_s_max": {
      "type": "integer",
      "description": "Custom registered target saturation upper bound, 0..255."
    },
    "target_v_min": {
      "type": "integer",
      "description": "Custom registered target value lower bound, 0..255."
    },
    "target_v_max": {
      "type": "integer",
      "description": "Custom registered target value upper bound, 0..255."
    },
    "deadzone_px": {
      "type": "integer",
      "default": 12
    },
    "x_gain": {
      "type": "number",
      "default": 0.035
    },
    "y_gain": {
      "type": "number",
      "default": 0.035
    },
    "max_track_step_degrees": {
      "type": "number",
      "default": 2
    },
    "x_gpio": {
      "type": "integer",
      "default": 4
    },
    "y_gpio": {
      "type": "integer",
      "default": 5
    },
    "servo_max_step_degrees": {
      "type": "number",
      "default": 2
    }
  }
}

Tool Call Inputs

Start color tracking:

{
  "path": "{CUR_SKILL_DIR}/scripts/start_gimbal_color_detect.lua",
  "args": {},
  "timeout_ms": 0,
  "name": "gimbal_color_detect",
  "exclusive": "gimbal_color_detect",
  "replace": false
}

Start tracking a built-in blue preset:

{
  "path": "{CUR_SKILL_DIR}/scripts/start_gimbal_color_detect.lua",
  "args": {
    "target_color_name": "blue"
  },
  "timeout_ms": 0,
  "name": "gimbal_color_detect",
  "exclusive": "gimbal_color_detect",
  "replace": false
}

Start tracking a custom AI-returned color:

{
  "path": "{CUR_SKILL_DIR}/scripts/start_gimbal_color_detect.lua",
  "args": {
    "target_color_name": "custom teal",
    "target_h_min": 78,
    "target_h_max": 92,
    "target_s_min": 64,
    "target_s_max": 255,
    "target_v_min": 46,
    "target_v_max": 255
  },
  "timeout_ms": 0,
  "name": "gimbal_color_detect",
  "exclusive": "gimbal_color_detect",
  "replace": false
}

When the user says "track this object" / "追踪这个物体", activate the take_picture skill and follow its own script instructions. Pass filename: "gimbal_target.jpg" and skip_frames: 3; resolve its script path from that skill's directory, not this package.

After this call returns the saved JPEG path, inspect that image with AI vision. Then start start_gimbal_color_detect.lua using the recognized preset color or custom HSV range. Do not skip the photo and do not start with empty args for a visible-object request.

Stop color tracking:

{
  "name": "gimbal_color_detect",
  "wait_ms": 2000
}

Recommended Flow

  1. Activate board_hardware_info first when the user wants pin validation.
  2. If the user refers to a visible object instead of naming a color, run the take_picture skill first. Use a simple filename such as gimbal_target.jpg.
  3. Inspect the saved photo with AI vision and identify the intended foreground object's dominant visible color.
  4. Tell the user the recognized color before starting tracking.
  5. If the recognized color is one of red, orange, yellow, green, cyan, blue, or purple, register it by passing only target_color_name.
  6. If the recognized color is not one of those presets, ask AI vision for a practical HSV range and register it with target_color_name plus target_h_min, target_h_max, target_s_min, target_s_max, target_v_min, and target_v_max.
  7. If the user gives no color and does not refer to a specific visible object, run the tracker with no color args; it defaults to the built-in green preset.
  8. Run the tracking script with lua_run_script_async.
  9. Use timeout_ms: 0, name: "gimbal_color_detect", exclusive: "gimbal_color_detect", and replace: false.
  10. Tune x_gain / y_gain signs if servo direction is reversed on a specific mount.

Color Registration Hints

The script has built-in presets for these colors. When AI identifies one of these names, pass only target_color_name; do not pass HSV values unless you need to override the preset for local lighting. For red, the preset uses a hue range that wraps through 0. Chinese names 红色, 橙色, 黄色, 绿色, 青色, 蓝色, and 紫色 are accepted as aliases.

Colortarget_h_mintarget_h_maxNotes
red17010Wraps through 0
orange824Often needs higher target_s_min
yellow2240Sensitive to warm lighting
green5088Default when no color is specified
cyan82100Narrow if blue objects also appear
blue95130Use lower target_v_min for dark blue
purple128158Includes violet/magenta edge

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.