gimbal_color_detect
Skill37 starsChanged 33 days agoMITView on GitHub
Use this skill when the user wants esp-claw to use the camera as a visual servo target tracker. If the user asks to track "this object", "the object in front", "the thing in front", "这个物体", "前面这个东西", or any other visible object without naming a specific color, you MUST NOT start color tracking directly and MUST NOT fall back to the default green preset. First use the take_picture skill to capture the object, inspect the saved photo with AI vision, identify…
In this repository24 other files it gives its agents.
- air_mouseSkill
- bilibili_up_fansSkill
- china_a_share_quoteSkill
- cloudmusicSkill
- current_ip_infoSkill
- current_weatherSkill
- dfrobot_matrix_lidar_8x8_i2cSkill
- dfrobot_stcc4_i2cSkill
- example_skillSkill
- flashbackSkill
- gimbal_motion_detectSkill
- github_repo_starSkill
- movement_detectionSkill
- network_radioSkill
- servo_dogSkill
- stock_quotes_displaySkill
- token_usageSkill
- unihiker_buttonSkill
- unihiker_expansion_batterySkill
- unihiker_expansion_irSkill
- unihiker_expansion_motor_controlSkill
- unihiker_expansion_servo_controlSkill
- unihiker_expansion_WS2812Skill
- weather_clockSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
13 rules. About 0 are common, 3 are distinctive, and 10 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
10
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 20
Use this skill when the user wants esp-claw to use the camera as a visual servo target tracker.
- Distinctive: line 25
If the user asks to track "this object", "the object in front", "the thing in front", "这个物体", "前面这个东西", or any other visible object without …
- Very specific: line 33
If the recognized color is one of the built-in presets, pass only target_color_name.
- Distinctive: line 186
Start tracking a built-in blue preset:
- Distinctive: line 201
Start tracking a custom AI-returned color:
- Very specific: line 222
When the user says "track this object" / "追踪这个物体", activate the take_picture skill and follow its own script instructions.
- Very specific: line 227
After this call returns the saved JPEG path, inspect that image with AI vision.
- Very specific: line 244
If the user refers to a visible object instead of naming a color, run the take_picture skill first.
- Very specific: line 247
If the recognized color is one of red, orange, yellow, green, cyan, blue, or purple, register it by passing only target_color_name.
- Very specific: line 248
If the recognized color is not one of those presets, ask AI vision for a practical HSV range and register it with target_color_name plus …
- Very specific: line 249
If the user gives no color and does not refer to a specific visible object, run the tracker with no color args.
- Very specific: line 250
Run the tracking script with lua_run_script_async.
- Very specific: line 251
Use timeout_ms: 0, name: "gimbal_color_detect", exclusive: "gimbal_color_detect", and replace: false.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Gimbal Color Detect
Use this skill when the user wants esp-claw to use the camera as a visual
servo target tracker.
Visible Object Rule
If the user asks to track "this object", "the object in front", "the thing in
front", "这个物体", "前面这个东西", or any other visible object without naming a
specific color, you MUST NOT start color tracking directly and MUST NOT fall
back to the default green preset. First use the take_picture skill to capture
the object, inspect the saved photo with AI vision, identify the intended
foreground object's dominant visible color, tell the user the recognized color,
and only then start this color tracking script with the recognized color.
If the recognized color is one of the built-in presets, pass only
target_color_name. If it is not a preset, ask AI vision to return an HSV range
and pass target_h_min, target_h_max, target_s_min, target_s_max,
target_v_min, and target_v_max.
The built-in green preset is used only when the user gives no color and also
does not refer to a specific visible object. HSV thresholds match ESP-DL:
hue uses 0..180, saturation uses 0..255, and value uses 0..255.
If target_h_min > target_h_max, the
script treats the hue range as wrapping through 0, which is useful for red
targets.
Preview display follows the esp-hello-new/examples/gimbal_base camera path by
default: crop a centered 240x240 square from the board-default camera stream,
and draw it centered on the 284x240 LCD. Detection runs on that same centered
crop so off-screen colors do not affect the tracked box. Each display frame
uses screen:begin(), draws the image and overlay, then submits with
screen:present({ full = true }). Camera discovery uses camera.list_devices();
the built-in screen is opened with display.open().
Default Hardware
- Camera device: from
camera.list_devices() - Camera stream: board default mode from
camera.open(dev_path) - LCD panel: from
display.open() - X servo GPIO:
4 - Y servo GPIO:
5 - PWM driver:
ledc,50 Hz,500 us .. 2500 us - X angle range:
0 .. 180, initial90 - Y angle range:
10 .. 70, initial50 - Runtime shape: a single async Lua job owns camera, LCD, color detection, and X/Y LEDC servos.
Start Script Args Schema
{
"type": "object",
"properties": {
"run_seconds": {
"type": "integer",
"default": 0,
"description": "0 means run until the async Lua job is stopped."
},
"frame_interval_ms": {
"type": "integer",
"default": 0,
"description": "Extra delay after each frame. Keep 0 for maximum camera/display throughput."
},
"display_every_n": {
"type": "integer",
"default": 1
},
"perf_log_every_n": {
"type": "integer",
"default": 10,
"description": "Print average get/convert/detect/display/loop timing every N frames. Set 0 to disable."
},
"display_crop_size": {
"type": "integer",
"default": 240,
"description": "Centered square crop size before drawing to the LCD."
},
"detect_stride": {
"type": "integer",
"default": 4,
"description": "Kept for compatibility. ESP-DL color detection evaluates the HSV mask for the selected source crop."
},
"detect_min_pixels": {
"type": "integer",
"default": 250,
"description": "Minimum connected-component size, matching the ESP-DL gimbal_base detector scale."
},
"detect_max_blob_percent": {
"type": "integer",
"default": 35,
"description": "Reject connected components larger than this percentage of the display crop."
},
"target_color_name": {
"type": "string",
"default": "green",
"description": "Registered target color name. Built-in presets: red, orange, yellow, green, cyan, blue, purple."
},
"target_h_min": {
"type": "integer",
"description": "Custom registered target hue lower bound, 0..180. Required only when target_color_name is not a built-in preset."
},
"target_h_max": {
"type": "integer",
"description": "Custom registered target hue upper bound, 0..180. Values lower than target_h_min wrap through 0."
},
"target_s_min": {
"type": "integer",
"description": "Custom registered target saturation lower bound, 0..255."
},
"target_s_max": {
"type": "integer",
"description": "Custom registered target saturation upper bound, 0..255."
},
"target_v_min": {
"type": "integer",
"description": "Custom registered target value lower bound, 0..255."
},
"target_v_max": {
"type": "integer",
"description": "Custom registered target value upper bound, 0..255."
},
"deadzone_px": {
"type": "integer",
"default": 12
},
"x_gain": {
"type": "number",
"default": 0.035
},
"y_gain": {
"type": "number",
"default": 0.035
},
"max_track_step_degrees": {
"type": "number",
"default": 2
},
"x_gpio": {
"type": "integer",
"default": 4
},
"y_gpio": {
"type": "integer",
"default": 5
},
"servo_max_step_degrees": {
"type": "number",
"default": 2
}
}
}Tool Call Inputs
Start color tracking:
{
"path": "{CUR_SKILL_DIR}/scripts/start_gimbal_color_detect.lua",
"args": {},
"timeout_ms": 0,
"name": "gimbal_color_detect",
"exclusive": "gimbal_color_detect",
"replace": false
}Start tracking a built-in blue preset:
{
"path": "{CUR_SKILL_DIR}/scripts/start_gimbal_color_detect.lua",
"args": {
"target_color_name": "blue"
},
"timeout_ms": 0,
"name": "gimbal_color_detect",
"exclusive": "gimbal_color_detect",
"replace": false
}Start tracking a custom AI-returned color:
{
"path": "{CUR_SKILL_DIR}/scripts/start_gimbal_color_detect.lua",
"args": {
"target_color_name": "custom teal",
"target_h_min": 78,
"target_h_max": 92,
"target_s_min": 64,
"target_s_max": 255,
"target_v_min": 46,
"target_v_max": 255
},
"timeout_ms": 0,
"name": "gimbal_color_detect",
"exclusive": "gimbal_color_detect",
"replace": false
}When the user says "track this object" / "追踪这个物体", activate the
take_picture skill and follow its own script instructions. Pass
filename: "gimbal_target.jpg" and skip_frames: 3; resolve its script path
from that skill's directory, not this package.
After this call returns the saved JPEG path, inspect that image with AI vision.
Then start start_gimbal_color_detect.lua using the recognized preset color or
custom HSV range. Do not skip the photo and do not start with empty args for a
visible-object request.
Stop color tracking:
{
"name": "gimbal_color_detect",
"wait_ms": 2000
}Recommended Flow
- Activate
board_hardware_infofirst when the user wants pin validation. - If the user refers to a visible object instead of naming a color, run the
take_pictureskill first. Use a simple filename such asgimbal_target.jpg. - Inspect the saved photo with AI vision and identify the intended foreground object's dominant visible color.
- Tell the user the recognized color before starting tracking.
- If the recognized color is one of
red,orange,yellow,green,cyan,blue, orpurple, register it by passing onlytarget_color_name. - If the recognized color is not one of those presets, ask AI vision for a practical HSV range and register it with
target_color_nameplustarget_h_min,target_h_max,target_s_min,target_s_max,target_v_min, andtarget_v_max. - If the user gives no color and does not refer to a specific visible object, run the tracker with no color args; it defaults to the built-in
greenpreset. - Run the tracking script with
lua_run_script_async. - Use
timeout_ms: 0,name: "gimbal_color_detect",exclusive: "gimbal_color_detect", andreplace: false. - Tune
x_gain/y_gainsigns if servo direction is reversed on a specific mount.
Color Registration Hints
The script has built-in presets for these colors. When AI identifies one of
these names, pass only target_color_name; do not pass HSV values unless you
need to override the preset for local lighting. For red, the preset uses a hue
range that wraps through 0. Chinese names 红色, 橙色, 黄色, 绿色,
青色, 蓝色, and 紫色 are accepted as aliases.
| Color | target_h_min | target_h_max | Notes |
| red | 170 | 10 | Wraps through 0 |
| orange | 8 | 24 | Often needs higher target_s_min |
| yellow | 22 | 40 | Sensitive to warm lighting |
| green | 50 | 88 | Default when no color is specified |
| cyan | 82 | 100 | Narrow if blue objects also appear |
| blue | 95 | 130 | Use lower target_v_min for dark blue |
| purple | 128 | 158 | Includes violet/magenta edge |
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

