Qwen2.5-VL 零样本目标检测实战：提示词定位与 自定义数据集测试

这篇教程是我根据 Qwen2.5-VL 多模态定位能力和零样本目标检测复现过程整理出来的。重点演示如何用自然语言 prompt 让模型输出坐标，再用 supervision 转换为检测框可视化。

Qwen2.5-VL 可以根据文本描述定位图像中的目标。本文保留原始英文 prompt、模型 ID 和推理参数，避免改变模型输出。

本文会重点跑通以下流程：

• 加载 Qwen2.5-VL 并构造视觉问答输入
• 用英文 prompt 控制检测目标和空间约束
• 将 Qwen 输出解析为supervision.Detections
• 在示例图片和 自定义数据集上验证零样本检测效果

如果你正在系统学习计算机视觉、多模态模型或 项目复现，建议收藏本文；配套 notebook、示例图片和运行环境说明后续会继续整理。如果环境配置卡住，可以在评论区说明具体报错。

⚙️ 环境准备

配置 HuggingFace Token 和数据集后台 API Key，检查 GPU，安装 maestro 与 supervision。

importosfromgoogle.colabimportuserdata os.environ["HF_TOKEN"]=userdata.get("HF_TOKEN")

!nvidia-smi

!pip install-q"maestro[qwen_2_5_vl]==1.1.0rc2"!pip install-q supervision==0.26.0

# 后台可私信获得下载数据集，上传到当前目录，或替换为对应资源链接。

🧠 加载 Qwen2.5-VL 模型

使用 maestro 封装加载 Qwen2.5-VL-7B-Instruct。

frommaestro.trainer.models.qwen_2_5_vl.checkpointsimportload_model,OptimizationStrategy MODEL_ID_OR_PATH="Qwen/Qwen2.5-VL-7B-Instruct"MIN_PIXELS=512*28*28MAX_PIXELS=2048*28*28processor,model=load_model(model_id_or_path=MODEL_ID_OR_PATH,optimization_strategy=OptimizationStrategy.NONE,min_pixels=MIN_PIXELS,max_pixels=MAX_PIXELS)

🛠️ 推理与可视化工具函数

定义推理函数和图片标注函数，后续所有示例复用这套流程。

fromPILimportImagefromtypingimportOptional,Tuple,Unionfrommaestro.trainer.models.qwen_2_5_vl.inferenceimportpredict_with_inputsfrommaestro.trainer.models.qwen_2_5_vl.loadersimportformat_conversationfrommaestro.trainer.common.utils.deviceimportparse_device_specfromqwen_vl_utilsimportprocess_vision_infodefrun_qwen_2_5_vl_inference(model,processor,image:Image.Image,prompt:str,system_message:Optional[str]=None,device:str="auto",max_new_tokens:int=1024,)->Tuple[str,Tuple[int,int]]:device=parse_device_spec(device)conversation=format_conversation(image=image,prefix=prompt,system_message=system_message)text=processor.apply_chat_template(conversation,tokenize=False,add_generation_prompt=True)image_inputs,_=process_vision_info(conversation)inputs=processor(text=text,images=image_inputs,return_tensors="pt",)input_h=inputs['image_grid_thw'][0][1]*14input_w=inputs['image_grid_thw'][0][2]*14response=predict_with_inputs(**inputs,model=model,processor=processor,device=device,max_new_tokens=max_new_tokens)[0]returnresponse,(int(input_w),int(input_h))

importsupervisionassvfromPILimportImagedefannotate_image(image:Image,detections:sv.Detections)->Image:text_scale=sv.calculate_optimal_text_scale(resolution_wh=image.size)thickness=sv.calculate_optimal_line_thickness(resolution_wh=image.size)ifdetections.maskisnotNone:mask_annotator=sv.MaskAnnotator(color_lookup=sv.ColorLookup.INDEX)image=mask_annotator.annotate(image,detections)else:box_annotator=sv.BoxAnnotator(color_lookup=sv.ColorLookup.INDEX,thickness=thickness)image=box_annotator.annotate(image,detections)label_annotator=sv.LabelAnnotator(color_lookup=sv.ColorLookup.INDEX,text_color=sv.Color.BLACK,text_scale=text_scale,text_thickness=thickness-1,smart_position=True)image=label_annotator.annotate(image,detections)returnimage

🪑 示例 1：检测 each chair

用 prompt 定位图中每一把椅子。

IMAGE_PATH="/content/dog-3.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of each chair and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

👤 示例 2：检测坐着人的椅子

通过更具体的描述约束目标范围。

IMAGE_PATH="/content/dog-3.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of chair with man sitting on it and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

📌 示例 3：多类别检测

一次给出多个类别，观察模型对复杂 prompt 的处理。

IMAGE_PATH="/content/dog-3.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of chair, dog, table, shoe, light bulb, coffee, hat, glasses, car, tail, umbrella and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

IMAGE_PATH="/content/dog-3.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of chair, dog, table, shoe, coffee, hat, glasses, car, tail, umbrella and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

🥤 示例 4：检测玻璃杯

在另一张图片上测试单类别定位。

IMAGE_PATH="/content/dog-2.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of glass and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

➡️ 示例 5：检测最右侧玻璃杯

用空间关系约束目标。

IMAGE_PATH="/content/dog-2.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of glass most to the right and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

🥢 示例 6：检测吸管

继续测试细长目标定位。

IMAGE_PATH="/content/dog-2.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of straw and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

🧂 示例 7：检测 pepper 和 salt

测试两个相近小物体的定位。

IMAGE_PATH="/content/dog-2.jpeg"SYSTEM_MESSAGE=NonePROMPT="Outline the position of pepper, salt and output all the coordinates in JSON format."image=Image.open(IMAGE_PATH)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

📦 在自定义数据集上测试

下载安全背心数据集，并对测试集图片运行零样本检测。

# 如需额外的数据集下载依赖，请按数据集后台说明安装。

fromtypesimportSimpleNamespace# 从数据集后台下载并解压数据集后，修改 DATASET_DIR 指向数据集目录。DATASET_DIR="/content/dataset"# 修改为数据集后台导出的数据集目录dataset=SimpleNamespace(location=DATASET_DIR)

importsupervisionassv ds=sv.DetectionDataset.from_coco(images_directory_path=f"{dataset.location}/test",annotations_path=f"{dataset.location}/test/_annotations.coco.json",)

image_path,_,annotations=ds[0]SYSTEM_MESSAGE=NonePROMPT="Outline the position of helmet and output all the coordinates in JSON format."image=Image.open(image_path)resolution_wh=image.size response,input_wh=run_qwen_2_5_vl_inference(model=model,processor=processor,image=image,prompt=PROMPT,system_message=SYSTEM_MESSAGE)print(response)

importsupervisionassv detections=sv.Detections.from_vlm(vlm=sv.VLM.QWEN_2_5_VL,result=response,input_wh=input_wh,resolution_wh=resolution_wh)image=annotate_image(image=image,detections=detections)image.thumbnail((800,800))image

📌 小结

这篇教程完整整理了Zero-Shot Object Detection with Qwen2.5-VL的核心复现流程。实际复现时，建议先确认 API Key、GPU、依赖版本和数据集路径，再逐段运行 notebook。

• 加载 Qwen2.5-VL 并构造视觉问答输入
• 用英文 prompt 控制检测目标和空间约束
• 将 Qwen 输出解析为supervision.Detections
• 在示例图片和 自定义数据集上验证零样本检测效果

后续我会继续按源项目顺序整理 项目教程 中的目标检测、实例分割、OCR、多目标跟踪和视觉大模型教程。

📚 同系列教程汇总

• Google Gemini 3.5 Flash 零样本目标检测教程：从提示词到可视化结果

• GLM-OCR 文档识别实战教程：从验证码、公式到车牌 OCR

• RF-DETR + ByteTrack 多目标跟踪实战教程：从命令行到 Python 视频轨迹可视化

• SAM 3 图像分割实战教程：文本、框和点提示的多种分割方式

• SAM 3 视频分割实战教程：用文本提示分割并跟踪视频中的目标