i+!dZddlmZddlZddlZddlZddlmZddlm Z m Z m Z m Z m Z ejeZehdZd!d Zd"dZd#dZd$dZd%dZd&dZd'dZdd gZdS)(uRouting helpers for inbound user-attached images. Two modes: native — attach images as OpenAI-style ``image_url`` content parts on the user turn. Provider adapters (Anthropic, Gemini, Bedrock, Codex, OpenAI chat.completions) already translate these into their vendor-specific multimodal formats. text — run ``vision_analyze`` on each image up-front and prepend the description to the user's text. The model never sees the pixels; it only sees a lossy text summary. This is the pre-existing behaviour and still the right choice for non-vision models. The decision is made once per message turn by :func:`decide_image_input_mode`. It reads ``agent.image_input_mode`` from config.yaml (``auto`` | ``native`` | ``text``, default ``auto``) and the active model's capability metadata. In ``auto`` mode: - If the user has explicitly configured ``auxiliary.vision.provider`` (i.e. not ``auto`` and not empty), we assume they want the text pipeline regardless of the main model — they've opted in to a specific vision backend for a reason (cost, quality, local-only, etc.). - Otherwise, if the active model reports ``supports_vision=True`` in its models.dev metadata, we attach natively. - Otherwise (non-vision model, no explicit override), we fall back to text. This keeps ``vision_analyze`` surfaced as a tool in every session — skills and agent flows that chain it (browser screenshots, deeper inspection of URL-referenced images, style-gating loops) keep working. The routing only affects *how user-attached images on the current turn* are presented to the main model. ) annotationsN)Path)AnyDictListOptionalTuple>autotextnativerawrreturnstrct|tsdS|}|tvr|SdS)z5Normalize a config value into one of the valid modes.r ) isinstancerstriplower _VALID_MODES)r vals 8/home/ubuntu/.hermes/hermes-agent/agent/image_routing.py _coerce_moder1sG c3  v ))++    C l 6cfgOptional[Dict[str, Any]]boolcht|tsdS|dpi}t|tsdS|dpi}t|tsdSt|dpd}t|dpd}t|dpd}|dvr|s|sdSd S) zTrue when the user configured a specific auxiliary vision backend. An explicit override means the user *wants* the text pipeline (they're paying for a dedicated vision model), so we don't silently bypass it. F auxiliaryvisionprovidermodelbase_url)r r T)rdictgetrrr)rauxrrr!r"s r_explicit_aux_vision_overrider&;s  c4 u ''+   $"C c4 u WWX   $"F fd # #u6::j))/R006688>>@@H  7##)r * * 0 0 2 2E6::j))/R006688H<hu 4rrr!Optional[bool]c|r|sdS ddlm}|||}n5#t$r(}td|||Yd}~dSd}~wwxYw|dSt |jS)z:Return True/False if we can resolve caps, None if unknown.Nr)get_model_capabilitiesu2image_routing: caps lookup failed for %s:%s — %s)agent.models_devr) Exceptionloggerdebugrsupports_vision)rr!r)capsexcs r_lookup_supports_visionr1Ts 5t;;;;;;%%h66  I8UZ\_```ttttt |t $ % %%s A AA c>d}t|trN|dpi}t|tr"t|d}|dkrdS|dkrdSt |rdSt ||}|durdSdS)a1Return ``"native"`` or ``"text"`` for the given turn. Args: provider: active inference provider ID (e.g. ``"anthropic"``, ``"openrouter"``). model: active model slug as it would be sent to the provider. cfg: loaded config.yaml dict, or None. When None, behaves as auto. r agentimage_input_moder r T)rr#r$rr&r1)rr!rmode_cfg agent_cfgsupportss rdecide_image_input_moder8csH#tGGGG$$* i & & G#IMM2D$E$EFFH8x6v%S))v&x77H4x 6rpathrctjt|\}}|r|dr|S|j}ddddddd|dS)Nzimage/z image/jpegz image/pngz image/gifz image/webpz image/bmp)z.jpgz.jpegz.pngz.gifz.webpz.bmp) mimetypes guess_typer startswithsuffixrr$)r9mime_r>s r _guess_mimerAs"3t99--GD! )) [   F    c&, r Optional[str]c |}n4#t$r'}td||Yd}~dSd}~wwxYwt |}t j|d}d|d|S)u!Encode a local image as a base64 data URL at its native size. Size limits are NOT enforced here — the agent retry loop (``run_agent._try_shrink_image_parts_in_messages``) shrinks on the provider's first rejection. Keeping this simple means providers that accept large images (OpenAI 49 MB+, Gemini 100 MB) don't pay a silent quality tax just because one other provider is stricter. Returns None only if the file can't be read (missing, permission denied, etc.); the caller reports those paths in ``skipped``. u'image_routing: failed to read %s — %sNasciizdata:z;base64,) read_bytesr+r,warningrAbase64 b64encodedecode)r9r r0r?b64s r_file_to_data_urlrKsoo @$LLLttttt t  D  3   & &w / /C &4 & & & &&s AAA user_text image_paths List[str]&Tuple[List[Dict[str, Any]], List[str]]c4g}g}|pd}|r|d|d|D]}t|}|r|s#|t |\t |}|s#|t ||dd|id|s2td|Dr|ddd d||fS) uVBuild an OpenAI-style ``content`` list for a user turn. Shape: [{"type": "text", "text": "..."}, {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}, ...] Images are attached at their native size. If a provider rejects the request because an image is too large (e.g. Anthropic's 5 MB per-image ceiling), the agent's retry loop transparently shrinks and retries once — see ``run_agent._try_shrink_image_parts_in_messages``. Returns (content_parts, skipped_paths). Skipped paths are files that couldn't be read from disk. r r )typer image_urlurl)rQrRc3HK|]}|ddkVdS)rQrRN)r$).0ps r z-build_native_content_parts..s1DDf 4DDDDDDrrzWhat do you see in this image?) rappendrexistsis_filerrKanyinsert)rLrMpartsskippedr raw_pathrVdata_urls rbuild_native_content_partsrasJ&#%EG O " " $ $D 5 fd33444    NNxxzz   NN3x== ) ) ) $Q''  NN3x== ) ) )  *      TCDDeDDDDDT Q1QRRSSS '>rr8ra)r rrr)rrrr)rrr!rrr')rrr!rrrrr)r9rrr)r9rrrB)rLrrMrNrrO)__doc__ __future__rrGloggingr;pathlibrtypingrrrrr getLogger__name__r, frozensetrrr&r1r8rArKra__all__rrrlsI  D#""""" 33333333333333  8 $ $y33344 2 & & & &`    "'''',,,,,`  r