ibddlmZddlmZddlmZddlZddlmZddl m Z m Z m Z m Z mZmZmZddlmZmZmZdd lmZdd lmZmZdd lmZmZd d lmZmZm Z m!Z!m"Z"m#Z#ddl$m%Z%m&Z&ddl'm(Z(m)Z)ddl*m+Z+m,Z,ddl-m.Z.ddl/m0Z0ddl1m2Z2ddl3m4Z4ddl5m6Z6ddl7m8Z8ddgZ9GddeZ:GddeZ;GddZ<GddZ=Gdd Z>Gd!d"Z?dS)#) annotations)Optional)LiteralN)_legacy_response)BodyOmitQueryHeadersNotGivenomit not_given) path_templatemaybe_transformasync_maybe_transform)cached_property)SyncAPIResourceAsyncAPIResource)to_streamed_response_wrapper"async_to_streamed_response_wrapper) OutputItemsAsyncOutputItemsOutputItemsWithRawResponseAsyncOutputItemsWithRawResponse OutputItemsWithStreamingResponse%AsyncOutputItemsWithStreamingResponse)SyncCursorPageAsyncCursorPage)run_list_paramsrun_create_params)AsyncPaginatormake_request_options)Metadata)RunListResponse)RunCancelResponse)RunCreateResponse)RunDeleteResponse)RunRetrieveResponseRuns AsyncRunsc eZdZdZed/dZed0dZed1dZeed d d e d d2dZ d d d e dd3d Z eeeed d d e d!d4d*Z d d d e dd5d,Z d d d e dd6d.Zd S)7r*,Manage and run evals in the OpenAI platform.returnrc*t|jSr-)r_clientselfs g/home/ubuntu/.hermes/hermes-agent/venv/lib/python3.11/site-packages/openai/resources/evals/runs/runs.py output_itemszRuns.output_items(s4<(((RunsWithRawResponsec t|Sa This property can be used as a prefix for any HTTP method call to return the raw response object instead of the parsed content. For more information, see https://www.github.com/openai/openai-python#accessing-raw-response-data-eg-headers )r7r2s r4with_raw_responsezRuns.with_raw_response-s#4(((r6RunsWithStreamingResponsec t|Sz An alternative to `.with_raw_response` that doesn't eagerly read the response body. For more information, see https://www.github.com/openai/openai-python#with_streaming_response )r;r2s r4with_streaming_responsezRuns.with_streaming_response7s)...r6Nmetadataname extra_headers extra_query extra_bodytimeouteval_idstr data_sourcerun_create_params.DataSourcer@Optional[Metadata] | OmitrA str | OmitrBHeaders | NonerC Query | NonerD Body | NonerE'float | httpx.Timeout | None | NotGivenr'c |std||td|t|||dtjt ||||tS)  Kicks off a new run for a given evaluation, specifying the data source, and what model configuration to use to test. The datasource will be validated against the schema specified in the config of the evaluation. Args: data_source: Details about the run's data source. metadata: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. name: The name of the run. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds 6Expected a non-empty value for `eval_id` but received /evals/{eval_id}/runsrFrHr@rArBrCrDrEbodyoptionscast_to) ValueError_postrrr!RunCreateParamsr#r' r3rFrHr@rArBrCrDrEs r4createz Runs.create@sN caV]aabb bzz 17 C C C #. (  "1 )+Q[el&   r6rVrun_idr)c |std||std||td||t||||tS)P Get an evaluation run by ID. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rR5Expected a non-empty value for `run_id` but received /evals/{eval_id}/runs/{run_id}rFr`rVrYrZr[_getrr#r)r3r`rFrBrCrDrEs r4retrievez Runs.retrieveys0 caV]aabb b a_U[__`` `yy :GTZ [ [ [(+Q[el(    r6afterlimitorderstatusrBrCrDrErlrm int | OmitrnLiteral['asc', 'desc'] | OmitroJLiteral['queued', 'in_progress', 'completed', 'canceled', 'failed'] | OmitSyncCursorPage[RunListResponse]c|std||td|ttt |||| t ||||dtjtSa Get a list of runs for an evaluation. Args: after: Identifier for the last run from the previous pagination request. limit: Number of runs to retrieve. order: Sort order for runs by timestamp. Use `asc` for ascending order or `desc` for descending order. Defaults to `asc`. status: Filter runs by status. One of `queued` | `in_progress` | `failed` | `completed` | `canceled`. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rRrSrT)rlrmrnro)rBrCrDrEquery)pagerYmodel) r[ _get_api_listrrr%r#rr RunListParams r3rFrlrmrnrorBrCrDrEs r4listz Runs.listsJ caV]aabb b!! 17 C C C0(+'%%!&!&!&"(  $1 "%"   r6r(c |std||std||td||t||||tS)G Delete an eval run. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rRrcrdrerVrfr[_deleterr#r(ris r4deletez Runs.deletes0 caV]aabb b a_U[__`` `|| :GTZ [ [ [(+Q[el&    r6r&c |std||std||td||t||||tS)U Cancel an ongoing evaluation run. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rRrcrdrerVrfr[r\rr#r&ris r4cancelz Runs.cancels0 caV]aabb b a_U[__`` `zz :GTZ [ [ [(+Q[el&    r6)r.r)r.r7)r.r;rFrGrHrIr@rJrArKrBrLrCrMrDrNrErOr.r'r`rGrFrGrBrLrCrMrDrNrErOr.r))rFrGrlrKrmrprnrqrorrrBrLrCrMrDrNrErOr.rsr`rGrFrGrBrLrCrMrDrNrErOr.r(r`rGrFrGrBrLrCrMrDrNrErOr.r&__name__ __module__ __qualname____doc__rr5r:r>r rr_rjr|rrr6r4r*r*%sn66)))_))))_)///_//3)-$("&;D7 7 7 7 7 7 @)-$("&;D" " " " " " P! /3]a)-$("&;D: : : : : : F)-$("&;D" " " " " " V)-$("&;D" " " " " " " " r6c eZdZdZed/dZed0dZed1dZeed d d e d d2dZ d d d e dd3d Z eeeed d d e d!d4d*Z d d d e dd5d,Z d d d e dd6d.Zd S)7r+r-r.rc*t|jSr0)rr1r2s r4r5zAsyncRuns.output_items%s  ---r6AsyncRunsWithRawResponsec t|Sr9)rr2s r4r:zAsyncRuns.with_raw_response*s(---r6AsyncRunsWithStreamingResponsec t|Sr=)rr2s r4r>z!AsyncRuns.with_streaming_response4s.d333r6Nr?rFrGrHrIr@rJrArKrBrLrCrMrDrNrErOr'c K|std||td|t|||dtjd{Vt ||||td{VS)rQrRrSrTrUNrVrW)r[r\rrr!r]r#r'r^s r4r_zAsyncRuns.create=sN caV]aabb bZZ 17 C C C,#. (  "1 )+Q[el&          r6rVr`r)c K|std||std||td||t||||td{VS)rbrRrcrdrerVrfNrgris r4rjzAsyncRuns.retrievevs0 caV]aabb b a_U[__`` `YY :GTZ [ [ [(+Q[el(          r6rkrlrmrprnrqrorrAAsyncPaginator[RunListResponse, AsyncCursorPage[RunListResponse]]c|std||td|ttt |||| t ||||dtjtSru) r[ryrrr%r#rr rzr{s r4r|zAsyncRuns.listsJ caV]aabb b!! 17 C C C 1(+'%%!&!&!&"(  $1 "%"   r6r(c K|std||std||td||t||||td{VS)r~rRrcrdrerVrfNrris r4rzAsyncRuns.deletes0 caV]aabb b a_U[__`` `\\ :GTZ [ [ [(+Q[el& "         r6r&c K|std||std||td||t||||td{VS)rrRrcrdrerVrfNrris r4rzAsyncRuns.cancels0 caV]aabb b a_U[__`` `ZZ :GTZ [ [ [(+Q[el&           r6)r.r)r.r)r.rrr)rFrGrlrKrmrprnrqrorrrBrLrCrMrDrNrErOr.rrrrrr6r4r+r+"sn66..._...._.444_4/3)-$("&;D7 7 7 7 7 7 @)-$("&;D" " " " " " P! /3]a)-$("&;D: : : : : : F)-$("&;D" " " " " " V)-$("&;D" " " " " " " " r6c.eZdZd dZed dZdS) r7runsr*r.Nonec@||_tj|j|_tj|j|_tj|j|_tj|j|_tj|j|_dSN)_runsrto_raw_response_wrapperr_rjr|rrr3rs r4__init__zRunsWithRawResponse.__init__ s &> K   )@ M   %< I   '> K   '> K   r6rc4t|jjSr0)rrr5r2s r4r5z RunsWithRawResponse.output_items3s*$**ABBBr6Nrr*r.r)r.rrrrrrr5rr6r4r7r7sN    &CCC_CCCr6r7c.eZdZd dZed dZdS) rrr+r.rc@||_tj|j|_tj|j|_tj|j|_tj|j|_tj|j|_dSr)rrasync_to_raw_response_wrapperr_rjr|rrrs r4rz!AsyncRunsWithRawResponse.__init__:s &D K   )F M   %B I   'D K   'D K   r6rc4t|jjSr0)rrr5r2s r4r5z%AsyncRunsWithRawResponse.output_itemsMs/tz/FGGGr6Nrr+r.r)r.rrrr6r4rr9sN    &HHH_HHHr6rc.eZdZd dZed dZdS) r;rr*r.rc||_t|j|_t|j|_t|j|_t|j|_t|j|_dSr)rrr_rjr|rrrs r4rz"RunsWithStreamingResponse.__init__Ts 2 K   5 M   1 I   3 K   3 K   r6rc4t|jjSr0)rrr5r2s r4r5z&RunsWithStreamingResponse.output_itemsgs0 0GHHHr6Nr)r.rrrr6r4r;r;SsN    &III_IIIr6r;c.eZdZd dZed dZdS) rrr+r.rc||_t|j|_t|j|_t|j|_t|j|_t|j|_dSr)rrr_rjr|rrrs r4rz'AsyncRunsWithStreamingResponse.__init__ns 8 K   ; M   7 I   9 K   9 K   r6rc4t|jjSr0)rrr5r2s r4r5z+AsyncRunsWithStreamingResponse.output_itemss5TZ5LMMMr6Nr)r.rrrr6r4rrmsN    &NNN_NNNr6r)@ __future__rtypingrtyping_extensionsrhttpxr_typesrr r r r r r_utilsrrr_compatr _resourcerr _responserrr5rrrrrr paginationrr types.evalsr r! _base_clientr"r#types.shared_params.metadatar$types.evals.run_list_responser%types.evals.run_cancel_responser&types.evals.run_create_responser'types.evals.run_delete_responser(!types.evals.run_retrieve_responser)__all__r*r+r7rr;rrr6r4rs1#"""""%%%%%% !!!!!!LLLLLLLLLLLLLLLLLLLLLLLLLLLL'''''';;;;;;;;ZZZZZZZZ;:::::::>>>>>>>>AAAAAAAA555555======AAAAAAAAAAAAAAAAAAEEEEEE ; z z z z z ?z z z zz z z z z  z z z zCCCCCCCC4HHHHHHHH4IIIIIIII4NNNNNNNNNNr6