jrSrSSKrSSKrSSKJr SSKJr SSKJr SSK J r \ "5(aJSSK J r SS K JrJr SS KJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJ r J!r!J"r"J#r#J$r$ SS K%J&r& SS K'J(r(J)r)J*r* S SKJ+r+J,r,J-r-J.r.J/r/J0r0J1r1J2r2 \(a SSK3J4r4J5r5J6r6J7r7 \Rp"\95r:"SS\&SS9r;1Skr<"SS5r="SS\,5r>S\?S\?S\*4Sjr@g)z~ Handler for the /v1/responses endpoint (OpenAI Responses API). Supports streaming (SSE) and non-streaming (JSON) responses. N)AsyncGenerator) TYPE_CHECKING)logging)is_serve_available) HTTPException) JSONResponseStreamingResponse)ResponseResponseCompletedEventResponseContentPartAddedEventResponseContentPartDoneEventResponseCreatedEvent ResponseErrorResponseErrorEventResponseFailedEvent&ResponseFunctionCallArgumentsDoneEventResponseFunctionToolCallResponseInProgressEventResponseOutputItemAddedEventResponseOutputItemDoneEventResponseOutputMessageResponseOutputTextResponseReasoningItemResponseReasoningTextDeltaEventResponseReasoningTextDoneEventResponseTextDeltaEventResponseTextDoneEvent)ResponseCreateParamsStreaming)InputTokensDetailsOutputTokensDetails ResponseUsage)BaseGenerateManager BaseHandlerModality ReasoningTextToolCall _StreamErrorbuild_response_parserparse_assistant_message)GenerationConfigPreTrainedModelPreTrainedTokenizerFastProcessorMixinc*\rSrSr%\\S'\\S'Srg))TransformersResponseCreateParamsStreamingMgeneration_configseedN)__name__ __module__ __qualname____firstlineno__str__annotations__int__static_attributes__r5گC:\Users\johnn\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\transformers/cli/serving/response.pyr1r1Ms  Ir>r1F)total> textuserstorepromptinclude background truncation tool_choice service_tier top_logprobsmax_tool_callsprevious_response_idc *\rSrSrSrS\S\4SjrS\4SjrS\SS 4S jr S\ \4S jr S\ \4S jr S \S\ \4Sjr S\ \4SjrS\ \4SjrS \S\ \4SjrS\ \4SjrS\S\S\S\ \4SjrS\S\ \4SjrS\ \4SjrSrg)_ResponseStreamBuilderbz=Builds SSE events for one streaming Responses API generation. request_idresponse_defaultscX lSU3UlSU3UlSU3UlSUlSUlSUlSUlSUlSUl SUl SUl /Ul g)Nresp_msg_rs_rF) _response_defaultsresp_idmsg_id reasoning_idseq output_indexcontentfull_reasoningreasoning_open message_openreasoning_item message_item tool_calls)selfrPrQs r?__init___ResponseStreamBuilder.__init__esx"3zl+ ZL) !*.  #!<@:>:<r>returnc\[R"U5nU=RS- slU$)Nr#)r% chunk_to_sser[)rdeventsses r?_emit_ResponseStreamBuilder._emitts$&&u- A  r>statusr c 8[S0URDSU0DUD6$)Nrnr5)r rW)rdrnextras r? _response _ResponseStreamBuilder._responseys J$11J&JEJJr>c UR[SURURS/S9S95UR[ SURURS/S9S95/$)Nzresponse.createdqueued)outputtypesequence_numberresponsezresponse.in_progress in_progress)rlrr[rqrrds r?start_response%_ResponseStreamBuilder.start_response|sj JJ$+$(HH!^^HR^@  JJ'/$(HH!^^M"^E   r>cSUlUR[SURUR[ UR S//SS9S95/$)NTresponse.output_item.added reasoningrzidrwsummaryr]rnrwrxr\item)r_rlrr[r\rrZr{s r?start_reasoning&_ResponseStreamBuilder.start_reasoningsV" JJ,5$(HH!%!2!2.,,;TV_l    r>rAc U=RU- slUR[SURURUR SUS95/$)Nzresponse.reasoning_text.deltar)rwitem_idrxr\ content_indexdelta)r^rlrrZr[r\rdrAs r?reasoning_delta&_ResponseStreamBuilder.reasoning_deltasS t# JJ/8 --$(HH!%!2!2"#    r>c [URS/SURS./SS9UlUR [ SURUR URSURS95UR [S UR URURS 95/nS Ul U=RS - slU$) Nrreasoning_textrwrA completedrzresponse.reasoning_text.doner)rwrrxr\rrAresponse.output_item.donerFr#) rrZr^rarlrr[r\rr_)rdpartss r?finish_reasoning'_ResponseStreamBuilder.finish_reasonings3  .8K8KLM   JJ.7 --$(HH!%!2!2"#,,   JJ+4$(HH!%!2!2,,   ($ Q r>c SUlUR[SURUR[ UR SSS/S9S95UR[SUR URURS [S S /S 9S 95/$)NTrmessagerz assistant)rrwrnroler]rzresponse.content_part.addedr output_textrVrwrA annotationsrwrrxr\rpart) r`rlrr[r\rrYr rr{s r? start_message$_ResponseStreamBuilder.start_messages  JJ,5$(HH!%!2!2.;;Y}S^hj   JJ-6 KK$(HH!%!2!2"#+RUWX    r>c U=RU- slUR[SURURUR SU/S95/$)Nzresponse.output_text.deltar)rwrrxr\rrlogprobs)r]rlrrYr[r\rs r? text_delta!_ResponseStreamBuilder.text_deltasQ  JJ&5 KK$(HH!%!2!2"#   r>c [SUR/S9n[URSSSU//S9UlUR [ SURURURSUR/S 95UR [S URURURSUS 95UR [S URURURS 95/nSUl U$)Nrrrrrrrwrnrr]rzresponse.output_text.doner)rwrrxr\rrArzresponse.content_part.donerrrF) rr]rrYrbrlrr[r\rrr`)rdoutput_text_partrs r?finish_message%_ResponseStreamBuilder.finish_messages-=t||acd1{{%&   JJ%4 KK$(HH!%!2!2"#  JJ,5 KK$(HH!%!2!2"#)   JJ+4$(HH!%!2!2**  - >" r>tc_idname argumentsc U=RS- sl[UUSUUSS9nURRU5 UR [ SUR URUS95UR [SUR UURUUS95UR [S UR URUS95/$) Nr# function_callrrcall_idrwrrrnrrz%response.function_call_arguments.done)rwrxrr\rrr) r\rrcappendrlrr[rr)rdrrrrs r? tool_call _ResponseStreamBuilder.tool_call!s Q'    t$ JJ,5$(HH!%!2!2   JJ6@$(HH!!%!2!2'   JJ+4$(HH!%!2!2  '  r>msgcUR[SURUS95UR[SURUR S/[ SUS9S9S95/$) Nerror)rwrxrzresponse.failedfailed server_error)coder)rurrv)rlrr[rrqr)rdrs r?r_ResponseStreamBuilder.errorIse JJ)wZ]^ _ JJ#*$(HH!^^ =n^a3b,   r>c (/nURbURUR5 URUR5 URUR5 UR [ SURURSX!S9S95/$)Nzresponse.completedr)ruusagerv) rarrbextendrcrlr r[rq)rdr all_outputs r?r _ResponseStreamBuilder.completedWs    *   d11 2$++,$//* JJ&-$(HH!^^K ^X   r>) rWr]r^rbr`rYr\rZrar_rXr[rcN)r6r7r8r9__doc__r:dictrerlrqlistr|rrrrrrrrrr=r5r>r?rNrNbsG =c =d =c KKK S  $  c    C  DI  $s)@ tCy 2 s tCy *S *X& s& #& #& $s)& P    c   $s) r>rNc>^\rSrSrSr\r\rS\ S\ S\ \ -4Sjr \S\\ S-S\\ S-4S j5r\S\ S\\ 4S j5r\S \\ S\\ 4S j5rS\ S SSSS\ S\ S\ SSS\S\ 4SjrS\ S SSSS\ S\ S\ SSS\S\ 4SjrSS\ SSS\4U4SjjjrSrU=r$)ResponseHandlerihz+Handler for the ``/v1/responses`` endpoint.bodyrPrgc 0# URU5 URU5up4nURRXES9nURR XF5n[ RSUSU35 URRX7S9nURU5n URX5n [SU 55n 0n U (aSU S'U RUR5 U RURS5=(d 05 URURS 55n UR "U 4S U U(aS OS S S U["R$:H=(a U S .U D6nU(dUR'UR(5nUR+XR,US9nU(aUR/XO5 URSS 5nU(aUR1UUUUUUUUS9$UR3UUUUUUUUS9IS hvN $N7f)aJValidate, load model, dispatch to streaming or non-streaming. Args: body (`dict`): The raw JSON request body (OpenAI Responses API format). request_id (`str`): Unique request identifier (from header or auto-generated). Returns: `StreamingResponse | JSONResponse`: SSE stream or JSON depending on ``body["stream"]``. ) processorz[Request received] Model: z, CB: use_cbc3# UHXn[URS5[5(aURS5O/HnURS5S:Hv M MZ g7f)r]rwvideoN) isinstancegetr).0rcs r? 1ResponseHandler.handle_request..sU ',6swwy7I4,P,Pcggi(VXX EE&MW $X %'sA A" num_frameschat_template_kwargstoolsTNpt)add_generation_promptrreturn_tensors return_dicttokenizeload_audio_from_videostream) gen_manager)_validate_request_resolve_model model_managerget_model_modalitygeneration_stateuse_continuous_batchingloggerwarning get_manager_normalize_input"get_processor_inputs_from_messagesanyupdaterr_normalize_toolsapply_chat_templater& MULTIMODALtodevice_build_generation_configr3init_cb _streaming_non_streaming)rdrrPmodel_idmodelrmodalityrrmessagesprocessor_inputs has_videorrinputs gen_config streamings r?handle_requestResponseHandler.handle_requestns/ t$%)%8%8%>"%%888T&&>>uO3H:VF8LM++777P ((.BB8V '  &( 13  .##D$=$=>##DHH-C$D$JK%%dhhw&78..   "&#)4t"*h.A.A"A"Oi  #  YYu||,F2249P9PY_2`     2HHXt, ??'#  ,,'-    sH HHHrNc U(dU$UVVVs/sH;nSU;a0SUR5VVs0sHup#US:wdM X#_M snnS.OUPM= snnn$s snnfs snnnf)aXNormalize Responses API tool definitions for ``apply_chat_template``. The Responses API uses a flat format: ``{"type": "function", "name": ..., "parameters": ...}`` while ``apply_chat_template`` expects a nested format: ``{"type": "function", "function": {"name": ..., "parameters": ...}}``. Already-nested tools are passed through unchanged. functionrw)rwr)items)rtkvs r?r ResponseHandler._normalize_toolssmL ^hop]pZqwwy-XytqAQWKdady-X Yvw w  -X s A AA AAcrUSnURS5n[U[5(aSUS./nOL[U[5(a-U(aSUS;aSUS./nO [R U5nO [ SSS 9eU(a1U(aUSSS :Xa X#SS 'U$URSS US.5 U$) uNormalize the Responses API ``input`` field into chat messages. The Responses API accepts multiple input formats. This method converts them into a structure close to what ``apply_chat_template`` expects (messages with ``role``, ``content``, ``tool_calls``, ``tool_call_id``). Further processing is done by ``get_processor_inputs_from_messages``. NOTE: if this conversion logic grows too complex, consider having separate ``get_processor_inputs_from_messages`` implementations for chat completions and the Responses API instead of funneling both through the same path. Formats handled: - **String** → single user message. - **Flat content list** (``input_text``, ``input_image``, no ``role``) → user message. - **Multi-turn list** — messages and tool call items (``function_call``, ``function_call_output``) from a previous response, converted via :meth:`_normalize_response_items`. If ``instructions`` is present, it is prepended as a system message. input instructionsrBrr]rrz 'input' must be a string or list status_codedetailsystemr])rrr:rr_normalize_response_itemsrinsert)rinpr rs r?r ResponseHandler._normalize_inputs,7mxx/ c3  !'C89H T " "vSV+%+<=*DDSIC8Z[ [ HQK/8;)5 I&H#NOr>rch/nSnUGH&nURS5nUS:Xa2SRSURS5=(d /55nMMSU;a=USURSS5S.nUbUSS :XaX%S 'SnURU5 MUS :XaaUS US USS.S.nU(a2USSS :Xa&USRS/5RU5 MURS U/S.5 MUS:XaURSUS USS.5 GM[ SSU<3S9e U$)uConvert a list of Responses API items into chat messages. Input items may be a mix of: - Messages (``EasyInputMessageParam`` with ``role``, or ``type: "message"``). - ``reasoning`` — buffered and attached as ``reasoning_content`` to the next assistant message. - ``function_call`` — merged as ``tool_calls`` onto the preceding assistant message. - ``function_call_output`` — converted to ``role: "tool"`` messages. NrwrrVc3*# UH oSv M g7f)rANr5)rrs r?r.s+Y?X!fI?Xsr]rr rreasoning_contentrrrr)rr)rrrc)rrcfunction_call_outputtoolru)r tool_call_idr]r zUnsupported input item type: r )rjoinr setdefaultr)rrpending_reasoningr item_typertcs r?r)ResponseHandler._normalize_response_itemss`(,D(IK'$&GG+Ytxx ?R?XVX?X+Y$Y!~#F|B8OP$0T&\[5P/@+,(,%$o-y/)-fDDU V V 4 CRL++L"=DDRHOO[$MN44 &(,Y#'>$>[\e[hrr-rz(ProcessorMixin | PreTrainedTokenizerFastrrrr,rc t^^ ^ ^^[X2US5n URUUUUTU S9um mUSn [U [5(a [ U 5OU R Sm ST3[ R "5US/URSS5SS .mS [[S 44U U UUU4S jjn [U "5S S9$)zDGenerate a streaming Responses API reply (SSE) using DirectStreamer. input_ids)rPresponse_parserrrSryparallel_tool_callsFauto)r created_atrobjectrr%rHrgNc># [TT S9nSRUR557v SnSnU(d<TR5IShvN /nUR TR 55 M!UR(a"SRUR!557v UR"(d"SRUR3557v SRUR%557v SRUR7[9TT R:5557v gN![ Ra Of=f/nUGH#nUcSn GO[U[5(ab[RSUR35 URURUR55 SRU57v g[U[5(aUR(aURUR!55 UR"(aURUR%55 URUR'TSU3UR(UR*55 US- nGM;[U[,5(aSUR(dURUR/55 URUR1U55 GMUR(aURUR!55 UR"(dURUR355 URUR5U55 GM& U(aSRU57v U(dGMiGN/![<[ R>4a T RA5 ef=f7f) N)rPrQrVrFTz"Exception in response generation: _tool_call_r#)!rNrr|rr get_nowaitasyncio QueueEmptyrr)rrrrr(r_rr`rrrrr'rrrrr compute_usage total_tokens GeneratorExitCancelledErrorcancel) buildertool_call_indexdonebatchrr input_lenqueuerPrQstreamers r? event_stream0ResponseHandler._streaming..event_streamVs, VghG> ggg44677"##(99;./E"!LL)9)9);<#V))'''":":"<==++'''"7"7"9::ggg44677ggg// iI^I^0_`aag/#--(*E %<#'D!%dL99"LL+MdhhZ)XY!LLtxx)@A"$''%.0"%dH55 '55 % W-E-E-G H&33 % W-C-C-E F!LL ' 1 1'1l+o=N$OQUQZQZ\`\j\j!" ,q0O'm<<#*#9#9 % W-D-D-F G!LL)@)@)FG&55 % W-E-E-G H#*#7#7 % W-B-B-D E!LL););D)AB?!&B ggen,W$j"7#9#9: !  sb OAN'D8N'"D:8B?N'7O8N':EN'EB N'OGN'',OOztext/event-stream) media_type) r*generate_streamingrrlenshapetimerrr:r )rdrPrrrrrrrr$r#r:r7r8rQr9s ` @@@@r?rResponseHandler._streaming2s0 &BUV%88    !+ 9 x;' &0D&A&AC NyWYGZ *&))+ #'88,A5#I!  @ N39$=@ @ D!c # URX#XgUS9IShvN upn [X2XSU S9upn /nU (a#UR[SU3S/SU S./S S 95 UR[ S U3S S S [ SU /S9//S95 U (aM[ U 5H>unnUSU3nUR[UUSURURS S95 M@ [U [U 55n[SU3[R"5S UUSU/URSS5SS9 n[UR!SS95$GN(7f)z;Generate a non-streaming Responses API reply (single JSON).)rPNr#)r#cleaned_contentrUrrrrrrTrrrrrr*rrrSryr%Fr&) rr'rnrrur(rrr%rHT) exclude_none)generate_non_streamingr+rrrr enumeraterrrr.r>r r@rr model_dump)rdrPrrrrrrrr]r7 generated_idsrparsed_tool_calls output_itemsir rrrys r?rResponseHandler._non_streamings3>2T2T fZ3U3 - )M9P mk7J\c9 5$5    %ZL)$&6@QRS&    !*&" +WZ\]^    "#452%,k!5##, %,WW"$,,*  6i]);<zl#yy{ $)> F  H//T/BCCw- sED?D(Emodel_generation_configrcn>[TU]XUS9nURS5b[US5UlU$)zXApply Responses API params (``max_output_tokens``) on top of the base generation config.rmax_output_tokens)superrrr<max_new_tokens)rdrrMrr3 __class__s r?r(ResponseHandler._build_generation_configsC!Gr5)F)r6r7r8r9rr1_valid_params_classUNUSED_RESPONSE_FIELDS_unused_fieldsrr:r r r staticmethodrrrrr$rrboolrr= __classcell__)rRs@r?rrhs5C+NNN3NCTWcCcNd  T T 1  d4j46G    *t*T **X0d0T 00hfQfQ!fQ> fQ  fQ  fQfQ'fQ)fQ fQTGDGD!GD> GD  GD  GDGD'GD)GD GDV!T!L^!hl!!r>r input_tokens output_tokensrgc t[UUX-[SSS0S[R;aSS0O0D6[SS9S9$)zBuild a ``ResponseUsage`` object for a Responses API reply. Args: input_tokens (`int`): Number of prompt tokens. output_tokens (`int`): Number of generated tokens. Returns: `ResponseUsage`: Usage statistics with zero-filled detail fields. cached_tokensrcache_write_tokens)reasoning_tokens)rZr[r/input_tokens_detailsoutput_tokens_detailsr5)r"r model_fieldsr!)rZr[s r?r.r.sV !#!1/  +?CUCbCb+b#Q'hj 21E  r>)Arr,r@collections.abcrtypingrutilsrutils.import_utilsrfastapirfastapi.responsesr r openai.types.responsesr r r rrrrrrrrrrrrrrrrr-openai.types.responses.response_create_paramsr%openai.types.responses.response_usager r!r"r$r%r&r'r(r)r*r+ transformersr,r-r.r/ get_loggerr6rr1rUrNrr<r.r5r>r?rns  * 4%A,\ll   gg   H %0MUZ  C C LF!kF!R CMr>