jMSSKJr SSKrSSKJr SSKJrJr SSKJ r SSK J r SS K J r Jr SS KJrJrJr SS jrSS jrSSS .SSjjr"SS5rg)) annotationsN)Callable) isfunctionismethod)Any)get_json_schema)STREAMABLE_PARSERS process_field)ResponseTemplateResponseTemplateFieldload_response_templatec[U[5(dgURS5n[U[5(aU/O/n[U[5(aUR SU55 URS5=(d /HnUR [ U55 M URS5(aSU;aURS5 [U5$)Ntypec3T# UHn[U[5(dMUv M g7fN) isinstancestr).0ts ڽC:\Users\johnn\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\transformers/utils/chat_parsing/response_parser.py _schema_types..!s?1Jq#,>QQs( (anyOfnullablenull) rdictgetrlistextend _schema_typesappendtuple)schemadeclaredtypeschoices rr#r#s fd # #zz&!H$Xs33XJE(D!! ???**W%++ ]6*+, zz*&"5 V <cxUGH nUS:Xa [U5s $US:XaX[U5nX3:wdU[S5[S54;aMJUR5(aSU;a [U5s $Us $US:XaEUR5R 5S;a#UR5R 5S;s $US :XaUR5S ;a gUS ;a=[ R "U5n[XBS :Xa[O[5(aUs $GM GM# U$![a GM5f=f) Nintegernumberinfz-inf.boolean)true1false0)r1r2r)rNone)objectarrayr6) intfloat is_integerstriplowerjsonloadsrrr! ValueError)rawr( type_namer-decodeds r_coercerC)s  I%3xH$s#v%,f 1N'N&,&7&7&9&9cns6{XRXXI%#))+*;*;*=A\*\yy{((*m;;F"syy{6F'F//**S/gH/Dt$OO"NP0( J   s5D*0D* %D*5D*9AD*D* =D** D98D9prefixtoolscv[U5n[XUS9nURU5 UR5upVU$)aHThe main function for response parsing when you don't want streaming. Takes generated output and the prompt prefix and parses them without streaming any events, then returns the parsed message. Pass `tools` (in the same format as `apply_chat_template` accepts) to cast tool-call arguments using the calling tool's JSON schema. rD)rResponseParserfeedfinalize)textresponse_templaterErFstreammessage_s rparse_responserPAs<//@A -E JF KK"JG Nr*c\rSrSrSrSSS.SSjjjrSSjrSSjrSSjrSS jr SS jr SS jr SS jr SS jr SSjrSSjrSSjrS SjrSrg)!rHUaThis class implements a streaming parser with a `response_template`. If you don't need streaming and just want to parse a complete message, use the `parse_response` function above. Streaming parsing emits events indicating when regions (message fields) are opened and closed, with the model writing to the region that is currently open. Usage: parser = ResponseParser(response_template, prefix=chat_prompt) for event in parser.initial_events: handle(event) for chunk in model_text_stream: for event in parser.feed(chunk): handle(event) message, final_events = parser.finalize() for event in final_events: handle(event) Pass `tools=` (in the same format as `apply_chat_template` accepts) to cast tool-call arguments using the calling tool's JSON schema as each region closes. Events can be either "region_open", "region_chunk", or "region_close". ResponseParser requires the chat `prefix` (i.e. the chat history, the prefill before the current generation). This is because chat templates or assistant prefills can sometimes write part of the message, and if we only see the model output, and not the template, then we can't reliably parse the message in those cases. Any events produced while consuming the prefix are exposed as `initial_events`, so renderers can show prefill regions before the model writes anything; closed prefill regions also land in the output dict. N)rFc[U5UlUc [S5e0UlU=(d /Hn[ U5(d[ U5(a [ U5n[U[5(aURSU5OSn[U[5(dMn[URS5[5(dMURS5n[U[5(aURS5OSn[U[5(aUO0URUS'M SUl SUl [URR5UlURRUlUR Ul0UlSUlSUlSUl/UlU(aUR/U5 gg) Nah`ResponseParser`/`parse_response` requires `prefix` (the chat prompt sent to the model before generation), because chat templates often pre-write part of the assistant message (e.g. an opening `` tag) that the parser must see to parse the output correctly. If the generation already contains the complete message, pass `prefix=""` to opt out explicitly.functionname parameters propertiesrF)r_specr? _tool_paramsrrr rrr r_buffer_posdefaults_outputimplicit_implicit_name_current _captures_body_opened _finalizedinitial_events_consume_prefix)selfrLrErFtoolfnrVrWs r__init__ResponseParser.__init__rsp,,=> >a .0KRKD$8D>>&t,/9$/E/E*d+4B"d## 266&>3(G(GVVL1 =G TX=Y=YZ^^L9_c >HUY>Z>Z `b!!"V*-   '+DJJ,?,?'@ *.***=*= %)$7$7 )+ " %*,    ( r*cURRU5nU(dgX lURURSS9 g)aLoads the prefix (the chat prefill sent to the model), right-truncates it to the start of the assistant message (as determined by start_anchor) and then runs the remainder through the parser. Events produced while processing the prefix are stashed on `initial_events` so callers can replay them into a renderer before feeding model output. Think of this as the "get the parser up to speed on the story so far" method. NFeos)rYtruncate_past_last_anchorr[_processrf)rhrE truncateds rrgResponseParser._consume_prefixs9JJ88@    d))u 5r*cUR(a [S5eU(aU=RU- sl/nURUSS9 U$)zFeeds more text/tokens from the model output into the tokenizer, and returns any events that result (regions entered or left). This is the method you want to call after each generation step. ResponseParser already finalizedFrn)re RuntimeErrorr[rq)rhrKeventss rrIResponseParser.feedsB ??AB B  LLD L f% ( r*cHSSjnUR(a [S5e/nURUSS9 URRR 5VVs/sH+up4UR (aMX0R;dM)UPM- nnnU(a[SU35eURRnURR 5VVs0sHupxXv;dU"U5(aMXx_M snnUlSUlURU4$s snnfs snnf)zClose the stream and return the final message dict together with any finalization events. This is necessary because some regions may only end at the end of the sequence, so you won't see the event telling you they're ready until the sequence is finalized.cjUSL=(d) [U[[[45=(a U(+$r)rr!rr)vs r _is_empty*ResponseParser.finalize.._is_emptys&9LAdC/@!A!K!e Lr*ruTrnz>Required response_template fields missing from parsed output: )r{rreturnbool) rervrqrYfieldsitemsoptionalr^r?r]) rhr|rwnfmissingr]kr{s rrJResponseParser.finalizes  M ??AB B f$ '!%!2!2!8!8!:g!:!**1QRZfZfQf1!:g ]^e]fgh h::&&)-););)=c)=V_`aVb)=c ||V## hdsD6 DDD4DcUR5nURX25upEUbUupgnUR5UR:a6UR XR URUR55 UR 5UlUS:Xa$URU5 URXU5 OGURn URU5 U (d#UR5UR 5:XagGM U(awUR[UR 5:aBUR XR URS5 [UR 5UlURU5 gXPR:a.UR XR URU5 XPlg)Nopen) _watchlist_scanstartr\ _accumulater[end_close_current_open_explicitrdlen) rhrwrowatchbest hold_startkindfieldm had_contents rrqResponseParser._processsXOO%E#zz%5 D!%Q779tyy($$V\\$))aggi-PQEEG 6>''/''q9#',,K''/'1779+?99s4<<00$$V\\$))+-FG #DLL 1DI##F+ II%  dii*)MN& r*cURbQURUR:wa7URRURnURbSU4/$/$/nURRR 5H%nUR cMURSU45 M' URbCURRURnURbURSU45 U$)zPatterns we care about right now: the close of the currently-open explicit region, or -- if we're in the implicit/null region -- every explicit open plus the implicit's own close (if any).closer)rar`rYrclose_revaluesopen_rer$)rhrrimpls rrResponseParser._watchlists == $$:M:M)MJJ%%dmm4E).)CWe$% K K9;ZZ&&--/E}}( fe_-0    *::$$T%8%89D}}( gt_- r*cSnSn[UR5nUGHupgUS:Xa URO URnU(a'UR URUR 5n O%UR URUR SS9n U cM|U(dCU R (dURXgU 5(a[XYR55nMU R5U R5U R5- *US:XaSOSUR4n UbX:dGMXXy4pCGM UbUSR5U:aSnXE4$)aSingle pass over the watched delimiters, using the `regex` module's partial matching to decide -- per delimiter -- whether it can be committed now or must be held. Returns `(best, hold_start)`: * `best` is the earliest-starting delimiter we can safely commit *now* (longest on ties, opens before closes), or `None`. * `hold_start` is the leftmost buffer position occupied by a still-pending match: a partial (incomplete) delimiter, or a complete one ending at the buffer edge that more input could still grow. Bytes before it are safe to emit; bytes from it onward must be held. It stays `len(self._buffer)` when nothing is pending, letting the caller flush the whole buffer. A complete match is committable only if it starts strictly before `hold_start` -- otherwise an earlier (or co-located) pending delimiter could turn out to be the real one. At EOS nothing can grow, so partial matching is skipped and every complete match is committable. (The `regex` module always reports the empty string as a live prefix, so a partial search with no real match returns a zero-width match at the buffer end; that lands in the pending branch with `start == len(self._buffer)`, a no-op for `hold_start`.) NrT)partialrr r) rr[rrsearchr\r _can_growminrrrU) rhrrobest_keyrrrrpatternrkeys rrResponseParser._scans2"&>B&  KD'+v~emm5>>GNN4<<;NN4<<DNIyAIIQ)G)G WWY7 779!'') 3446>aqRWR\R\]C3>!$U&6$!$  Q : =Dr*c"UR5[UR5:wagUR5UR5:XagUS:XaURUR 4OUR UR4upEUSL=(d U$)aWhether a *complete* match ending at the current buffer edge could still change as more input arrives -- in which case we defer rather than commit. A match ending before the edge has already seen its terminating byte and is final. At the edge: zero-width matches (`$` / `\Z`) are only real at true EOS; a fully-present literal that no other literal in its set extends cannot grow (the fast path that keeps literal delimiters zero-latency); anything else (regex delimiters, prefix-overlapping literal lists) might.FTrN)rrr[r open_literalsopen_literal_can_extendclose_literalsclose_literal_can_extend)rhrrrliterals can_extends rrResponseParser._can_grow4s 557c$,,' ' 779 v~ %"?"? @&&(F(FG  4-:-r*cnU(a URcgURRURnUR(d%UR SURS.5 SUlU=R U- slUR [;nUR SURX$S.5 g)avRoute `text` into the currently active region. When the current region is the null sink (no implicit declared, no explicit open), we silently discard. Every routed chunk emits a `region_chunk` event so consumers can render live; `dirty=True` flags chunks from structured parsers (json, xml-inline, kv-lines) whose raw bytes will only be parsed into the final value on close.N region_openrrT region_chunk)rrrKdirty)rarYrrdr$rccontentr )rhrwrKrrs rrResponseParser._accumulateGst}},  !!$--0|| MM=4==I JDL d  %77 ~ tder*cURUlUR5R5VVs0sH upEUcM XE_M snnUlSUlSUlURSURS.5 gs snnf)NrXTrr)rUra groupdictrrbrcrdr$)rhrwrrrr{s rrResponseParser._open_explicitXse  +,;;=+>+>+@R+@41A$!$+@R   }uzzBCSs A<A<cURbUR(dUR5 gURRURn[ UR X R5nUR(aURU5nURb[U[5(d/[SURS[U5R S35eUR"R%UR5nUcUOXBR-U-UR"UR'O_UR&(a6UR"R)UR/5R+U5 OX0R"UR'UR+SURUS.5 UR5 g)zClose the current region and reset to the implicit/null region. Skipped (aside from the reset) when the current region never opened -- avoids vacuous open/close pairs at every explicit boundary.NzField 'z8': 'join' requires each match to parse to a string, got r/ region_close)rrvalue)rard_reset_to_implicitrYrr rcrbrZ_coerce_tool_callsjoinrrr?rUr__name__r^r repeats setdefaultr$)rhrwrrpreviouss rrResponseParser._close_current_sS ==   # # %  !!$--0djj%@   ++E2E :: !eS)) ejj\*;//03||'' 6H3;3C%T^T^I^afIfDLL ' ]] LL # #DMM2 6 = =e D*/LL ' ~ PUVW !r*c*[U[5(a!UVs/sHo RU5PM sn$[U[5(aUR S5OSn[U[5(dU$UR S5UR S5pT[U[ 5(a[U[5(dU$UR R U5=n(aUR5HupxXv;d[Xg5=n (dM [U[ 5(a[X5XW'MD[U[5(dM[UVs/sH%n[U[ 5(a [X)5OUPM' snXW'M U$s snfs snf)NrTrU arguments) rr!rrr rrZrr#rC) rhritemrjrUrrWrargumentr(s rr!ResponseParser._coerce_tool_callsysC eT " ">CDed++D1eD D&0&=&=UYYz "4"d##L&&."&&*=i$$$Jy$,G,GL**..t4 4: 4!*!2 (- :X1X1Xh,,%,X%=IN$//jr%sjrbfjs>S>Sgd&:Y]&]jr%sIN "3 E&ts F ,FcPURUl0UlSUlSUlg)NrXF)r`rarbrcrd)rhs rr!ResponseParser._reset_to_implicits$++   r*) rcr[rbrarer`rdr^r\rYrZrfr)rLdict | ResponseTemplaterE str | NonerFlist[dict | Callable] | None)rErr~r5)rKrr~ list[dict])r~ztuple[dict, list[dict]])rwrrorr~r5)r~'list[tuple[str, ResponseTemplateField]])rrrorr~z9tuple[tuple[str, ResponseTemplateField, Any] | None, int])rrrrrrr~r)rwrrKrr~r5)rwrrrrrr~r5)rwrr~r5)rrr~r)r~r5)r __module__ __qualname____firstlineno____doc__rkrgrIrJrqrrrrrrrr__static_attributes__rr*rrHrHUs>"() /3 ()2()() , ()T 6 $*%N"0 <0 CG0 B0 d.&f"D"4&r*rH)r&rr~tuple[str, ...])r@rr(rr~r) rKrrLrrErrFrr~r) __future__rr=collections.abcrinspectrrtypingrchat_template_utilsr content_parsersr r response_templatesr rrr#rCrPrHrr*rrss# $(1>__ 8*.  .   (   ({{r*