jB :SSKrSSKrSSKrSSKrSSKJrJr SSKJr SSK J r SSK J r SSK JrJrJr \"5(aSSKrSSKJr \"5(aSSKr\ R*"\5rS r\R2"S 5rS \S \4S jrSrS S\S-S\S\S-4SjjrS S\S-S\S\S-4Sjjr S\RB4Sjr"Sr#\R2"S5r$Sr%Sr&S\S-4Sjr'S!S\S\S\4Sjjr(\"SS9\S"S\S-S\S\4Sjj55r)g)#N)contextmanagerredirect_stdout)StringIO_get_torch_distributed_rank)logging)is_torch_availableis_torch_distributed_availablerequires) save_filec[5S:H$)z7Return True if rank=0 or we aren't running distributed.rrڰC:\Users\johnn\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\transformers/model_debugging_utils.py _is_rank_zeror(s & (A --rzobject at 0x[0-9A-Fa-f]+x_strreturnc.[RSU5$)z Replace memory addresses in an object's repr with a stable placeholder so that beautiful JSON diffs won't be ruined by ephemeral addresses. zobject at 0xXXXXXXXX)MEMORY_ADDRESS_REGEXsub)rs r_sanitize_repr_for_diffr0s # #$:E BBrcR[5(aS[UR53$g)z@Return a stable string representation for a DTensor-like object.zDTensor (rank0) -> zDTensor(non-rank0))rrepr _local_tensor)xs r _dtensor_reprr8s#$T!//%:$;<< r debug_pathuse_repr path_to_valuec [R"SS9 U(a [U5nOU(aURS5(dUS- nU(a[R R X5OUn[SUR5R5R50U5 SU3nO[SU<SU<S35e[UR5[UR5US .nUR[R[R [R"1;aUR%['[UR)555['[UR+555['[UR-555['[UR/555S .5 U$) aa Converts Tensors and DTensors to a JSON-serializable dictionary representation. Args: value: Any Python object, often including torch Tensors, lists, dicts, etc. debug_path (`str`, *optional*, defaults to `None`): Directory to dump debug JSON and SafeTensors files. use_repr (bool, *optional*, defaults to `True`): Whether to save a `repr()`-ized version of the tensor as the `value` property in the associated FULL_TENSORS.json file, or to store the full tensors in separate SafeTensors file and store the relative path to that file in the `value` property in the dictionary. path_to_value (`str`, *optional*, defaults to `None`): The file name for the SafeTensors file holding the full tensor value if `use_repr=False`. Returns: A nested Python structure (list, dict, or sanitized string) that is safe to json.dump. T)sci_modez .safetensorsdataz./z use_repr=z and path_to_value=z cannot both be falsy.)shapedtypevalue)meanstdminmax)torchset_printoptions _repr_to_listendswithospathjoinr contiguousdetachcpu ValueErrorrr$r%float16float32bfloat16updaterr'r(r)r*)r&rrr value_outfilepathouts r_serialize_tensor_like_ior=?sO$ D)!%( %%n55 ^ +M>H277<< :m65++-446::<=xH( IH;&:M+;;QRSSekk"ekk" C  {{u}}emmU^^DD /UZZ\0BC.tEIIK/@A.tEIIK/@A.tEIIK/@A    Jrc [U[[45(a-[U5VVs/sHupE[ XQX#SU3S9PM snn$[U[ 5(a3UR 5VVs0sHupeU[ XQX#SU3S9_M snn$[US5(a[URXUS9$[U[R5(a [XX#S9$[[U55$s snnfs snnf)aR Recursively build a JSON-serializable Python structure from `value`. Tensors and DTensors become either sanitized repr strings, or are saved to disk as SafeTensors files and their relative paths are recorded in the returned Python structure. Lists/tuples/dicts are recursed into. All memory addresses are replaced with a stable placeholder. Args: value: Any Python object, often including torch Tensors, lists, dicts, etc. debug_path (`str`, *optional*, defaults to `None`): Directory to dump debug JSON and SafeTensors files. use_repr (bool, *optional*, defaults to `True`): Whether to save a `repr()`-ized version of the tensors as the `value` property in the associated FULL_TENSORS.json file, or to store full tensors in separate SafeTensors files and store the relative path to that file in the `value` property. path_to_value (`str`, *optional*, defaults to `None`): The file name for the SafeTensors file holding the full tensor value if `use_repr=False`. Returns: A nested Python structure (list, dict, or sanitized string) that is safe to json.dump. _rrr r) isinstancelisttuple enumerate _serialize_iodictitemshasattrr=rr+Tensorrr)r&rrr ivks rrErEps (%$''"%( ( !XWffghigjUk l(  %  % }QZiijkljmXno o%  uo&&(   JYf  %&&(PXvv "4; //'  s C41C:r&c&[R"SSS9 [5n[U5 [ U5 UR 5nSSS5 SSS5 [ W5R5$!,(df  N/=f!,(df  N8=f)z Converts a tensor into a sanitized multi-line string representation. Args: value (`torch.Tensor`): The tensor to represent. Returns: `list[str]`: List of string lines representing the tensor. Tx)r" linewidthN)r+r,rrprintgetvaluer splitlines)r&bufraws rr-r-s_ DC8 sOC0 e lln1 #3 ' 2 2 4410s" BA1B1 A? ;B BcURS5(a*URSS5 USHn[U5 M gg)Nchildrenoutputs)getpopprune_outputs_if_children)nodechilds rrZrZs= xx  D!*%E %e ,&rz (.*)\.(\d+)$c^[RURSS55nU(aURS5(dgURS5m[ U4SjUS55$)z Checks whether a node represents a layer block with submodules. Args: node (`dict`): A node from the call tree. Returns: `bool`: Whether the node is a layer block. module_pathrVFc3V># UHnSTS3URSS5;v M g7f).r^r_NrX).0r\numbers r !is_layer_block..s+[JZ6(!} - <E ,, [[^F [$zJZ[ [[rcURS5(dg[US5VVs/sHup[U5(dMX4PM nnn[U5S:aFUSSVVs/sHupUPM nnn[US5VVs/sHupX;dM UPM snnUS'USHn[ U5 M gs snnfs snnfs snnf)z Recursively removes intermediate layers from the tree to improve readability. Keeps at least the first and last layers if many consecutive layers are present. Args: node (`dict`): The root or subnode to prune recursively. rVNr`r)rXrDrllenprune_intermediate_layers)r[rJr\ layer_blocksr? to_removes rrprps 88J  /8j9I/Jd/J81n]bNcJQJ/JLd <1#/"#56#541Q#5 62;D^U4SjmT"URS055 T"URS055 URS/5H nT"U5 M g)Nc>[U[5(a2URSS5 UR5H nT"U5 M g[U[5(aUH nT"U5 M gg)Nr&)rArFrYvaluesrB)valrKitemcleans rr:log_model_debug_trace..strip_values..cleansX#t$$&A!H&C&&D$K 'rinputsrWrVrc)r[r\r strip_valuess @rr+log_model_debug_trace..strip_valuessM  dhhx$% dhhy"%&XXj"-E  .r)r/makedirsr0r1_debugger_module_dump_name Exceptionr5loggerinforZ _call_treeopenjsondumploadsdumps) rmodelbasee full_path summary_pathf tree_copyrs @rlog_model_debug_tracers6 X KK T 277<< ,L,L},\]D//-? KK)$u56++I/)Le../ i  %""Aa0   4::e&6&678I lC A )q) ! E XA*QOPVW W X  . ! s0AD5!EE(5 E?EE E%( E6do_prune_layersc^^^^^ ^ TRRm T SS/S.Tl/TlT TlT(a[ R "TSS9 UUU4SjnTR5HupgUS:XaM U"UT SU35 M TRm [R"T 5U UUUU U4S j5nUTl g![an[STS35UeSnAff=f) a Attaches a debugging wrapper to every module in the model. This records structured inputs and outputs during the forward pass into a call tree. Args: model (`PreTrainedModel`, `nn.Module`): Model to wrap. debug_path (`str`): Optional directory to dump debug JSON files. do_prune_layers (`bool`, *optional*, defaults to `True`): Whether to prune intermediate layers. use_repr (bool, *optional*, defaults to `True`): Whether to save a `repr()`-ized version of the tensors as the `value` property in the associated FULL_TENSORS.json file, or to store full tensors in separate SafeTensors files and store the relative path to that file in the `value` property. Nr^rrWrVTrtrwrbcz>^^^TRm[R"T5UUUUUU4Sj5nUTlg)Nc>[5(a[XS.nUVs0sHn[X#5S:dMX2U_M nnT[UTT TS3S9S/S.nT RR U5 [ R "5 T "U0UD6nSSS5 [5(a[ST R555S:aSWS'O[WTT TS3S9WS'T RR5nUS (dURS 5 T R(a!T RS S R U5 W$s snf!,(df  N=f) Nargskwargsr_inputsr@rc3&# UHnSv M g7f)rNr)rdr?s rrfX_attach_debugger_logic..wrap_forward..wrapped_forward..@s:"9Qq"9srW_outputsrVrn) rrorE_debugger_model_call_stackappendr+no_gradsumnamed_childrenrY) inpskws dict_inputsrLr[r<finishedrrrmodule orig_forwardrs rwrapped_forwardE_attach_debugger_logic..wrap_forward..wrapped_forward+sS'+; :Ea+Q[^I\_`I`0qa.0+ a#,+##-!))2 7&;  $ " 0077="D0C0!:&"7"7"9::Q>&*DO&3#-!))2 8&< 'DO!;;??A +LL,3344R8DKKHUJEb!sE E E  E)forward functoolswraps)rrrrrrrs`` @r wrap_forward,_attach_debugger_logic..wrap_forward(s6~~  &% %  '% N)rr_c>[5(a4TS3[XS.TT TS3S9S/S.nT RRU5 T "U0UD6n[5(aT R(a[UTT TS3S9WS'T RR 5nUST R S'UST R S'US T R S '[ T R R55Vs/sH5nT R U(aMT R R US5PM7 nT(a[T R 5 [TT S 9 U$s snf) Nz (top-level)rrr@rrrWrrV)rr) rrErrrYrrBkeysrpr) rrtop_noder<rrL class_namerrrreal_top_forwardrs rtop_wrapped_forward3_attach_debugger_logic..top_wrapped_forward^sa ??",\:'!1)%%/L"8    H  , , 3 3H =,, ??u??"/%!!+ H5 #HY  77;;=H)1();E  X &*29*=E  Y '+3J+?E  Z (489I9I9N9N9P4Q m4QqY^YiYijkYl *U   ! !!T *4Q m)%*:*:; !Zu E  ns 4E E) __class____name__rrrr/rrr5 named_modulesrrr) rrrrrrname submodulerrrs ```` @@r_attach_debugger_logicrs&))J(2Td`bcE')E$'1E$ X KK T 2+)\!..0 2: Y:,av 671 }}__%&##'#J(EM XA*QOPVW W XsC C" CC")r+)backendsc#8# UR5VVs0sHupEXUR_M nnnURX`'[XX#5 Uv UR5H upxXlM gs snnf!UR5H upxXlM f=f7f)a # Model addition debugger - context manager for model adders This context manager is a power user tool intended for model adders. It tracks all forward calls within a model forward and logs a slice of each input and output on a nested JSON file. If `use_repr=True` (the default), the JSON file will record a `repr()`-ized version of the tensors as a list of strings. If `use_repr=False`, the full tensors will be stored in separate SafeTensors files and the JSON file will provide a relative path to that file. To note, this context manager enforces `torch.no_grad()`. ## Usage add the context manager to a model to debug ```python import torch from PIL import Image from transformers import LlavaProcessor, LlavaForConditionalGeneration, model_addition_debugger_context torch.random.manual_seed(673) # load pretrained model and processor model_id = "llava-hf/llava-1.5-7b-hf" processor = LlavaProcessor.from_pretrained(model_id) model = LlavaForConditionalGeneration.from_pretrained(model_id) # create random image input random_image = Image.fromarray(torch.randint(0, 256, (224, 224, 3), dtype=torch.uint8).numpy()) # prompt prompt = "Describe this image." # process inputs inputs = processor(text=prompt, images=random_image, return_tensors="pt") # call forward method (not .generate!) with model_addition_debugger_context(model, debug_path="Your_debug_path", do_prune_layers=False): output = model.forward(**inputs) ``` N)rrrrG) rrrrr?m orig_forwardsmodule_instanceforward_methods rmodel_addition_debugger_contextrsf/4.A.A.CD.CdaQ \.CMD ==M5oH5 /rs  6:\\+&'''   H %. zz"=>C3C3C ^b.Tz.48.PSVZPZ.b(0S4Z(0$(0^adh^h(0V55"-**_-\")((*cDj(*Z  |(|(|( |(~ :"  85d 8585 8585r