jJSrSSKJr SSKrSSKJs Jr SSKJ r SSK J r SSK J r \ R"\5rSSSjjrSSS jjrSSS jjrSSS jjrSSS jjrSSS jjrSSSjjrSSSjjrSSSjjrg)uVision utility functions for pre-computing very dynamic and data-dependent tensors that can break model graph capturing. All functions are standalone (no model weights) and compute tensors from `grid_thw` + config scalars. They are used by vision encoders and can be precomputed before `torch.compile` / `torch.export` tracing since they use untraceable ops (`repeat_interleave`, `.tolist()`, `nonzero()`, loops). Each `get_*` accepts an optional `kwargs` dict; if it contains the precomputed tensor under the natural key (`"cu_seqlens"`, `"position_ids"`, …), the function pops and returns it instead of computing. Vision encoders write `x = get_vision_x(..., kwargs=kwargs)` and the matching key is removed from the caller's kwargs as a side-effect of the pop. ) annotationsN)PreTrainedConfig)logging)get_max_seqlencUbURSS5=nbU$[RR5(a URO[R nU(aUSS2S4USS2S4-USS2S4-nO/[R "USS2S4USS2S4-USS2S45n[R"URSUS9SSS9$) uGet cumulative sequence lengths from vision grid info, or pop from `kwargs` if precomputed. Args: grid_thw: `(num_images_or_videos, 3)` — temporal, height, width per entry. merge_temporal: when `False` (default), each frame is its own attention segment (`h * w` per frame, `t` segments per entry — the qwen2_vl / glm4v convention). When `True`, the whole clip is a single segment (`t * h * w`), i.e. attention spans all frames jointly (the kimi_k25 convention). kwargs: optional caller kwargs — if it contains `"cu_seqlens"` it is popped and returned. Returns: `cu_seqlens`: `(num_segments + 1,)` int32 cumulative sequence boundaries. N cu_seqlensrr)dimdtyperr)value) poptorchjit is_tracingr int32repeat_interleaveFpadcumsum)grid_thwmerge_temporalkwargsr r seqlenss ڧC:\Users\johnn\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\transformers/vision_utils.pyget_vision_cu_seqlensr*s VZZ d-KKzX#ii2244HNN%++E1a4.8AqD>1HQTNB))(1a4.8AqD>*I8TUWXTX>Z 55AU3V1 EEc0[XUS9n[XAUS9nXE4$)zGet cumulative and maximum sequence lengths for packed vision attention. ``merge_temporal`` is forwarded to [`get_vision_cu_seqlens`] (``True`` for clip-level attention, e.g. kimi_k25).)rr)r)rr)rconfigrrr max_seqlens rget_vision_attention_seqlensr"Ds&'xW]^J 6BJ  !!rc UbURSS5=nbU$URn[U[5(a.[R "U/US9R [U55n/n[UR5UR55GHUuupgp[R"[R"XuS9[R"XS9SS9upXy-XU -U 4n U RU 5RSS5R5n U RU 5RSS5R5n U(al[R"XeS9RXx-5n UR![R""XR%U5U R%U5/SS 95 GM UR![R""X/SS 9R%US55 GMX [R&"US S 9$) uGet position IDs for vision rotary embeddings, or pop from `kwargs` if precomputed. Args: grid_thw: `(num_images_or_videos, 3)` spatial_merge_size: merge block size — either a single `int` (same for all images) or a `(num_images_or_videos,)` tensor (per-image). kwargs: optional caller kwargs — if it contains `"position_ids"` it is popped and returned. include_temporal: when ``True``, prepend a temporal-index column and return `(total_tokens, 3)` — for encoders whose rotary embedding rotates T/H/W axes (minimax_m3_vl). When ``False`` (default), return `(total_tokens, 2)` for the 2-axis (h, w) case (qwen2_5_vl / qwen3_vl / glm4v / paddleocr_vl); the h/w indices are still repeated ``t`` times for video inputs. Returns: `position_ids`: `(total_tokens, 3)` long if ``include_temporal`` else `(total_tokens, 2)`, with the spatial indices laid out block-major over ``m×m`` spatial-merge blocks. N position_idsdeviceij)indexingrr r r)rr& isinstanceintrtensorexpandlenziptolistmeshgridarangereshape transposeflattenrappendstackrepeatcat)rspatial_merge_sizeinclude_temporalrr$r&thw merge_sizehpos_idswpos_ids block_shapetpos_idss rget_vision_position_idsrEQs.vzz.$/OO|\ __F$c**"\\+=*>vNUUVYZbVcdL!$X__%68J8Q8Q8S!T q"^^ LL * LL *   OZP ##K0::1a@HHJ##K0::1a@HHJ ||A5GGNH    Xq7I8??[\K],^df g h    X,@b I P PQRTU V W"U 99\q ))rc8Ub-URSS5nURSS5nUbUbXV4$/nS/nSnX!-U-nUS-n UR5GHhupn [U 5[U 5[U 5pn X-n X-n[R"X-U-5R XU5nXU-- nXU-- nU U-U-nUU-U-n[ R"USUSU4SS5nUR U UUUU5nURSSS SS 5R U UU-X5nUS:gRSS /5R S 5nUR S 5nUUS:gnURUU-5 URS5U -US -nURUR55 XzU -U-- nGMk [R"USS 9n[R"X`R[R S 9n[R""U5nXV4$)uxGet window attention indices, or pop `"window_index"`/`"cu_window_seqlens"` from `kwargs` if both precomputed. Args: grid_thw: `(num_images_or_videos, 3)` spatial_merge_size: merge block size from vision config. window_size: window size from vision config. patch_size: patch size from vision config. kwargs: optional caller kwargs — if it contains both `"window_index"` and `"cu_window_seqlens"` they are popped and returned. Returns: `window_index`: `(total_tokens,)` long — reorder indices for windowed attention. `cu_window_seqlens`: `(num_windows + 1,)` int32 — cumulative window boundaries. N window_indexcu_window_seqlensrr constantirr)r*)r&r )rr1r,rr3r4rrpermutesumr7rextendr:r-r&runique_consecutive)rr; window_size patch_sizerrGrHwindow_index_idvit_merger_window_sizespatial_merge_unitgrid_tgrid_hgrid_w llm_grid_h llm_grid_windexpad_hpad_w num_windows_h num_windows_w index_paddedr index_newcu_seqlens_tmps rget_vision_window_indexrbs`(zz.$7 "JJ':DA  #(9(E2 2L cO(>*L+Q."*//"3!$Vc&k3v;1 1  V0:=>FFv[ef&6L)LL&6L)LL#e+0FF #e+0FF uuUQq%$8*dK #++ M#9=J` $++Aq!Q:BB MM13I  4',,aV4<>!TAX &DAtG uQW~-7<<>H zx<&&1&- Q("a!e,88CaG q1u$01q58HDq1uL++h!mT7'!m1H0DEF =rcUb-URSS5nURSS5nUbUbXx4$Un Un URn USS2S4USS2S4-USS2S4-n [R"USS2S4U 5n [R"USS2S4U 5n[R"[R "U R S5SSS5U 5n[R"U R5U S 9U- X--nX-nUU -nUU -U -nUX--U-nUX-U--nUU -U-nUU -U-n[UXX#U5unn[UXX#U5unnURSnUSS2SS2S4U -USS2SSS24-RSUU-5nUSS2SS2S4USS2SSS24-RSUU-5nUU4$) uPer-patch gather indices/weights that resample a square learned `(num_grid_per_side, num_grid_per_side)` position-embedding table to each image's `(h, w)` grid, or pop `"interp_indices"`/`"interp_weights"` from `kwargs` if both precomputed. Reproduces `F.interpolate(mode=mode, align_corners=align_corners)` as `(total_patches, n_taps)` indices + weights (`n_taps` = 4 for `"bilinear"`, 16 for `"bicubic"` — the Keys kernel with `a=-0.75`), consumed by a fused `F.embedding_bag` or a weighted `embedding` sum. Fully vectorised over packed patches (ragged `(h, w)` handled with `repeat_interleave`, no per-image loop), so it supports dynamic shapes like the other grid_thw precompute helpers. `spatial_merge_size > 1` emits patches in spatial-merge-block order (for encoders that consume merged tokens); `1` keeps raster order. Args: grid_thw: `(num_images_or_videos, 3)` — temporal, height, width per entry. num_grid_per_side: `int(num_position_embeddings ** 0.5)` from the vision config. mode: `"bilinear"` or `"bicubic"`. align_corners: matches the corresponding `F.interpolate` flag. spatial_merge_size: merge block size; `1` keeps raster patch order. kwargs: optional caller kwargs — if it contains both `"interp_indices"` and `"interp_weights"` they are popped and returned. Returns: `indices`: `(total_thw, n_taps)` long — gather indices into the flattened pos_embed table. `weights`: `(total_thw, n_taps)` float — interpolation weights. Ninterp_indicesinterp_weightsrrr r)r r%) rr&rrrrrr3rMrshaper4)rnum_grid_per_sidertrur;rvrrrrsmerger&countsheightswidthsstartswithinblocks_win_colin_row block_col block_rowrowcolh_taps h_weightsw_taps w_weightsnindicesr|s r,get_vision_interpolation_indices_and_weightsrs D$4d;$4d;  %.*D!1 1 D E __F ad^hq!tn ,x1~ =F%%hq!tnf=G  $ $Xad^V rsr # 1)   H %QUFF,0FBMFF:! " "  " "  " % " # .*.**.*.*  .*  .*l :+:+:+:+ :+  :+ ' :+|hp% %+%36%>A%RV%ad%&%VBBB B B  B  B B'BR <<<< <  < ' <6RV##69#CN##D\`bb4CbMXbbr