KgjOJ%SrSSKrSSKrSSKrSSKrSrSrSrS\4Sjr S\ S\ 4S jr S \ \ \ S 4-S\4S jrS \ S\S\ S\S\ \ \ S 4S 4S\ \ \ \ S 4-S 4S\ \ S 44Sjr"SS5rSq\S-\S'S\4SjrSCS\S\ S-S\S-S\ \ \ S 4S 4S-S\ \ \ \ S 4-S 4S-S\ \ \ S 4S 4S-4 SjjrSSSSSSS.S\ S\S\S\ S\ S \ S!\ S-S"\ S-S\ 4S#jjrSS$SS%.S&\ S'\ S\S\S!\ S\ S\ 4S(jjrSDSS)SS*.S+\ S\S\S \ S\ S-S\ 4 S,jjjr\SSSSS-.S.\ S/\S0\ S1\ S\S!\ S\S-S\ 4S2jjrS3S4SS5SSS6.S7\ S8\ S\S9\S\S\ S\ S \ S!\ S-S\ 4S:jjr\S;.S<\ S=\ S0\ S\ 4S>jjr\SS?.S<\ S=\ S@\ S0\ S \ S\ 4 SAjjr\S;.S<\ S=\ S0\ 4SBjjrg)Ea  Unified Buffer (UB) Manager for Ascend NPU. This module provides UB capacity detection and tiling strategy computation for running Triton kernels on Ascend NPU. It automatically calculates optimal block sizes based on UB capacity constraints to prevent UB overflow. Niiireturnc[[S5(a[RR5$SSKJn U"5$![a Nf=f![a gf=f)zDetect Ascend NPU availability.npuris_torch_npu_availableF)hasattrtorchr is_available Exceptiontransformers.utilsr ImportErrorrs کC:\Users\johnn\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\fla/utils/ascend_ub_manager.pyis_npu_availablers_ue 99))+ +=%''     s"A A AA A A reasoncR[R"S[S-SU3SS9 [$)NzUsing fallback UB capacity (z bytes): ) stacklevel)warningswarn_FALLBACK_UB_CAPACITY_BITS)rs r_fallback_ub_capacityr.s2 MM &'AQ'F&GyQWPXY &% tiling_dim.c[U[5(aU1$[U[5(a [U5$[5$)z Normalize tiling dimension specification to a set of dimension indices. Args: tiling_dim: Either an int (single dimension) or tuple of ints (multiple dimensions). Returns: Set of dimension indices that can be tiled. ) isinstanceinttupleset)rs r_normalize_tiling_dimsr 6s7*c""| J & &:u rub_capacity_bits safety_margin dtype_sizememory_multipliershapes tiling_dimsc J^U(aU(dg/n[XE5Humn[U5nU(d[SUS35e[U4SjU55(a[SUSTS[ T5S35eSn [ T5H$upX;dM U S ::aSn OU [ U 5-n M& U S ::aSn [X-5n [XU -U-S --5n [S U 5n [R"U S -5S -nURU5 M [U5$) a Default tiling strategy: calculate maximum safe block size based on UB capacity. This is a unified strategy function that works for all kernels by abstracting the memory calculation as: memory_multiplier * BLOCK_SIZE * unit_param * dtype_size * 8 bits Args: ub_capacity_bits: UB capacity in bits safety_margin: Safety margin as a float (e.g., 0.80 for 80%) dtype_size: Size of data type in bytes (e.g., 2 for float16, 4 for float32) memory_multiplier: Memory multiplier for estimating peak memory usage shapes: Tuple of full shapes. Each shape is a tuple of dimension sizes. - For ROPE: ((n_q_head, hd), (n_kv_head, hd)) - For GEGLU: ((n_cols,),) tiling_dims: Tuple specifying which dimensions can be tiled for each shape. Each element can be: - int: single dimension index (e.g., 0 for first dimension) - tuple of ints: multiple dimensions that can be tiled together - For ROPE: (0, 0) means first dimension of each shape can be tiled - For GEGLU: (0,) means first dimension of the shape can be tiled Length must match len(shapes). Returns: Tuple of maximum safe block sizes, one for each shape. Each element is a power of 2. Note: For each shape, fixed dimensions (non-tiling) are multiplied together to get unit_param. The final block size is computed in compute_default_tiling_strategy by taking min(desired_block_size, max_safe_block_size) where desired_block_size = triton.next_power_of_2(original_dim). Invalid tiling_dim: 9. tiling_dim must be an int or a non-empty tuple of ints.c3V># UHoS:=(d U[T5:v M g7frNlen).0dim_idxshapes r $_default_strategy..~s$R>{3gU33>&) for shape -. All dimension indices must be in range [0, ).g?rr) zipr ValueErroranyr. enumeratefloatrmaxtritonnext_power_of_2appendr)r!r"r#r$r%r&max_safe_sizesrtiling_dim_set unit_paramr0dim_sizeSAFE_UB_CAPACITY_BITSmax_block_sizesafe_block_sizer1s @r_default_strategyrJHsWN N 5z/ ;&zl2kl  R>R R R&zl+eWE>>A%j\M   !*5!1 G,q=!$JeHo- "2 ?J!$$4$D E2:7UXb7bef7fghQ/!00!1CDIo.S6V   rc\rSrSrSrS S\S-4Sjjr\S\4Sj5r\S\4Sj5r \S\ 4S j5r S\ 4S jr S\4S jr S rg) UBManagerz Unified Buffer Manager for Ascend NPU. Provides UB capacity detection and management for Ascend NPU devices. The UB capacity is used by tiling strategy functions to calculate optimal block sizes. Nr!cjUR5UlU=(d UR5Ulg)zs Initialize UB Manager. Args: ub_capacity_bits: UB capacity in bits. If None, will be detected automatically. N)_detect_npu_model _npu_model_detect_ub_capacity_ub_capacity_bits)selfr!s r__init__UBManager.__init__s*002!1!OT5M5M5OrrcUR$)zGet UB capacity in bits.rRrSs rr!UBManager.ub_capacity_bitss%%%rc URS-$)zGet UB capacity in bytes.rrWrXs rub_capacity_bytesUBManager.ub_capacity_bytess%%**rcUR$)zGet detected NPU model name.)rPrXs r npu_modelUBManager.npu_modelsrc[5(dg[RRS5nUR$![ a gf=f)z(Detect NPU model from device properties.unknownrdefault)rr rget_device_propertiesnamer )rS dev_propss rrOUBManager._detect_npu_modelsF!!  77:I>> !   s*= A  A c[R"S5nUb[U5nUS:aU$[ 5(afSSKJnJn [RnURUR55nU"U5 U"S5nUbUS::a[SU35eUS-nU$[S 5$![a Nf=f![a [S5s$[an [S U S 35sSn A $Sn A ff=f) z Detect UB capacity from environment variable or get_soc_spec. Returns: UB capacity in bits. Falls back to a conservative default with a warning when detection fails. ASCEND_UB_CAPACITY_BITSNr) get_soc_specset_current_compile_soc_infoUB_SIZEz#Invalid UB_SIZE from get_soc_spec: rzfCannot import tbe.common.platform.get_soc_spec. Source CANN set_env.sh or set ASCEND_UB_CAPACITY_BITS.z0Failed to detect UB capacity from get_soc_spec: z*. Set ASCEND_UB_CAPACITY_BITS to override.z>NPU is not available. Set ASCEND_UB_CAPACITY_BITS to override.)osgetenvrr;rtbe.common.platformrirjr rget_device_namecurrent_devicer rr ) rS env_capacity capacity_bitsrirjdevicesoc_info ub_size_bytesr!es rrQUBManager._detect_ub_capacitys-yy!:;  #  #L 1  1$((%    Z!11&2G2G2IJ,X6!-Y 7  (MQ,>$'J=/%Z[[$11#4 ''% L  E  . ,M ,FqcJ?? s5B/A$B?/ B<;B<?C8 C8C3-C83C8)rPrR)N)__name__ __module__ __qualname____firstlineno____doc__rrTpropertyr!r[strr^rOrQ__static_attributes__r(rrrLrLsPtP&#&&+3++3 3 3 S3 rrL _ub_managerc0[c [5q[$)zGet global UB manager instance.)rrLr(rrget_ub_managerr sk rc @^[5nUbU(a UbU(dg[U5[U5:wagUbUS::aSnUbUS::aSn[URUUUUU5nU(a[U5[U5:wag/n[ X4U5Hupn [ U5m[ U 5n U (d[SU S35e[U4SjU 55(a[SU SUS [T5S 35eU H:n TU n [R"U 5n[X5n[S U5nUTU 'M< [T5H&un nX;dM [R"U5TU 'M( UR[T55 M [U5$) a` Compute tiling strategy using the default strategy function. This function directly calls the default strategy and computes the final tiling result. All kernels use the same unified strategy function, so there's no need for kernel_name-based lookup. Args: safety_margin: Safety margin as a float (e.g., 0.80 for 80%). Default is 0.80. dtype_size: Size of data type in bytes (e.g., 2 for float16, 4 for float32). Must be provided. If None or <= 0, defaults to 4 (float32). memory_multiplier: Memory multiplier for estimating peak memory usage. - For GEGLU: typically 10.0 for backward, 4.0 for forward - For ROPE: typically 3.0 If None, defaults to 10.0 (conservative estimate). shapes: Tuple of full shapes. Each shape is a tuple of dimension sizes. - For ROPE: ((n_q_head, hd), (n_kv_head, hd)) - For GEGLU: ((n_cols,),) Can pass original shapes (will handle padding internally) or padded shapes. tiling_dims: Tuple specifying which dimensions can be tiled for each shape. Each element can be: - int: single dimension index (e.g., 0 for first dimension) - tuple of ints: multiple dimensions that can be tiled together - For ROPE: (0, 0) means first dimension of each shape can be tiled - For GEGLU: (0,) means first dimension of the shape can be tiled Length must match len(shapes). Cannot be empty. Returns: Tuple of tiled shapes with same structure as input shapes. Tiling dimensions are replaced with computed block sizes (power of 2), while non-tiling dimensions are padded to next power of 2. - For ROPE: ((block_size_q, pad_hd), (block_size_kv, pad_hd)) - For GEGLU: ((block_size,),) Returns None if shapes or tiling_dims is None or empty. Examples: >>> # ROPE forward >>> strategy = compute_default_tiling_strategy( ... safety_margin=0.90, ... dtype_size=4, ... memory_multiplier=3.0, ... shapes=((32, 128), (32, 128)), ... tiling_dims=(0, 0) ... ) >>> # Returns: ((block_size_q, 128), (block_size_kv, 128)) >>> # GEGLU forward >>> strategy = compute_default_tiling_strategy( ... safety_margin=0.80, ... dtype_size=2, ... memory_multiplier=7.0, ... shapes=((4096,),), ... tiling_dims=(0,) ... ) >>> # Returns: ((block_size,),) Nrg$@r)r*c3V># UHoS:=(d U[T5:v M g7fr,r-)r/r0 result_shapes rr22compute_default_tiling_strategy..zs%Y.w{:g\):::.r4r5r6r7r8)rr.rJr!r:listr r;r<r@rAminr?r=rBr)r"r#r$r%r& ub_manager max_supportedresultr1rmax_saferDr0 original_dimdesired final_valrFrs @rcompute_default_tiling_strategyrs| !J ~V{':+ 6{c+&&Z1_  $5$: &## M C .#f+=F'*6 'N#8E{ 0 ;&zl2kl  Y.Y Y Y&zl+eWE>>A,>O=PPRT &G'0L,,\:GG.IAy)I$-L ! &"+(>x(H W%"9  eL)*?(OB =rg?rr8)r"r#fallback min_block max_blockrrFrrrrcUc[R"U5n[UUUU44SS9nU(a[XxSS5O [Xt5n [ XY5n Ub [X5n U $)z:Compute UB-safe block size for a single tilable dimension.rr"r#r$r%r&rr@rArrr?) rFr$r"r#rrrr tile_shapesblocks rcompute_ub_block_sizerss((21#+ ~ K0;CQ* +G@VE  !EE% Lri )r"rr vocab_sizenum_rowsc[UUUUS9n[S[[US5-5n[R"X-S- U-5n[ [Xh5U5$)z:UB-safe vocab tile size respecting Ascend grid dim0 limit.r"rr8)rr?ASCEND_MAX_GRID_DIMr@rAr) rrr$r"rrub_block max_splitsgrid_mins rcompute_vocab_block_sizersd%# H Q+s8Q/??@J%%z'>'Bz&QRH s8& 22ri)r"rr n_elementscUc[n[UUUUS9n[U[R"U[ -S- [ -55n[ XV5$)z)H4 N5F**5115 LrTg333333?) tiling_rowr"r#rrrrow_dim fixed_dimrc<U(aX44n Sn [R"U5n OX44n Sn [R"U5n [UUUU U S9n U (a#U(aU SSOU SSn [X5n O [X5n [ X}5n Ub [X5n U $)z;UB-safe tile along one axis of a 2D [row, col] kernel tile.r)r8rrr8r)rrr$rr"r#rrrr%r&rrrs rcompute_row_tile_block_sizers&( ((1&( ((31#+ K%/ Aq![^A5FG#G&  !EE% Lrr axis_sizeother_grid_productc4[SU[US5-5$)zLMax launch chunks along one grid axis while keeping the product <= max_grid.r8)r?)rrrs rmax_grid_axis_chunksr!s q(c"4a88 99r)rr ub_safe_blockc,[U[X 55$)zCPick a UB-safe tile size; host-side chunking handles grid overflow.)r?r)rrrrrs rcompute_grid_limited_tile_sizer+s y#m7 88rc#j# [XUS9n[SX5HnU[X0U- 54v M g7f)z=Yield ``(offset, chunk_len)`` for host-side grid-axis tiling.rrN)rranger)rrr max_chunksoffsets riter_axis_launch_chunksr7s9&ihWJ91c*&&89992s13)g?NNNN)g@)r|rlrr r@rrrboolrr~rrrrr r>rJrLr__annotations__rrrrrrrrrrr(rrrsZ  && $ &#&#&sU38_'<$X!X!X!X! X! %S/3& ' X! sU38_,c12 X! 38_X!vb b L!% Y $  !&*15<@ ||d |t|| %S/3& '$ . | sU38_,c12T9 |  5c?C  4' |F      Tz4Z D3333  3  33 3. ## ### #  # Dj # #8(&*     t| B % %%%  %  %%%%Tz% %X( ::: :  :( 9 9 9 9  9  9  9 ( : : : :r