oMj, SSKJr SSKrSSKJr SSKJr SSKJr SSK J r SS K J r J r JrJr \"\5rS rS r\"S S 55r"SS\ 5r"SS\5r"SS\5rS\R0R2S\SS4SjrS\R0R2S\S\SS4SjrSS\R0R2S\S\SS4Sjjrg)) dataclassN) get_logger) unwrap_module)"_ALL_TRANSFORMER_BLOCK_IDENTIFIERS)TransformerBlockRegistry) BaseState HookRegistry ModelHook StateManagerfbc_leader_block_hookfbc_block_hookc(\rSrSr%SrSr\\S'Srg)FirstBlockCacheConfig a3 Configuration for [First Block Cache](https://github.com/chengzeyi/ParaAttention/blob/7a266123671b55e7e5a2fe9af3121f07a36afc78/README.md#first-block-cache-our-dynamic-caching). Args: threshold (`float`, defaults to `0.05`): The threshold to determine whether or not a forward pass through all layers of the model is required. A higher threshold usually results in a forward pass through a lower number of layers and faster inference, but might lead to poorer generation quality. A lower threshold may not result in significant generation speedup. The threshold is compared against the absmean difference of the residuals between the current and cached outputs from the first transformer block. If the difference is below the threshold, the forward pass is skipped. g? thresholdN) __name__ __module__ __qualname____firstlineno____doc__rfloat__annotations____static_attributes__rگC:\Users\johnn\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\diffusers/hooks/first_block_cache.pyrr s Iurrc2^\rSrSrSU4SjjrSrSrU=r$)FBCSharedBlockState3cZ>[TU]5 SUlSUlSUlSUlgNT)super__init__head_block_outputhead_block_residualtail_block_residualsshould_compute)self __class__s rr%FBCSharedBlockState.__init__4s. JN15 MQ!$(rc SUlSUlgr#)r(r))r*s rresetFBCSharedBlockState.reset<s$(!"r)r&r'r)r()returnN)rrrrr%r.r __classcell__r+s@rr r 3s)##rr c\rSrSrSrS\S\4SjrSrS\ RR4Sjr S r \ RRS \ R S \4S j5rS rg)FBCHeadBlockHookAT state_managerrc*XlX lSUlgN)r6r _metadata)r*r6rs rr%FBCHeadBlockHook.__init__Ds*"rcf[U5n[R"UR5UlU$r8rr getr+r9r*moduleunwrapped_modules rinitialize_hook FBCHeadBlockHook.initialize_hookI+(01556F6P6PQ rr?cURRSX#5nURR"U0UD6n[ U[ 5nU(aXPRR U- nOXT- nURR5nS=pURU5n Xl U (dU(a)URSXPRR -n OURSU-n URRb1U(deURSXPRR-n U(aKS/[U5-n XURR 'XURR'[ U 5n OU n U nU$U(aFS/[U5-n XPRR U S'XPRRU S'OUn XlXxlU$)N hidden_statesrr)r9_get_parameter_from_args_kwargsfn_reforiginal_forward isinstancetuplereturn_hidden_states_indexr6 get_state _should_compute_remaining_blocksr)r("return_encoder_hidden_states_indexlenr&r')r*r?argskwargsoriginal_hidden_statesoutputis_output_tuplehidden_states_residual shared_staterEencoder_hidden_statesr) return_outputr&s r new_forwardFBCHeadBlockHook.new_forwardNs!%!O!OP_ae!n--t>v>$VU3 %+NN,U,U%VYo%o "%+%D ",0,>,>,H,H,J 044 >>?UV&4# 55a86..BkBk;ll!- A A! Dv M ~~@@L&& 55a86..BsBs;tt&!%V 4 KXdnnGGHShdnnOOP %m 4 - "F %)FS[$8!'-nn.W.W'X!!$'-nn._._'`!!$$*!-> */E , rc:URR5 U$r8)r6r.)r*r?s r reset_stateFBCHeadBlockHook.reset_states   " rrUr0c&URR5nURcgURnX- R5R 5nUR5R 5nXE- R 5nX`R :$r#)r6rLr'absmeanitemr)r*rUrVprev_hidden_states_residualabsmeanprev_hidden_states_absmeandiffs rrM1FBCHeadBlockHook._should_compute_remaining_blockss))335  + + 3&2&F&F#)GLLNSSU%@%D%D%F%K%K%M"4::<nn$$r)r9r6rN)rrrr _is_statefulr rr%rAtorchnnModulerYr\compilerdisableTensorboolrMrrrrr4r4AsjLlu  1%((//1f ^^%u||%X\%%rr4ct^\rSrSrS S\S\4U4SjjjrSrS\RR4Sjr Sr U=r $) FBCBlockHookr6is_tailcH>[TU]5 XlX lSUlgr8)r$r%r6rrr9)r*r6rrr+s rr%FBCBlockHook.__init__s  * rcf[U5n[R"UR5UlU$r8r<r>s rrAFBCBlockHook.initialize_hookrCrr?cURRSX#5nSnURRbURRSX#5nURR 5nUR (aUR R"U0UD6nUR(aS=p[U[5(aQXpRRURS- nXpRRURS- n OXvR- nX4Ul U$UcUn U $SS/n XJURR'XZURR'[U 5n U $)NrErWrr)r9rFrNr6rLr)rGrHrrrIrJrKr&r() r*r?rPrQrRoriginal_encoder_hidden_statesrVrSrUencoder_hidden_states_residualrXs rrYFBCBlockHook.new_forwardsb!%!O!OP_ae!n)-& >> < < H-1^^-[-['. *))335  & &[[114B6BF||JNN&fe,,~~HHILLjLjklLmm+~~PPQT`TrTrstTuu3.46T6T-T*5K4l 1M ) 12M  "4LMG]$..CC DOm$..KK L!-0Mr)r9rrr6F)rrrrr rnr%rArhrirjrYrr1r2s@rrprps9lT  !%((//!!rrpr?configr0cl[[S05n/nUR5HfupEU[;d)[ U[ R R5(dM:[U5HupgURUSU3U45 M Mh URS5upURS5up[RSUS35 [XUR5 UH)upG[RSUS35 [Xr5 M+ [RSU S35 [XS S 9 g ) aC Applies [First Block Cache](https://github.com/chengzeyi/ParaAttention/blob/4de137c5b96416489f06e43e19f2c14a772e28fd/README.md#first-block-cache-our-dynamic-caching) to a given module. First Block Cache builds on the ideas of [TeaCache](https://huggingface.co/papers/2411.19108). It is much simpler to implement generically for a wide range of models and has been integrated first for experimental purposes. Args: module (`torch.nn.Module`): The pytorch module to apply FBCache to. Typically, this should be a transformer architecture supported in Diffusers, such as `CogVideoXTransformer3DModel`, but external implementations may also work. config (`FirstBlockCacheConfig`): The configuration to use for applying the FBCache method. Example: ```python >>> import torch >>> from diffusers import CogView4Pipeline >>> from diffusers.hooks import apply_first_block_cache, FirstBlockCacheConfig >>> pipe = CogView4Pipeline.from_pretrained("THUDM/CogView4-6B", torch_dtype=torch.bfloat16) >>> pipe.to("cuda") >>> apply_first_block_cache(pipe.transformer, FirstBlockCacheConfig(threshold=0.2)) >>> prompt = "A photo of an astronaut riding a horse on mars" >>> image = pipe(prompt, generator=torch.Generator().manual_seed(42)).images[0] >>> image.save("output.png") ``` r.rzApplying FBCHeadBlockHook to ''zApplying FBCBlockHook to 'z%Applying FBCBlockHook to tail block 'T)rrN)r r named_childrenrrIrhri ModuleList enumerateappendpoploggerdebug_apply_fbc_head_block_hookr_apply_fbc_block_hook) r?r|r6remaining_blocksname submoduleindexblockhead_block_name head_blocktail_block_name tail_blocks rapply_first_block_cachersB!!4b"=M!002 9 9IW\W_W_WjWjAkAk %i0LE  # #vQug%6$> ?13 #3"6"6q"9O"2"6"6r":O LL1/1B!DEz&:J:JK'  1$q9:e3( LL88IKL*TBrrr6rcr[R"U5n[X5nURU[5 gr8)r check_if_exists_or_initializer4 register_hook_FBC_LEADER_BLOCK_HOOK)rr6rregistryhooks rrrs-99%@H M 5D 4!78rrrcr[R"U5n[X5nURU[5 gr8)r rrpr_FBC_BLOCK_HOOK)rr6rrrrs rrrs,99%@H  /D 41rr{) dataclassesrrhutilsrutils.torch_utilsr_commonr_helpersr hooksr r r r rrrrrr r4rprirjrrrrnrrrrrs" -7.CC H 0"  $ #) #M%yM%`-9-`5CEHHOO5C=R5CW[5Cp9ehhoo9l9_d9im9 222X\2im2r