oMjIISSKrSSKJs Jr SSKJr SSKJrJr SSKJ r \ "5(aSSK J r J r \R"\5rS\R 4SjrS\R 4S jrSS \4S jjr"S S \RR 5r"SS\R 5r"SS\R 5r"SS\R05r"SS\R45rg)N)nn) deprecatelogging)is_transformers_available) CLIPTextModelCLIPTextModelWithProjection text_encoderc0/n[U[[45(aX[URR R 5H)up#SUS3nURnURXE45 M+ U$[SURR35e)Ntext_model.encoder.layers.z .self_attnz.do not know how to get attention modules for: ) isinstancerr enumerate text_modelencoderlayers self_attnappend ValueError __class____name__)r attn_modulesilayernamemods ڣC:\Users\johnn\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\diffusers/models/lora.pytext_encoder_attn_modulesr'sL,0K LMM!,"9"9"A"A"H"HIHA/s*=D//C    ,J I,J`J`JiJiIjkllc0/n[U[[45(aX[URR R 5H)up#URnSUS3nURXT45 M+ U$[SURR35e)Nr z.mlpz(do not know how to get mlp modules for: ) r rr rrrrmlprrrr)r mlp_modulesrrmlp_modrs rtext_encoder_mlp_modulesr#5sK,0K LMM!,"9"9"A"A"H"HIHAiiG/s$7D    /J CLDZDZDcDcCdeffr lora_scalec[U5Hfup#[UR[5(dM&XRlXR lXR lXRlMh [U5HFup$[UR[5(dM&XRlXRlMH gN) rr q_projPatchedLoraProjectionr$k_projv_projout_projr#fc1fc2)r r$_ attn_module mlp_modules radjust_lora_scale_text_encoderr1Cs3LA k((*? @ @,6   ),6   ),6   ).8 + B2,?  jnn&; < <(2NN %(2NN %@rcZ^\rSrSrS U4SjjrSSSS.U4SjjrS SjrS rS rS r U=r $)r(QNc Z>Sn[SSU5 [T U] 5 SSKJn XlUR R RnUc UR R RnU"UR RUR RUUUUS9Ul X l g)NzsUse of `PatchedLoraProjection` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`.r(1.0.0r)LoRALinearLayer) network_alphadevicedtyperank) rsuper__init__ models.lorar6regular_linear_layerweightr8r9 in_features out_featureslora_linear_layerr$) selfr>r$r7r:r9deprecation_messager6r8rs rr<PatchedLoraProjection.__init__RsT)74GH 1$8!**1188 =--44::E!0  % % 1 1  % % 2 2' " %rF destinationprefix keep_varscv>URcURR"XAX#S.6$[TU]"XAX#S.6$)NrG)rBr> state_dictr;)rCrHrIrJargsrs rrL PatchedLoraProjection.state_dictmsA  ! ! ),,77v w!4eerc @URcgURRRRURRRR pCURRRR 5nURRRRR 5nURRRRR 5nURRb/X`RR-URR- nXQ[R"USSS24USSS245S--nU(aF[R"U5R5R5(a[!SUS35eUR#XCS9URRlSUlUR%5UlUR%5UlXlgNraThis LoRA weight seems to be broken. Encountered NaN values when trying to fuse LoRA weights for .LoRA weights will not be fused.r8r9)rBr>r?datar9r8floatupdownr7r:torchbmmisnananyitemrtocpuw_upw_downr$ rCr$ safe_fusingr9r8w_origr_r` fused_weights r _fuse_lora PatchedLoraProjection._fuse_loraus  ! ! ) 1188==CCTE^E^EeEeEjEjEqEqv**1166<<>%%((//44::<'',,3388>>@  ! ! / / ;00>>>AWAWA\A\\DeiiT1W vdTUg.WXY.Z!Z[ 5;;|488:??AAOOSfU22  1=v0[!!((-"&HHJ jjl $rc D[USS5b[USS5cgURRRnURUR p2UR RUS9R5nURRU5R5nUR5UR[R"USSS24USSS245S-- nURX2S9URRlSUlSUl gNr_r`r8rrS) getattrr>r?rTr9r8r_r]rUr`r$rXrYrCrdr9r8r_r`unfused_weights r _unfuse_lora"PatchedLoraProjection._unfuse_lorasfd+7GD(TX0A0A0A0]!!((-  rcURcSUlURcURU5$URU5URURU5--$)N?)r$rBr>)rCinputs rforwardPatchedLoraProjection.forwards[ ?? "!DO  ! ! ),,U3 3((/4??TE[E[\aEb3bccr)rBr$r>r`r_)NNrpF) r __module__ __qualname____firstlineno__r<rLrermrr__static_attributes__ __classcell__rs@rr(r(Qs4%6-1uff%@ ddrr(c^\rSrSrSrSS\S\S\S\S-S\R\ -S-S \RS-4 U4S jjjr S \RS \R4S jr SrU=r$)r6aM A linear layer that is used with LoRA. Parameters: in_features (`int`): Number of input features. out_features (`int`): Number of output features. rank (`int`, `optional`, defaults to 4): The rank of the LoRA layer. network_alpha (`float`, `optional`, defaults to `None`): The value of the network alpha used for stable learning and preventing underflow. This value has the same meaning as the `--network_alpha` option in the kohya-ss trainer script. See https://github.com/darkstorm2150/sd-scripts/blob/main/docs/train_network_README-en.md#execute-learning device (`torch.device`, `optional`, defaults to `None`): The device to use for the layer's weights. dtype (`torch.dtype`, `optional`, defaults to `None`): The dtype to use for the layer's weights. Nr@rAr:r7r8r9c>[TU]5 Sn[SSU5 [R"XSXVS9Ul[R"X2SXVS9UlX@lX0lX l Xl [RRUR RSU- S9 [RRUR R5 g)NzmUse of `LoRALinearLayer` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`.r6r5F)biasr8r9rtstd)r;r<rrLinearrWrVr7r:rAr@initnormal_r?zeros_) rCr@rAr:r7r8r9rDrs rr<LoRALinearLayer.__init__s N#W.ABIIkeFX ))DU6W+ (&  ((a$h7 tww~~&r hidden_statesreturnc.URnURRRnURURU55nUR U5nUR bXPR UR - -nURU5$r&r9rWr?r]rVr7r:rCr orig_dtyper9down_hidden_statesup_hidden_statess rrrLoRALinearLayer.forward"((    &&!YY}'7'7'>?77#56    )  2 2TYY > > "":..r)rWr@r7rAr:rV)ruNNN)rrwrxry__doc__intrUrXr8strr9r<Tensorrrrzr{r|s@rr6r6s0&*,0$(''' ' t| '  s"T) '{{T!''4 /U\\ /ell / /rr6c^\rSrSrSrSS\S\S\S\\\\4-S\\\\4-S \\\\4-\-S \S-4U4S jjjr S \ RS \ R4Sjr Sr U=r$)LoRAConv2dLayera A convolutional layer that is used with LoRA. Parameters: in_features (`int`): Number of input features. out_features (`int`): Number of output features. rank (`int`, `optional`, defaults to 4): The rank of the LoRA layer. kernel_size (`int` or `tuple` of two `int`, `optional`, defaults to 1): The kernel size of the convolution. stride (`int` or `tuple` of two `int`, `optional`, defaults to 1): The stride of the convolution. padding (`int` or `tuple` of two `int` or `str`, `optional`, defaults to 0): The padding of the convolution. network_alpha (`float`, `optional`, defaults to `None`): The value of the network alpha used for stable learning and preventing underflow. This value has the same meaning as the `--network_alpha` option in the kohya-ss trainer script. See https://github.com/darkstorm2150/sd-scripts/blob/main/docs/train_network_README-en.md#execute-learning Nr@rAr: kernel_sizestridepaddingr7c >[T U]5 Sn[SSU5 [R"XXEUSS9Ul[R"X2SSSS9UlXplX0l[RRUR RSU- S 9 [RRUR R5 g) NzmUse of `LoRAConv2dLayer` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`.rr5F)rrrrrtrt)rrrrtr) r;r<rrConv2drWrVr7r:rrr?r) rCr@rAr:rrrr7rDrs rr<LoRAConv2dLayer.__init__s N#W.ABIIk[ahotu ))DF6X]^+   ((a$h7 tww~~&rrrc.URnURRRnURURU55nUR U5nUR bXPR UR - -nURU5$r&rrs rrrLoRAConv2dLayer.forwardrr)rWr7r:rV)rurrrN)rrwrxryrrtuplerrUr<rXrrrrzr{r|s@rrrs4-3(./0&*''' ' 5c?* ' eCHo% 'uS#X&,'t|''8 /U\\ /ell / /rrc^\rSrSrSrSS.S\S-4U4SjjjrS\S-4SjrSS\S \ 4S jjr S r SS \ RS \S\ R4SjjrSrU=r$)LoRACompatibleConvi)z3 A convolutional layer that can be used with LoRA. N lora_layerrcP>Sn[SSU5 [TU]"U0UD6 Xlg)NzpUse of `LoRACompatibleConv` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`.rr5rr;r<rrCrrMkwargsrDrs rr<LoRACompatibleConv.__init__.s1Q&1DE $)&)$rc.Sn[SSU5 XlgNznUse of `set_lora_layer()` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`.set_lora_layerr5rrrCrrDs rr!LoRACompatibleConv.set_lora_layer5sO"G-@A$rr$rbc<URcgURRRURRRpCURRR 5nURR RRR 5nURRRRR 5nURRb/X`RR-URR- n[R"URSS9URSS95nURUR5nXQU--n U(aF[R"U 5R!5R#5(a[%SUS35eU R'XCS9URlSUlUR)5UlUR)5UlXlg)Nrt start_dimrQrRrS)rr?rTr9r8rUrVrWr7r:rXmmflattenreshapeshaperZr[r\rr]r^r_r` _lora_scale) rCr$rbr9r8rcr_r`fusionrds rreLoRACompatibleConv._fuse_lora;s ?? "  ((.. 0@0@0G0Gv!!'')!!((--335%%,,11779 ?? ( ( 4//777$//:N:NND$,,,3V^^a^5PQ/f!45 5;;|488:??AAOOSfU22  (??&?F HHJ jjl %rc[USS5b[USS5cgURRnURRURRp2UR R US9R5UlURR U5R5Ul[R"UR RSS9URRSS95nURUR5nUR5URU-- nUR X2S9URlSUlSUlg)Nr_r`rirtrrS)rjr?rTr9r8r_r]rUr`rXrrrrr)rCrdr9r8rrls rrmLoRACompatibleConv._unfuse_lora]sfd+7GD(TXSn[SSU5 [TU]"U0UD6 Xlg)NzrUse of `LoRACompatibleLinear` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`.rr5rrs rr<LoRACompatibleLinear.__init__s1S('3FG $)&)$rc.Sn[SSU5 Xlgrrrs rr#LoRACompatibleLinear.set_lora_layersO"G-@A$rr$rbc URcgURRRURRRpCURRR 5nURR RRR 5nURRRRR 5nURRb/X`RR-URR- nXQ[R"USSS24USSS245S--nU(aF[R"U5R5R5(a[SUS35eUR!XCS9URlSUlUR#5UlUR#5UlXlgrP)rr?rTr9r8rUrVrWr7r:rXrYrZr[r\rr]r^r_r`rras rreLoRACompatibleLinear._fuse_loras ?? "  ((.. 0@0@0G0Gv!!'')!!((--335%%,,11779 ?? ( ( 4//777$//:N:NNDeiiT1W vdTUg.WXY.Z!Z[ 5;;|488:??AAOOSfU22  (??&?F HHJ jjl %rc [USS5b[USS5cgURRnURURp2UR R US9R5nURR U5R5nUR5UR[R"USSS24USSS245S-- nUR X2S9URlSUlSUlgrh) rjr?rTr9r8r_r]rUr`rrXrYrks rrm!LoRACompatibleLinear._unfuse_lorasfd+7GD(TXURc[TU] U5nU$[TU] U5X RU5--nU$r&)rr;rr)rCrroutrs rrrLoRACompatibleLinear.forwardsD ?? "'/-0CJ'/-0EOOMrs0 &:G   H % BII  299  3U 3YdEHHOOYdx9/bii9/x=/bii=/@TOTOnG299Gr