oMjSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSK J r SSK J r J r SSKJr SSKJr SSKJrJrJrJr SSKrSSKrSSKrSSKJrJrJr SSKJr SS KJrJ r SS K!J"r" S S K#J$r$ S S K%J&r&J'r' S SK(J)r) S SK*J+r+J,r,J-r-J.r.J/r/J0r0J1r1J2r2J3r3J4r4J5r5J6r6J7r7J8r8J9r9J:r:J;r;Jr> S SK?J@r@JArAJBrB S SKCJDrD SSKEJFrFJGrGJHrHJIrI SSKJJKrKJLrLJMrMJNrNJOrOJPrPJQrQJRrR "SS5rS\=1.9.0TF)dispatch_model)load_offloaded_weightssave_offload_index parameterreturncSSKJn U"U5$![a Of=f[R"UR 5UR 55n[U5R$![ae S[RRS[[[[ 44SjnUR#US9n[U5nUSRs$f=f)Nr)_get_group_onload_devicemodulerncURR5VVs/sH%up[R"U5(dM"X4PM' nnnU$s snnfrA__dict__itemstorch is_tensorrqkvtupless rDfind_tensor_attributes4get_parameter_device..find_tensor_attributessB)/)>)>)@W)@EOOTUDVfqf)@FWMX !AAget_members_fnr/)hooks.group_offloadingrp ValueError itertoolschain parametersbuffersnextdevice StopIterationrvrModulerWtuplestrr_named_members)rmrpparameters_and_buffersr|gen first_tuples rDget_parameter_devicersA ' 22    %"+1E1E1GIZIZI\!]*+222  % 588?? tE#v+DV?W &&6L&M3i 1~$$$ %s AA))A,CCc^ [U[R5(a[UR5HGum n[ US5(dMUR nUR S5nUcM;URs $ SnUR5H}um nURn[ US5(a7UR(a&[U 4SjUR55(aMZUR5(dMqURs $ UR5H2nURnUR5(dM&URs $ UbU$S[RS[W[[ 44SjnUR#US 9n Sn U H.nUn US R5(dMUS Rs $ U bU S R$g) zr Returns the first found floating dtype in parameters if there is one, otherwise returns the last dtype it found. _diffusers_hooklayerwise_castingN_keep_in_fp32_modulesc3,># UH oT;v M g7frArY).0mnames rD &get_parameter_dtype..sG'F!I'FsrqrncURR5VVs/sH%up[R"U5(dM"X4PM' nnnU$s snnfrArsrxs rDr|3get_parameter_dtype..find_tensor_attributessB%+__%:%:%<S%> ! !}"""rGc## Sn[R5H)up[[RR X5 M+ Sv [R5H)up[[RR X5 M+ g![R5H)up[[RR X5 M+ f=f7f)z Context manager to globally disable weight initialization to speed up loading large models. To do that, all the torch.nn.init function are all replaced with skip. cgrArY)rOrPs rD _skip_init#no_init_weights.._skip_inits rGN)TORCH_INIT_FUNCTIONSrusetattrrvrinit)rr init_funcs rDno_init_weightsrs 0557 t084  499;OD EHHMM4 3 <399;OD EHHMM4 3 j5r<\;"\&RVRXRz5U4S?j5r=U4S@jr>U4SAjr?SBr@SSC.SD\A\B-\C-SE\D\\E4S-4SFjjrF\7SgSG\GSH\H\S<\\3Rh-SI\H\SJ\SK\SL\IS-SM\SN\\&RP-S-SO\H\S-SP\\/-\&R\-\D\\\/-\&R\-4-SQ\S-SR\\3Rh-S-SS\D\\J4S-ST\S-SU\4 SVjj5rK\7SW5rLSP\4SXjrM\7SN\&RPS \&RP4SYj5rN\S \&R\4SZj5r.\S \&RP4S[j5r(ShS\\S]\S \/4S^jjrOSiS_jrPS\&RRR4S`\S \S S4SajjrSSG\GS S4SbjrTScrUU=rV$)j ModelMixina Base class for all models. [`ModelMixin`] takes care of storing the model configuration and provides methods for loading, downloading and saving models. - **config_name** ([`str`]) -- Filename to save a model to when calling [`~models.ModelMixin.save_pretrained`]. )_diffusers_version _class_name _name_or_pathFNTc0>[TU]5 SUlgrA)superrE_gradient_checkpointing_func)rC __class__s rDrEModelMixin.__init__ s ,0)rGrrnc `>SUR;=(a [URSU5nXR;nU(a[U(dTSUS[U5RSUS[U5RSUS3 n[ SS US S S 9 UR U$[ TU]U5$) a^The only reason we overwrite `getattr` here is to gracefully deprecate accessing config attributes directly. See https://github.com/huggingface/diffusers/pull/3129 We need to overwrite __getattr__ here in addition so that we don't trigger `torch.nn.Module`'s __getattr__': https://pytorch.org/docs/stable/_modules/torch/nn/modules/module.html#Module _internal_dictzAccessing config attribute `z` directly via 'z1' object attribute is deprecated. Please access 'z' over 'z,'s config object instead, e.g. 'unet.config.z'.zdirect config name accessz1.0.0F) standard_warn stacklevel)rtrtyperRr"rr __getattr__)rCr is_in_config is_attributedeprecation_messagers rDrModelMixin.__getattr__s(4==8kWT]]ScEdfj=k }},  $@FVW[\`WaWjWjVkl]^b]ccklpqulvllk@@lmqlrrt#u  17.%s)m^lYZ7167TA._gradient_checkpointing_func9sR:J4QY:Z:Z6`b {{--88OO"rGT)enabler) _supports_gradient_checkpointingrrrR_set_gradient_checkpointing)rCrrs rDenable_gradient_checkpointing(ModelMixin.enable_gradient_checkpointing's`44>>**+,XY  ' . +G ' ((Rm(nrGcFUR(aURSS9 gg)z Deactivates gradient checkpointing for the current model (may be referred to as *activation checkpointing* or *checkpoint activations* in other frameworks). F)rN)rrrs rDdisable_gradient_checkpointing)ModelMixin.disable_gradient_checkpointingEs#  0 0  , ,E , : 1rGvalidc^^S[RR4UU4SjjmUR5H6n[ U[RR5(dM.T"U5 M8 g)z- Set the switch for the npu flash attention. rqc>[US5(aURT5 UR5H nT"U5 M g)Nset_use_npu_flash_attention)rrchildren)rqchild$fn_recursive_set_npu_flash_attentionrs rDrTModelMixin.set_use_npu_flash_attention..fn_recursive_set_npu_flash_attentionRs8v<==2259*4U;+rGNrvrrrr)rCrrqrs ` @rDr&ModelMixin.set_use_npu_flash_attentionMsK  < < <mmoF&%((//224V<&rGc&URS5 g)z, Enable npu flash attention from torch_npu TNrrs rDenable_npu_flash_attention%ModelMixin.enable_npu_flash_attention]s ((.rGc&URS5 g)z- disable npu flash attention from torch_npu FNrrs rDdisable_npu_flash_attention&ModelMixin.disable_npu_flash_attentionds ((/rGuse_xla_flash_attentionpartition_specc ^^^^S[RR4UUUU4SjjmUR5H6n[ U[RR5(dM.T"U5 M8 g)Nrqc>[US5(aUR"TT40TD6 UR5H nT"U5 M g)Nset_use_xla_flash_attention)rrr)rqr fn_recursive_set_flash_attentionrPrrs rDrPModelMixin.set_use_xla_flash_attention..fn_recursive_set_flash_attentionqsAv<==223JNe^de*07+rGr)rCrrrPrqrs ``` @rDr&ModelMixin.set_use_xla_flash_attentionksK  8UXX__ 8 8mmoF&%((//2208&rGc ,UR"SU40UD6 g)z: Enable the flash attention pallals kernel for torch_xla. TNr)rCrrPs rDenable_xla_flash_attention%ModelMixin.enable_xla_flash_attention|s ((~HHrGc&URS5 g)z; Disable the flash attention pallals kernel for torch_xla. FNrrs rDdisable_xla_flash_attention&ModelMixin.disable_xla_flash_attentions ((/rG attention_opc^^^S[RR4UUU4SjjmUR5H6n[ U[RR5(dM.T"U5 M8 g)Nrqc>[US5(aURTT5 UR5H nT"U5 M g)N+set_use_memory_efficient_attention_xformers)rr r)rqrrfn_recursive_set_mem_effrs rDr XModelMixin.set_use_memory_efficient_attention_xformers..fn_recursive_set_mem_effs:vLMMBB5,W*(/+rGr)rCrrrqr s `` @rDr 6ModelMixin.set_use_memory_efficient_attention_xformerssK 0UXX__ 0 0mmoF&%((//22(0&rGc(URSU5 g)ux Enable memory efficient attention from [xFormers](https://facebookresearch.github.io/xformers/). When this option is enabled, you should observe lower GPU memory usage and a potential speed up during inference. Speed up during training is not guaranteed. > [!WARNING] > ⚠️ When memory efficient attention and sliced attention are both enabled, memory efficient attention takes > precedent. Parameters: attention_op (`Callable`, *optional*): Override the default `None` operator for use as `op` argument to the [`memory_efficient_attention()`](https://facebookresearch.github.io/xformers/components/ops.html#xformers.ops.memory_efficient_attention) function of xFormers. Examples: ```py >>> import torch >>> from diffusers import UNet2DConditionModel >>> from xformers.ops import MemoryEfficientAttentionFlashAttentionOp >>> model = UNet2DConditionModel.from_pretrained( ... "stabilityai/stable-diffusion-2-1", subfolder="unet", torch_dtype=torch.float16 ... ) >>> model = model.to("cuda") >>> model.enable_xformers_memory_efficient_attention(attention_op=MemoryEfficientAttentionFlashAttentionOp) ``` TNr )rCrs rD*enable_xformers_memory_efficient_attention5ModelMixin.enable_xformers_memory_efficient_attentions< 88|LrGc&URS5 g)zc Disable memory efficient attention from [xFormers](https://facebookresearch.github.io/xformers/). FNrrs rD+disable_xformers_memory_efficient_attention6ModelMixin.disable_xformers_memory_efficient_attentions 88?rG storage_dtyperskip_modules_pattern.skip_modules_classes non_blockingcSSKJn SnUc SSKJn UnSnURbU[ UR5- nUR bU[ UR 5- n[ [U55n[5(a<U(d5SSK J n SS K J n SS K Jn XU 4Hn U[ U R5- nM Uc![ R#S 5 UR$nU"XX#XE5 g) a Activates layerwise casting for the current model. Layerwise casting is a technique that casts the model weights to a lower precision dtype for storage but upcasts them on-the-fly to a higher precision dtype for computation. This process can significantly reduce the memory footprint from model weights, but may lead to some quality degradation in the outputs. Most degradations are negligible, mostly stemming from weight casting in normalization and modulation layers. By default, most models in diffusers set the `_skip_layerwise_casting_patterns` attribute to ignore patch embedding, positional embedding and normalization layers. This is because these layers are most likely precision-critical for quality. If you wish to change this behavior, you can set the `_skip_layerwise_casting_patterns` attribute to `None`, or call [`~hooks.layerwise_casting.apply_layerwise_casting`] with custom arguments. Example: Using [`~models.ModelMixin.enable_layerwise_casting`]: ```python >>> from diffusers import CogVideoXTransformer3DModel >>> transformer = CogVideoXTransformer3DModel.from_pretrained( ... "THUDM/CogVideoX-5b", subfolder="transformer", torch_dtype=torch.bfloat16 ... ) >>> # Enable layerwise casting via the model, which ignores certain modules by default >>> transformer.enable_layerwise_casting(storage_dtype=torch.float8_e4m3fn, compute_dtype=torch.bfloat16) ``` Args: storage_dtype (`torch.dtype`): The dtype to which the model should be cast for storage. compute_dtype (`torch.dtype`): The dtype to which the model weights should be cast during the forward pass. skip_modules_pattern (`tuple[str, ...]`, *optional*): A list of patterns to match the names of the modules to skip during the layerwise casting process. If set to `None`, default skip patterns are used to ignore certain internal layers of modules and PEFT layers. skip_modules_classes (`tuple[Type[torch.nn.Module], ...]`, *optional*): A list of module classes to skip during the layerwise casting process. non_blocking (`bool`, *optional*, defaults to `False`): If `True`, the weight casting operations are non-blocking. r)apply_layerwise_castingTN)DEFAULT_SKIP_MODULES_PATTERNFr) LoHaLayer) LoKrLayer) LoraLayerzW`compute_dtype` not provided when enabling layerwise casting. Using dtype of the model.)hooksrhooks.layerwise_castingrrr _skip_layerwise_casting_patternssetr'peft.tuners.loha.layerrpeft.tuners.lokr.layerrpeft.tuners.lora.layerradapter_layer_namesloggerinfor) rCrrrrrruser_provided_patternsrrrrlayers rDenable_layerwise_casting#ModelMixin.enable_layerwise_castingsd 4!%  ' N#? %* "  % % 1 E$*D*D$E E  0 0 < E$*O*O$P P $S)=%>?   '= 9 8 8# :$e.G.G(HH$;  KKq r JJM FZ rGcpu block_level onload_deviceoffload_device offload_typenum_blocks_per_group use_stream record_streamoffload_to_disk_path block_modulesexclude_kwargsc SSKJn [USS5b0[USS5(aU(aSn [R U 5 UR (d"[ URRS35eU "UUUUUUUUUU U U S 9 g) aY Activates group offloading for the current model. See [`~hooks.group_offloading.apply_group_offloading`] for more information. Example: ```python >>> from diffusers import CogVideoXTransformer3DModel >>> transformer = CogVideoXTransformer3DModel.from_pretrained( ... "THUDM/CogVideoX-5b", subfolder="transformer", torch_dtype=torch.bfloat16 ... ) >>> transformer.enable_group_offload( ... onload_device=torch.device("cuda"), ... offload_device=torch.device("cpu"), ... offload_type="leaf_level", ... use_stream=True, ... ) ``` r)apply_group_offloading enable_tilingN use_tilingFa/Applying group offloading on autoencoders, with CUDA streams, may not work as expected if the first forward pass is executed with tiling enabled. Please make sure to either: 1. Run a forward pass with small input shapes. 2. Or, run a forward pass with tiling disabled (can still use small dummy inputs).z does not support group offloading. Please make sure to set the boolean attribute `_supports_group_offloading` to `True` in the class definition. If you believe this is a mistake, please open an issue at https://github.com/huggingface/diffusers/issues.) rqr/r0r1r2rr3r4low_cpu_mem_usager5r6r7) rr9getattrr'warning_supports_group_offloadingrrrR)rCr/r0r1r2rr3r4r<r5r6r7r9msgs rDenable_group_offloadModelMixin.enable_group_offloadsH 3 4$ / ;l\a@b@bgqe  NN3 ..>>**+,TU  ')%!5%!'/!5') rGbackendcjSSKJn SSKJnJnJnJn SSKJnJ n [RS5 XxU4n Sn UR5H3n [X5(dMU Rn [U SS5cM1S n O UR!5nUR"R%5V s1sHoR&iM nn X;a"[)S U<S 3S R+U5-5eU"U5nU (aGUR-U5(d1[/UR05n[)S UR&SUS35eU"U5 U"U5 UR5H=n [X5(dMU Rn U b[3U S5(dM7XlM? UR7U5 gs sn f)ai Set the attention backend for the model. Args: backend (`str`): The name of the backend to set. Must be one of the available backends defined in `AttentionBackendName`. Available backends can be found in `diffusers.attention_dispatch.AttentionBackendName`. Defaults to torch native scaled dot product attention as backend. r/AttentionModuleMixin)AttentionBackendName_AttentionBackendRegistry%_check_attention_backend_requirements"_maybe_download_kernel_for_backend AttentionMochiAttentionTAttention backends are an experimental feature and the API may be subject to change.F_parallel_configNTz `backend=z ` must be one of the following: , z>Context parallelism is enabled but current attention backend 'zS' does not support context parallelism. Please set a compatible attention backend: z' using `model.set_attention_backend()`._attention_backend) attentionrFattention_dispatchrGrHrIrJattention_processorrLrMr'r>rr processorr=lower __members__valuesvaluerjoin_is_context_parallel_availablesorted_supports_context_parallelrrQset_active_backend)rCrCrFrGrHrIrJrLrMattention_classesparallel_config_setrqrUxavailable_backendscompatible_backendss rDset_attention_backend ModelMixin.set_attention_backendVs 4   Cmn&8LM#llnFf88((Iy"4d;G&*# %--//C/O/O/V/V/XY/X!gg/XY  ,z *JKdiiXjNkkl l&w/ '@'_'_`g'h'h"()B)])]"^ PQXQ^Q^P_`>>Q=RRy{  .g6*73llnFf88((I  ;O(P(P+2 ( % "44W=3Zs)F0cSSKJn SSKJnJn [ R S5 X#U4nUR5H>n[XT5(dMURnUb[US5(dM7SUl M@ g)z Resets the attention backend for the model. Following calls to `forward` will use the environment default, if set, or the torch native scaled dot product attention. r/rErKrNNrQ) rRrFrTrLrMr'r>rrrUrrQ)rCrFrLrMr_rqrUs rDreset_attention_backend"ModelMixin.reset_attention_backendsj 4Bmn&8LMllnFf88((I  ;O(P(P+/I ( %rGsave_directoryis_main_process save_functionsafe_serializationvariantmax_shard_size push_to_hub use_flashpackc [RRU5(a[R SUS35 g[ USS5n U b{U SL=(a# [ U [5=(a U Rn U(aU (aU =(a U Rn U (d#[SU RRS35e[n U(a[n O U(a[n [!X5n U R#SS5R#S S 5n [R$"US S 9 U(aU R'S S5nU R'SS5nU R'SS5nU R'SS5nU R'SUR)[RR*5S5n[-US UUS9R.nUnU(aUR1U5 UR35n0nU bU R5UUS9unnU(ax[75(aSSKnO [R S5 [;S5eUR<R?U[RRAX5URBS9 GOn[EUXmS9nU(Ga![RF"U5GHnUURHRK5;aM$[RRAUU5n[RRU5(dMjU R#SS5R#S S5nUR#SS5nUR#SS5R#S S5nURMU5(dM[NRQU5cM[RR"U5 GM URHRU5HunnUVs0sHnUUURW5_M nn[RRAUU5nU(aSS0n U(aU RYU5 U RU5V!V"s0sH)un!n"U![ U"[Z5(d [[U"5OU"_M+ n n!n"[\R^RaUUU S9 M[^Rb"UU5 M URd(a[gURh5n U(aU RYU5 U URjS .n#U(a[lO[nn$[RRAU[!U$U55n$[qU$S!S"S#9n%[rRt"U#S$S S%9S&-n&U%RwU&5 SSS5 [RyS'US([{URH5S)U$S*35 O7[RRAX5n'[RyS+U'35 U(aR[}WWS,9n([U(5n(U(Rc[US-5R55 URUUUWWS.9 ggs snfs sn"n!f!,(df  N=f)/a Save a model and its configuration file to a directory so that it can be reloaded using the [`~models.ModelMixin.from_pretrained`] class method. Arguments: save_directory (`str` or `os.PathLike`): Directory to save a model and its configuration file to. Will be created if it doesn't exist. is_main_process (`bool`, *optional*, defaults to `True`): Whether the process calling this is the main process or not. Useful during distributed training and you need to call this function on all processes. In this case, set `is_main_process=True` only on the main process to avoid race conditions. save_function (`Callable`): The function to use to save the state dictionary. Useful during distributed training when you need to replace `torch.save` with another method. Can be configured with the environment variable `DIFFUSERS_SAVE_MODE`. safe_serialization (`bool`, *optional*, defaults to `True`): Whether to save the model using `safetensors` or the traditional PyTorch way with `pickle`. variant (`str`, *optional*): If specified, weights are saved in the format `pytorch_model..bin`. max_shard_size (`int` or `str`, defaults to `"10GB"`): The maximum size for a checkpoint before being sharded. Checkpoints shard will then be each of size lower than this size. If expressed as a string, needs to be digits followed by a unit (like `"5GB"`). If expressed as an integer, the unit is bytes. Note that this limit will be decreased after a certain period of time (starting from Oct 2024) to allow users to upgrade to the latest version of `diffusers`. This is to establish a common default size for this argument across different libraries in the Hugging Face ecosystem (`transformers`, and `accelerate`, for example). push_to_hub (`bool`, *optional*, defaults to `False`): Whether or not to push your model to the Hugging Face Hub after saving it. You can specify the repository you want to push to with `repo_id` (will default to the name of `save_directory` in your namespace). kwargs (`dict[str, Any]`, *optional*): Additional keyword arguments passed along to the [`~utils.PushToHubMixin.push_to_hub`] method. zProvided path (z#) should be a directory, not a fileN hf_quantizerzThe model is quantized with z and is not serializable - check out the warnings from the logger on the traceback to understand the reason why the quantized model is not serializable.z.binz {suffix}.binz .safetensorsz{suffix}.safetensorsTexist_okcommit_messageprivate create_prFtokenrepo_id)rtrvrx)rlrzSaving a FlashPack checkpoint in PyTorch, requires both PyTorch and flashpack to be installed. Please see https://pytorch.org/ and https://github.com/fal-ai/flashpack for installation instructions.zMPlease install torch and flashpack to save a FlashPack checkpoint in PyTorch.)state_dict_or_modeldestination_path target_dtype)rnfilename_patternz{suffix}formatpt)metadata)r weight_mapwzutf-8)encodingr)indent sort_keys z:The model is bigger than the maximum size per checkpoint (z) and is going to be split in z^ checkpoint shards. You can find where each parameters has been saved in the index located at .zModel weights saved in )rxz README.md)rxrurw)Cospathisfiler'errorr=rris_serializable"supports_safetensors_serializationrquantization_config quant_methodrrrrreplacemakedirspopsplitsepr ry save_config state_dictget_state_dict_and_metadatar& flashpack ImportError serialization pack_to_filerZrrlistdirfilename_to_tensorskeys startswith _REGEX_SHARD fullmatchremoveru contiguousupdater safetensorsrv save_filesave is_shardeddictrtensor_to_filenamerropenjsondumpswriter(lenr,r-ras_posix_upload_folder))rCrirjrkrlrmrnrorprPrrquantization_serializable weights_nameweights_name_patternrurvrwrxry model_to_saverquantization_metadatarstate_dict_splitfilename full_filenameweights_without_extfilename_without_exttensorstensorshardfilepathrryrzindexsave_index_filefcontentpath_to_weights model_cards) rDsave_pretrainedModelMixin.save_pretrainedsZ 77>>. ) ) LL?>*::]^ _ t^T:  #D(1|-?@1 00 & "&?-a,2a2a*- 2<3S3S3`3`2abyy $ 1L 3L#L: +33FNKSS 2  NT2 #ZZ(8$?NjjD1G ;6IJJw-EjjN,@,@,Mb,QRG!'D'QVW__G    % %n 5#--/ "  #0<0X0X/A1Y1 -J- %''  r""qrr  # # 0 0$.!#n!K!ZZ 1  B>    " > :H#3#G#G#L#L#NN $&GGLL$JM77>>-88 *>*F*Fvr*R*Z*Z[ikm*n'*=*E*EjRT*U'+3+;+;FB+G+O+OP^`b+c(!++,?@@(223GHT -0!; &6%I%I%O%O%Q!'OVWwVF!3!>!>!@@wW77<<A% ($/H, (=>W_WeWeWghWgtqRSAs1C1C3q6 JWgHh %%//x(/SJJuh/&R ** 0 9 9:(OO$9: ("2"E"E>P"9Ug"$'',,~|O]d?e"f/3AQ"jjqDIDPGGGG$B PQ_P`a #$4$H$H IJK((7'8; #%'',,~"L 5o5FGH 27%HJ,Z8J OOD=FFH I   -#   IX i$BAs&Y0Y +Y"" Y0cZ[USS5nUc [S5eURU5$)zv Potentially dequantize the model in case it has been quantized by a quantization method that support dequantization. rrNz?You need to first quantize your model in order to dequantize it)r=r dequantize)rCrrs rDrModelMixin.dequantizehs5 t^T:  ^_ _&&t,,rGpretrained_model_name_or_pathc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n U bU OU n URS S5n URSS5nURSS5nURSS5nURSS5nURS[5nURSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS05n[nU(aU(d [S5eU bH[ U [ R 5(d)[ Rn [RSU S35 SnUcSnSnU(a&[5(dSn[RS5 Ub[5(d [S 5eUb[S!S"5(d [S#5eUSLa[S!S"5(d [S$5eUSLaUb[S%US&35e[ U[ R5(aS'U0nO_[ U[5(a US(;aS'[ R"U50nO*[ U[ 5(aUS+:a [S,5eS'U0nUbUcSnOU(d [S-5eU(aUb[S!S.5(d [S/5e["S0S1[UR$5S2.n0nUnUR&"U4USSUUUU U U UUS3. UD6un nn![(R*"U 5n SU ;=(a U SSLn"U"(dUbCU"(a[,R."U SU5U S'OUU S'[,R0"U SU"S49n#OSn#U#bU#R3XS59 U#R5U 5n U#R7U5nU#R8R:R<US6'[>R@"U#R8RC5SS79n$[EU$5[F::aU$US8'UcSn[RIS95 OU(d [S:5eURJSL=(a U#SL=(d [MU#S;S5n%U%(aRURJn&[ U&[N5(dU&/n&UcSn[RIS<5 OU(d [S=5eO/n&Sn'Sn(Sn)Sn*[PRRRUU5n+U+UU =(d S'UUUUUUU U UU!US>.n,[WSW0U,D6n*Ub2U*b$[PRRRYU*5(d [[SW0U,D6n*U*bU(dU*R]5(aSn'U'(a[_UU*UUUU UU U =(d S'US?9 un(n)O@U(a[`n-OU(a[c[dU5n-OSn-U-b[gUU-UUUUU U U UU!US@9 n(U(c)U'(d"[gU[c[lU5UUUUU U U UU!US@9 n([ U([N5(dU(/n(Sn/U b_U [M[ SDS5:XdJ[ U [ R 5(d[U SE[oU 5S*35eURqU 5n/[s5/n0U(a$U0Ru[vRx"55 [{U05 UR0"U 40UD6n1SSS5 U(Ga#[}5(aS+SK?n2O [RkSF5 [SG5eUc,[RSH5 [ R"SI5n3ObUS'n4[ U4[5(aU4S(;a [SJ5e[ U4[ R5(d[ R"U45OU4n3U2RR"SWW1U(S+U3SK.UD6 U/b[ R"U/5 U(a[RSL5 U104$U1$U/b[ R"U/5 Sn5U'(d[U(S+UUSM9n5W1RU55 U'(aU)SNn6O[OU5R55n6U(n7U#bU#RU6U75n6U#bU#RW1UU&SO9 U#bU#R(dSn1XU U&U#5nU#bU#R3USP9 URU1U5U(UU6UUUUUU U#U&UUUSQ9un1n8n9n:n;n[U140U>D6 U#bU#RU15 U#U1lNU b0U [M[ SDS5:XaU#cU%(dU1RU 5n1U#bU1RXST9 OU1RUSU9 U1R5 UbU1RUSV9 U(aU1U=4$U1$![a [S)US*35ef=f![haCn.[RkSAUSBU.35 U(de[RSC5 Sn.A.GN>Sn.A.ff=f!,(df  GNP=f)Xu Instantiate a pretrained PyTorch model from a pretrained model configuration. The model is set in evaluation mode - `model.eval()` - by default, and dropout modules are deactivated. To train the model, set it back in training mode with `model.train()`. Parameters: pretrained_model_name_or_path (`str` or `os.PathLike`, *optional*): Can be either: - A string, the *model id* (for example `google/ddpm-celebahq-256`) of a pretrained model hosted on the Hub. - A path to a *directory* (for example `./my_model_directory`) containing the model weights saved with [`~ModelMixin.save_pretrained`]. cache_dir (`str | os.PathLike`, *optional*): Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used. dtype (`torch.dtype`, *optional*): Override the default `torch.dtype` and load the model with another dtype. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. proxies (`dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. output_loading_info (`bool`, *optional*, defaults to `False`): Whether or not to also return a dictionary containing missing keys, unexpected keys and error messages. local_files_only(`bool`, *optional*, defaults to `False`): Whether to only load local model weights and configuration files or not. If set to `True`, the model won't be downloaded from the Hub. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from `diffusers-cli login` (stored in `~/.huggingface`) is used. revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git. subfolder (`str`, *optional*, defaults to `""`): The subfolder location of a model file within a larger model repository on the Hub or locally. mirror (`str`, *optional*): Mirror source to resolve accessibility issues if you're downloading a model in China. We do not guarantee the timeliness or safety of the source, and you should refer to the mirror site for more information. device_map (`int | str | torch.device` or `dict[str, int | str | torch.device]`, *optional*): A map that specifies where each submodule should go. It doesn't need to be defined for each parameter/buffer name; once a given module name is inside, every submodule of it will be sent to the same device. Defaults to `None`, meaning that the model will be loaded on CPU. Examples: ```py >>> from diffusers import AutoModel >>> import torch >>> # This works. >>> model = AutoModel.from_pretrained( ... "stabilityai/stable-diffusion-xl-base-1.0", subfolder="unet", device_map="cuda" ... ) >>> # This also works (integer accelerator device ID). >>> model = AutoModel.from_pretrained( ... "stabilityai/stable-diffusion-xl-base-1.0", subfolder="unet", device_map=0 ... ) >>> # Specifying a supported offloading strategy like "auto" also works. >>> model = AutoModel.from_pretrained( ... "stabilityai/stable-diffusion-xl-base-1.0", subfolder="unet", device_map="auto" ... ) >>> # Specifying a dictionary as `device_map` also works. >>> model = AutoModel.from_pretrained( ... "stabilityai/stable-diffusion-xl-base-1.0", ... subfolder="unet", ... device_map={"": torch.device("cuda")}, ... ) ``` Set `device_map="auto"` to have 🤗 Accelerate automatically compute the most optimized `device_map`. For more information about each option see [designing a device map](https://huggingface.co/docs/accelerate/en/concept_guides/big_model_inference#the-devicemap). You can also refer to the [Diffusers-specific documentation](https://huggingface.co/docs/diffusers/main/en/training/distributed_inference#model-sharding) for more concrete examples. max_memory (`Dict`, *optional*): A dictionary device identifier for the maximum memory. Will default to the maximum memory available for each GPU and the available CPU RAM if unset. offload_folder (`str` or `os.PathLike`, *optional*): The path to offload weights if `device_map` contains the value `"disk"`. offload_state_dict (`bool`, *optional*): If `True`, temporarily offloads the CPU state dict to the hard drive to avoid running out of CPU RAM if the weight of the CPU state dict + the biggest shard of the checkpoint does not fit. Defaults to `True` when there is some disk offload. low_cpu_mem_usage (`bool`, *optional*, defaults to `True` if torch version >= 1.9.0 else `False`): Speed up model loading only loading the pretrained weights and not initializing the weights. This also tries to not use more than 1x model size in CPU memory (including peak memory) while loading the model. Only supported for PyTorch >= 1.9.0. If you are using an older version of PyTorch, setting this argument to `True` will raise an error. variant (`str`, *optional*): Load weights from a specified `variant` filename such as `"fp16"` or `"ema"`. use_safetensors (`bool`, *optional*, defaults to `None`): If set to `None`, the `safetensors` weights are downloaded if they're available **and** if the `safetensors` library is installed. If set to `True`, the model is forcibly loaded from `safetensors` weights. If set to `False`, `safetensors` weights are not loaded. disable_mmap ('bool', *optional*, defaults to 'False'): Whether to disable mmap when loading a Safetensors model. This option can perform better when the model is on a network mount or hard drive, which may not handle the seeky-ness of mmap very well. use_flashpack (`bool`, *optional*, defaults to `False`): If set to `True`, the model is loaded from `flashpack` weights. flashpack_kwargs(`dict[str, Any]`, *optional*, defaults to `{}`): Kwargs passed to [`flashpack.deserialization.assign_from_file`](https://github.com/fal-ai/flashpack/blob/f1aa91c5cd9532a3dbf5bcc707ab9b01c274b76c/src/flashpack/deserialization.py#L408-L422) > [!TIP] > To use private or [gated models](https://huggingface.co/docs/hub/models-gated#gated-models), log-in with `hf > auth login`. You can also activate the special > ["offline-mode"](https://huggingface.co/diffusers/installation.html#offline-mode) to use this method in a > firewalled environment. Example: ```py from diffusers import UNet2DConditionModel unet = UNet2DConditionModel.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", subfolder="unet") ``` If you get the error message below, you need to finetune the weights for your downstream task: ```bash Some weights of UNet2DConditionModel were not initialized from the model checkpoint at stable-diffusion-v1-5/stable-diffusion-v1-5 and are newly initialized because the shapes did not match: - conv_in.weight: found shape torch.Size([320, 4, 3, 3]) in the checkpoint and torch.Size([320, 9, 3, 3]) in the model instantiated You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. ``` cache_dirNignore_mismatched_sizesFforce_downloadproxiesoutput_loading_infolocal_files_onlyrxrevision torch_dtyper subfolder device_map max_memoryoffload_folderoffload_state_dictr<rmuse_safetensorsr dduf_entries disable_mmapparallel_configrpflashpack_kwargszEParallel loading is not supported when not using `low_cpu_mem_usage`.zPassed `torch_dtype` z7 is not a `torch.dtype`. Defaulting to `torch.float32`.Ta,Cannot initialize model with low cpu memory usage because `accelerate` was not found in the environment. Defaulting to `low_cpu_mem_usage=False`. It is strongly recommended to install `accelerate` for faster and less memory-intense model loading. You can do so with: ``` pip install accelerate ``` .zLoading and dispatching requires `accelerate`. Please make sure to install accelerate or set `device_map=None`. You can install accelerate with `pip install accelerate`.rhriztLoading and dispatching requires torch >= 1.9.0. Please either update your PyTorch version or set `device_map=None`.z~Low memory initialization requires torch >= 1.9.0. Please either update your PyTorch version or set `low_cpu_mem_usage=False`.zEYou cannot set `low_cpu_mem_usage` to `False` while using device_map=zO for loading and dispatching. Please make sure to set `low_cpu_mem_usage=True`.r)autobalancedbalanced_low_0 sequentialzWhen passing device_map as a string, the value needs to be a device name (e.g. cpu, cuda:0) or 'auto', 'balanced', 'balanced_low_0', 'sequential' but found rrznYou can't pass device_map as a negative int. If you want to put the model on the cpu, pass device_map = 'cpu' z>Passing along a `device_map` requires `low_cpu_mem_usage=True`z1.10z=`low_cpu_mem_usage` and `device_map` require PyTorch >= 1.10.modelpytorch) diffusers file_type framework model_class) rreturn_unused_kwargsreturn_commit_hashrrrrxrr user_agentr) pre_quantized)rrquant)r quant_configz>Set `low_cpu_mem_usage` to True as `hf_quantizer` is not None.zD`low_cpu_mem_usage` cannot be False or None when using quantization.use_keep_in_fp32_moduleszGSet `low_cpu_mem_usage` to True as `_keep_in_fp32_modules` is not None.zH`low_cpu_mem_usage` cannot be False when `keep_in_fp32_modules` is True.)is_localrrrrrmrrrrxrr commit_hashr)rrrrxrrrr) rrrrrrxrrrrrz(An error occurred while trying to fetch z: zXDefaulting to unsafe serialization. Pass `allow_pickle=False` to raise an error instead. float8_e4m3fnzA needs to be of type `torch.dtype`, e.g. `torch.float16`, but is zLoading a FlashPack checkpoint in PyTorch, requires both PyTorch and flashpack to be installed. Please see https://pytorch.org/ and https://github.com/fal-ai/flashpack for installation instructions.zMPlease install torch and flashpack to load a FlashPack checkpoint in PyTorch.z`device_map` has not been provided for FlashPack, model will be on `cpu` - provide `device_map` to fully utilize the benefit of FlashPack.r-zFlashPack `device_map` should not be one of `auto`, `balanced`, `balanced_low_0`, `sequential`. Use a specific device instead, e.g., `device_map='cuda'` or `device_map='cuda:0')rrrz6`output_loading_info` is not supported with FlashPack.)rrall_checkpoint_keys)rrkeep_in_fp32_modules)r) rr<rrrrrrrris_parallel_loading_enabledr) missing_keysunexpected_keysmismatched_keys error_msgs)r offload_dir offload_index)r_pre_quantization_dtype)r)configrY)Sr_LOW_CPU_MEM_USAGE_DEFAULTrNotImplementedErrorrrvrfloat32r'r>r#r(rrr RuntimeErrorintrrR load_configcopydeepcopyrmerge_quantization_configs from_configvalidate_environmentupdate_torch_dtypeupdate_device_maprrrYrrto_dictr"_MAX_QUANT_CONFIG_USER_AGENT_CHARSr(rr=rWrrisdirr7existsr8is_filer rrrr!IOErrorrrr_set_default_torch_dtyperappend accelerateinit_empty_weightsr=r&rrmixinassign_from_fileset_default_dtyper;_fix_state_dict_keys_on_loadrmaybe_update_loaded_keyspreprocess_modelsupports_parallel_loadingr5_load_pretrained_modelrjpostprocess_modelrrtoregister_to_configevalenable_parallelism)?clsrrPrrrrrrrxrrrrrrrrr<rmrrrrrrprr allow_pickler unused_kwargs config_pathrrrrrserialized_quant_configrrrresolved_model_filesharded_metadata index_filerindex_file_kwargsre dtype_orig init_contextsrrflashpack_devicerr loaded_keyscheckpoint_filesrrrrr loading_infodevice_map_kwargss? rDfrom_pretrainedModelMixin.from_pretrainedts LJJ{D1 "(**-F"N$4e<**Y-$jj)>F!::&8$? 7D)::j$/jj5  7D)$0ek JJ{D1 ZZ d3 ZZ d3 $4d;#ZZ(E4:JJ~t4T zz.%8 IOTegkIl ?E: !::&8"=&@# &/@%&mn n  ":k5;;+O+O--K NN' }4kl   ""OL %<%>%> %  NN.   !*A*C*C%`   !*:4*I*I%&   $-=dG-L-L%.   %**@WXbWcdRR  j%,, / /j)J  C ( (Z?s-s  %,,z":;  C ( (A~ E!*-  ! ($(!& !abb %.>tV.L.L !`aa% "s||,   4 .1__ . !%#)-!%. . * { v&.7eFCX*!($(! \]& !ghh$'#<#? -J"b.", 0 $&(  ';):;   J$6bggnnZ>X>X1F4EFJ  !|z7I7I7K7KJ 4O-#!1%!#/r) 5 1 !15  +,DgN # '*95%1"+'5 ')9#!)"+#-$/%1 +'.  &z"1-),@#-!1!#%') # -t44#6"7    ";'%Z^:_+_k5;;77 "m#deijuevdwwxy55kBJ(*+   !>!>!@ A ] +OOFSSKJn [USS5[R:Xax[USS5(a*[ SS5(a[ SURS 35e[US S5(a*[ SS 5(a[ S URS 35eU"U5(a/[RSURRS35 U$[TU]4"U0UD6$)Nr_is_group_offload_enabledquantization_methodis_loaded_in_8bitF<0.48.0zCalling `cuda()` is not supported for `8-bit` quantized models with the installed version of bitsandbytes. The current device is `L`. If you intended to move the model, please install bitsandbytes >= 0.48.0.is_loaded_in_4bit0.43.2zCalling `cuda()` is not supported for `4-bit` quantized models with the installed version of bitsandbytes. The current device is `L`. If you intended to move the model, please install bitsandbytes >= 0.43.2. The module 'zD' is group offloaded and moving it using `.cuda()` is not supported.)rr9r=rBITS_AND_BYTESr%rrr'r>rrRrcuda)rCrOrPr9rs rDrDModelMixin.cudasF 4. 59K9Z9Z Zt0%88=TUXZb=c=c ..2kk];GH2E::?VWZ\d?e?e ..2kk];GH %T * * NNt~~6677{| Kw|T,V,,rGc >SSKJn UR=(d /n[SU55=(d SU;nSU;nUH3n[ U[ 5(dM[ R"U5 SnM5 U(d,UH&n[ U[ R5(dM$Sn O U(a3Ub0[RSURRSUS 35 [US S 5(aU(a [S 5e[US S5[ R":Xax[USS 5(a*[%SS5(a[SURS35e[USS 5(a*[%SS5(a[SURS35eU"U5(a6U(a/[RSURRS35 U$[&TU]P"U0UD6$![a GMf=f)Nrr8c3V# UHn[U[R5v M! g7frA)rrvr)rargs rDr ModelMixin.to..s )XSWC*S%,,*G*GSWs')rrTzThere are modules in z! that should be kept in float32: z. Casting directly with `to()` can lead to inconsistent results; set `torch_dtype` in `from_pretrained()` instead to keep these modules in float32. is_quantizedFzCasting a quantized model to a new `dtype` is unsupported. To set the dtype of unquantized layers, please use the `torch_dtype` argument when loading the model using `from_pretrained` or `from_single_file`r:r;r<r=zCalling `to()` is not supported for `8-bit` quantized models with the installed version of bitsandbytes. The current device is `r>r?r@zCalling `to()` is not supported for `4-bit` quantized models with the installed version of bitsandbytes. The current device is `rArBzB' is group offloaded and moving it using `.to()` is not supported.)rr9rrrrrvrrrr'r>rrRr=rrrCr%rr ) rCrOrPr9 fp32_modulesdevice_arg_or_kwarg_presentdtype_present_in_argsrHrs rDr  ModelMixin.tos F117R &))XSW)X&X&n\dhn\n# '6 1Cc3''  S!.2+ %c5;;//,0) !\%= NN'(?(?'@@abnaopCD  4 / /$ z 4. 59K9Z9Z Zt0%88=TUXZb=c=c ..2kk];GH2E::?VWZ\d?e?e ..2kk];GH %T * */J NNt~~6677yz Kwz4*6**M   sG44 HHcX>[USS5(a [S5e[TU] "U6$)NrJFz`.half()` is not supported for quantized model. Please use the model as it is, since the model has already been cast to the correct `dtype`.)r=rrhalfrCrOrs rDrPModelMixin.halfs6 4 / /G  7<& &rGcX>[USS5(a [S5e[TU] "U6$)NrJFz`.float()` is not supported for quantized model. Please use the model as it is, since the model has already been cast to the correct `dtype`.)r=rrfloatrQs rDrTModelMixin.floats6 4 / /G  7=$' 'rGc0[USS5nU(d#[SURRS35eSnUR 5H3nURRU;dMUR "U0UD6 SnM5 U(d[SUS35eg) u Compiles *only* the frequently repeated sub-modules of a model (e.g. the Transformer layers) instead of compiling the entire model. This technique—often called **regional compilation** (see the PyTorch recipe https://docs.pytorch.org/tutorials/recipes/regional_compilation.html) can reduce end-to-end compile time substantially, while preserving the runtime speed-ups you would expect from a full `torch.compile`. The set of sub-modules to compile is discovered by the presence of **`_repeated_blocks`** attribute in the model definition. Define this attribute on your model subclass as a list/tuple of class names (strings). Every module whose class name matches will be compiled. Once discovered, each matching sub-module is compiled by calling `submodule.compile(*args, **kwargs)`. Any positional or keyword arguments you supply to `compile_repeated_blocks` are forwarded verbatim to `torch.compile`. _repeated_blocksNzM`_repeated_blocks` attribute is empty. Set `_repeated_blocks` for the class `z&` to benefit from faster compilation. FTz$Regional compilation failed because z% classes are not found in the model. )r=rrrRrcompile)rCrOrPrepeated_blockshas_compiled_regionsubmods rDcompile_repeated_blocks"ModelMixin.compile_repeated_blocks&s"$(:DA99=9P9P8QQwy $llnF((O;//&*#% #66GGlm #rG)cp_planrr^c [RS5 [RR 5(d.[RR 5(d [ S5eSSKJn SSK J n SSK J nJ n SSKJnJn [#U[$5(a ['US 9nO[#U[(5(a ['US 9n[RR+5n [RR-5n [R.R15R2n [R4"U 5n [R6"XU R95-5n XxU4nUR:bUR=5Hn[#X5(dMUR>nUb[AUS 5(dM7URBnUcURE5unnOU"U5nURGU5(dH[IURJ5n[MS URNRPS URRSUS35e O SnUR:b]UR:nURT=(d= [RRVRYU URZUR\S9nO`UR^bSUR^nURT=(d4 [RRVRYU UR`4SS9nURcXU US9 Xl2UR:bUR=5H>n[#X5(dMUR>nUb[AUS5(dM7UUl2M@ UcURfc [MS5eUbUO URfnU"XR:U5 UR^bURhc#[MSURNRPS35eUR^Rjn[mURnSS5nUbUU-S:wa[MSUSUS35eSSK8J9n U"XR^URh5 gg)Nz`enable_parallelism` is an experimental feature. The API may change in the future and breaking changes may be introduced at any time without warning.zXtorch.distributed must be available and initialized before calling `enable_parallelism`.r)apply_context_parallelr/rE)rGrHrK)context_parallel_config)tensor_parallel_configrQzrv distributed is_availableis_initializedrhooks.context_parallelr`rRrFrSrGrHrTrLrMrr0r2r3get_rankget_world_size_C_get_acceleratorrget_device_moduler device_countrarrUrrQget_active_backendr[r\r]rrrRrYrg device_meshinit_device_meshrdrerb tp_degreesetuprO_cp_plan_tp_plan _tp_degreer=rhooks.tensor_parallelri)rCrr^r`rFrGrHrLrMrank world_sizerc device_modulerr_rqrUattention_backend_rcrg cp_config tp_configrw num_headsris rDr#ModelMixin.enable_parallelismGs  d   --//8I8I8X8X8Z8Zj  D3WB f3 4 4#FCF  4 5 5#6BF  ))+&&557 hh//166 // < k-2L2L2N+NO&8LM  ) ) 5,,.!&<<",, $GI?S,T,T$-$@$@!$,+D+W+W+Y(%q(<=N(O%0OOPabb*01J1e1e*f'$VW`WjWjWsWsVtu-->-D-D,EFFFYEZ[@A3)6  ) ) 566I>>U%6%6%B%B%S%S'$//(77&T&D  * * 655I>>U%6%6%B%B%S%S'%//1&&T&D  TvD 9 &  ) ) 5,,.!&<<",, $GI?Q,R,R-3 * )4==#8 r")!4g$--G "4)G)G Q  ( ( 4}}$ //00FH55@@I -BDII$Y)>!)C = ;hirhssu!vww E !$(E(Et}} U 5rGrr)r1rassign_to_params_buffersrrr<rrrrrrrrcUR5n[UR55n[[U5[U5- 5nUbUR UUSS9n[[U5[U5- 5nUR b?UR H/nUVs/sHn[ R"UU5bMUPM! nnM1 /n/nU b<SU R5;a(Uc [S5e[R"USS9 U cSn U b[U U5n[UUX5 U bSU R5;a0OSnSunnU (a[R"5n0nUbU/n[ R""U(a[$O[&40SU_S U_S U _S U _S U_S U _SU_SU_SU_SU_SU_SU_SU_SU_SU _SU_6nU(aU"U5unnnn UU - nUU- nO`Un![)U5S:a/SS0n"[+5(dSU"S'[,R."U40U"D6n!U!Hn#U"U#5unnnn UU - nUU- nM [15 UbG[)U5S:a8[3UU5 SnU (a#[5UUU5 [6R8"U5 [)U5S:aASR;U5n$SU$;aU$S- n$[=S UR>R@S!U$35e[)U5S:a9[BRES"US#UR@S$S%R;U5/35 O-[BRGS&UR>R@S'35 [)U5S:a4[BRES(UR>R@S)US*US+35 OV[)U5S:XaG[BRGS,UR>R@S-US.UR>R@S/35 [)U5S:agS0R;UV%V&V's/sHun%n&n'S1U%S2U&S3U'S43PM sn'n&n%5n([BRES(UR>R@S)US5U(S+35 UUUUUU4$s snfs sn'n&n%f)6Nr)prefixdiskzThe current `device_map` had weights offloaded to the disk. Please provide an `offload_folder` for them. Alternatively, make sure you have `safetensors` installed if the model you are using offers the weights in this format.Trs)NNrmodel_state_dictrrrrrrr1rrrstate_dict_indexstate_dict_folderrr<rr/desczLoading checkpoint shardsdisablerz z size mismatchz_ You may consider adding `ignore_mismatched_sizes=True` in the model `from_pretrained` method.z#Error(s) in loading state_dict for z: z(Some weights of the model checkpoint at z! were not used when initializing z: rPz9All model checkpoint weights were used when initializing z. zSome weights of z3 were not initialized from the model checkpoint at z and are newly initialized: zo You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.zAll the weights of z/ were initialized from the model checkpoint at zf. If your task is similar to the task the model of the checkpoint was trained on, you can already use z* for predictions without further training.rz- z: found shape z in the checkpoint and z in the model instantiatedz= and are newly initialized because the shapes did not match: )$rrWrr"update_missing_keys"_keys_to_ignore_on_load_unexpectedresearchrXrrrr6r4tempfilemkdtemp functoolspartialr:r9rr*r)tqdmr.rlrkshutilrmtreerZrrrRr'r>r())r$rrr)rr1rrrrr<rrrrrrrrr expected_keysrrpatryrrexpanded_device_maprrrload_fn_mismatched_keys _error_msgs shard_filesshard_tqdm_kwargs shard_file error_msgkeyshape1shape2mismatched_warnings) rDr!ModelMixin._load_pretrained_models*!++--2245 C .[1AAB  #';;E   !f 0A0A0C&C% :  NT:!)%)"  !"4Z"O  %e-@% V(4:CTCTCV9V\` .8++  ( 0 0 2 !   !$., ##1L -Rb  . "    &  "6 & $ , ( * . 0 %< 0! "&# ( 'MTUhMi JM+-={ + %J / /O-K&'!+%+-H$I!.0037%i0%ll+>TBST ) QXYcQdN /1A;k) #33*   $]);a)? }n = M!&u.>@QR /0 z?Q  J/I)+w !DU__E]E]D^^cdmcnop p  ! # NN:;X:YYz{~|H|H{IINPTPYPYZiPjOkNlm  KKSTYTcTcTlTlSmmpq r | q NN"5??#;#;"<=122N|n]nn   !Q & KK%eoo&>&>%?@123CCH??C[C[B\]--   ! #!%0?/>+VV^F83J6(Rlm/>"  NN"5??#;#;"<=123./0AA lO_mU___C#\jsQ2>Q2Q7 c[R"UR5RnUR 5VVs0sH'up4UR [R :XdM%X4_M) nnn[UR 5VVs1sH'up4UR [R :wdM%UiM) snn5n[UR55S1- nXv4$s snnfs snnf)NrC) inspect signaturerErrudefault_emptyr"r)r$objrryrzrequired_parametersoptional_parametersexpected_moduless rD_get_signature_keysModelMixin._get_signature_keys\s&&s||4?? 0:0@0@0Bb0BaiiSZSaSaFatqt0Bb!1A1A1C"c1CqyyT[TbTbGb11C"cd2779:fXE44 c"cs$C&C$C# /C# c[5nU/n[U5S:aURS5nURRU;a{[ U[ 5(aJURc%[URRSUS35eU[UR5-nU[UR55- n[U5S:aM[U5$)aZ Get the modules of the model that should not be split when using device_map. We iterate through the modules to get the underlying `_no_split_modules`. Args: device_map (`str`): The device map value. Options are ["auto", "balanced", "balanced_low_0", "sequential"] Returns: `list[str]`: list of modules that should not be split rrzz does not support `device_map='z_'`. To implement support, the model class needs to implement the `_no_split_modules` attribute.) r"rrrrRrr_no_split_modulesrrWr)rCrrmodules_to_checkrqs rD_get_no_split_modules ModelMixin._get_no_split_modulesfs E 6"#a'%))"-F((0AAfj11//7(%//8899XYcXdeZZ ->FD\D\@],]) D):$;; "#a'%&&rGcUR(d[SURSUS35e[R SURSUS35 [ R "5n[ R"U5 U$)a7 Change the default dtype and return the previous one. This is needed when wanting to instantiate the model under specific dtype. Args: dtype (`torch.dtype`): a floating dtype to set to. Returns: `torch.dtype`: the original `dtype` that can be used to restore `torch.set_default_dtype(dtype)` if it was modified. If it wasn't, returns `None`. Note `set_default_dtype` currently only works with floating-point types and asserts if for example, `torch.int64` is passed. So if a non-float `dtype` is passed this functions will throw an exception. zCan't instantiate z model under dtype=z' since it is not a floating point dtypezInstantiating z model under default dtype r)rrrRr'r(rvget_default_dtyper)r$rr.s rDr#ModelMixin._set_default_torch_dtypesw"&&$S\\N2EeWLst   nS\\N2MeWTUVW,,.  &rGc[U5$)zu `torch.device`: The device on which the module is (assuming that all the module parameters are on the same device). )rrs rDrModelMixin.devices $D))rGc[U5$)zg `torch.dtype`: The dtype of the module (assuming that all the module parameters have the same dtype). )rrs rDrModelMixin.dtypes #4((rGonly_trainableexclude_embeddingsc[USS5nU(a[5(aSSKnO [S5eU(atUR 5VVs/sH+upV[ U[ R5(dM&US3PM- nnnUR5VVs/sHupXXW;dM UPM n nnO[UR55n /n U Hn U R(d U(aMU(a[ U WR R5(as[U S5(aU R5n O*[U S5(aU RR n OS n U R#U R%5S -U -5 MU R#U R%55 M ['U 5$s snnfs snnf) a Get number of (trainable or non-embedding) parameters in the module. Args: only_trainable (`bool`, *optional*, defaults to `False`): Whether or not to return only the number of trainable parameters. exclude_embeddings (`bool`, *optional*, defaults to `False`): Whether or not to return only the number of non-embedding parameters. Returns: `int`: The number of parameters. Example: ```py from diffusers import UNet2DConditionModel model_id = "stable-diffusion-v1-5/stable-diffusion-v1-5" unet = UNet2DConditionModel.from_pretrained(model_id, subfolder="unet") unet.num_parameters(only_trainable=True) 859520964 ``` r?FrNzbitsandbytes is not installed but it seems that the model has been loaded in 4bit precision, something went wrong make sure to install bitsandbytes with `pip install bitsandbytes`. You also need a GPU. z.weight element_size quant_storager/r)r=r$ bitsandbytesrrrr EmbeddingrrWr requires_grad Params4bitrrritemsizernumelsum) rCrrr?bnbr module_typeembedding_param_namesrmtotal_parameters total_numelr num_bytess rDnum_parametersModelMixin.num_parameterss0$D*=uE (*** p :>:L:L:N%:N%6TR\]hjljvjvRw 4& :N "%261F1F1H 1HodDLm 1H    $DOO$56  %E""..%E366;L;L)M)Mun55$)$6$6$8  88$)$7$7$@$@ $% &&u{{}q'89'DE&&u{{}5&;5% s%F67 F6 F<%F<cP[UR5Vs/sH#o"R5UR5-PM% sn5nU(aL[UR 5Vs/sH#oDR5UR5-PM% sn5nX5-nU$s snfs snf)a Get the memory footprint of a model. This will return the memory footprint of the current model in bytes. Useful to benchmark the memory footprint of the current model and design some tests. Solution inspired from the PyTorch discussions: https://discuss.pytorch.org/t/gpu-memory-that-model-uses/56822/2 Arguments: return_buffers (`bool`, *optional*, defaults to `True`): Whether to return the size of the buffer tensors in the computation of the memory footprint. Buffers are tensors that do not require gradients and not registered as parameters. E.g. mean and std in batch norm layers. Please see: https://discuss.pytorch.org/t/what-pytorch-means-by-buffers/120266/2 )rrnelementrr)rCreturn_buffersrmembufmem_bufss rDget_memory_footprintModelMixin.get_memory_footprintsHYZHYu>>#e&8&8&::HYZ[ 4<<>Z>CLLNS-=-=-??>Z[H.C [Zs *B'*B#rcSnUR5HBupE[US5(dM[RSUSUS35 X%lXlSnMD U(d#[ SURRS35eg) NFrz Setting `gradient_checkpointing=z` for ''Tz The module z does not support gradient checkpointing. Please make sure to use a module that supports gradient checkpointing by creating a boolean attribute `gradient_checkpointing`.) rrr'debugrrrrrR)rCrris_gradient_checkpointing_setrrqs rDr&ModelMixin._set_gradient_checkpointings).% ..0LDv788 ?xwtfTUVW6Q306-04- 1-dnn5567~ -rGch^^/mUU4SjmT"SU5 TGHnUS3U;aURUS35XS3'US3U;aURUS35XS3'US3U;aURUS35XS3'US 3U;aURUS 35XS 3'US 3U;aURUS 35XS 3'US 3U;aURUS 35XS3'US3U;aURUS35XS3'US3U;dMURUS35XS3'GM U$)a# This function fix the state dict of the model to take into account some changes that were made in the model architecture: - deprecated attention blocks (happened before we introduced sharded checkpoint, so this is why we apply this method only when loading non sharded checkpoints for now) c>[US5(a"UR(aTRU5 UR5Hup#US:XaUOUSU3nT"X#5 M g)N_from_deprecated_attn_blockrr)rrrnamed_children)rrqsub_name sub_module deprecated_attention_block_pathsrecursive_find_attn_blocks rDrJModelMixin._fix_state_dict_keys_on_load..recursive_find_attn_blocks^v<==&BdBd077=(.(=(=(?$'+rz8$q 7K)(?)@rGrz .query.weightz .to_q.weightz .query.biasz .to_q.biasz .key.weightz .to_k.weightz .key.biasz .to_k.biasz .value.weightz .to_v.weightz .value.biasz .to_v.biasz.proj_attn.weightz.to_out.0.weightz.proj_attn.biasz.to_out.0.bias)r)rCrrrrs @@rDr'ModelMixin._fix_state_dict_keys_on_loads,.( @ ""d+ 5D}%34>NNdV=CY4Z V<01{#z12<..D6AU2V V:./{#z14>NNdV;CW4X V<01y!Z/2<..D6AS2T V:./}%34>NNdV=CY4Z V<01{#z12<..D6AU2V V:./()Z78B$O`Ga8b V#345':56@nnv_E]6^ V>23354rG)rrOrA)rnN)TNTN10GBFF) FFNTNNNNNNFF)FF)T)WrRrSrTrUrVr config_name_automatically_saved_argsrrrrr!r?rWrOryrz _skip_keysrErrrpropertyboolrr rrrrrrrrr rrrvrrrr rrr+rrrArdrgrPathLikerr classmethodrrr5rrDr rPrTr\r2r0r3rr1r#rrWrr rrrrrrrrrrrX __classcell__rs@rDrrsK V',$)-& '+$!%HHJ1 )))$n4nn oTXodho<;==$= /0PT9'+9=E_9 9"IDI 0 1 1U]`dUd 1pt 1MxRVMbfM@@&+%8%8,07;IM" S {{S {{T)S $CHo4 S $D$93$>?$F S  S  S p(-||E':)+/" #+/$(%)B ||B  B  B "Dj B  B B B "DjB TzB d B  B H<>S<>T<>|0,!%)-#'"$*!#bkk) $  !  t c B -D C"++*T1!_`"&*D[#_`$%_`_`B55'':U[[U[[4* **)u{{)) > T> t> `c> @&"5;;KaKaKlKl@H $1{1t11rGrch^\rSrSrSr\\S\\R-S-4U4Sjj55r Sr U=r $)LegacyModelMixiniEz A subclass of `ModelMixin` to resolve class mapping from legacy classes (like `Transformer2DModel`) to more pipeline-specific classes (like `DiTTransformer2DModel`). rNc >SSKJn UR5nURSS5nURSS5nURSS5nURSS5nURSS5n URS S5n URS S5n Un [S S S .n UR "U 4USSUUUU U U U S. UD6un nU"X5nUULa[ [U]""U40UD6$UR"U40UD6$)Nr/)_fetch_remapped_cls_from_configrrFrrrxrrrr)rrrT) rrrrrrrxrrr) model_loading_utilsrrrrrrrr5)r$rrPr kwargs_copyrrrrrxrrr'rrrremapped_classrs rDr5 LegacyModelMixin.from_pretrainedKs6 Ikkm JJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 4 % "    !%#)-!    19E S )>J-1< "112O_S^_ _rGrY) rRrSrTrUrVrrrrrr5rXrrs@rDrrEs= /`C"++r s+"  #066 VV6"C?(>\\2   --$   H %zz12 &*"  wwWW**""ww..gg,,00ww..wwggnngg,,WW**ww..gg,,"D'""!%!&)K%EHHOO% %44#588??4#u{{4#n44&Q.Qh:7`z7`rG