jsSSKrSSKJr SSKJr SSKJr SSKJ r J r SSK J r SSK Jr SSKJrJrJr S S KJrJrJrJr "S S \5r\"S S\55r"SS\\5r\"SS9"SS\\55r/SQrg)N) IJepaConfig)initialization)BaseModelOutputWithPoolingImageClassifierOutput)PreTrainedModel)Unpack)TransformersKwargsauto_docstring torch_int) ViTEmbeddingsViTForImageClassificationViTModelViTPreTrainedModelc ^\rSrSrSS\S\SS4U4SjjjrS\RS\ S \ S\R4S jr SS \RS \RS-S \S\R4Sjjr Sr U=r$)IJepaEmbeddingsconfiguse_mask_tokenreturnNc>[TU]X5 U?URRn[ R "[R"SX1R55Ul g)N) super__init__ cls_tokenpatch_embeddings num_patchesnn Parametertorchrandn hidden_sizeposition_embeddings)selfrrr __class__s ڵC:\Users\johnn\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\transformers/models/ijepa/modular_ijepa.pyrIJepaEmbeddings.__init__sH 0 N++77 #%<< A{L^L^0_#` embeddingsheightwidthc,URSnURRSn[RR 5(dXE:XaX#:Xa UR$URnURSnX R -nX0R -n [ US-5n URSXU5nURSSSS5n[RRUX4SSS 9nURSSSS5RSSU5nU$) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rg?rrr bicubicF)sizemode align_corners) shaper$r!jit is_tracing patch_sizer reshapepermuter functional interpolateview) r%r*r+r,r num_positionspatch_pos_embeddim new_height new_widthsqrt_num_positionss r'interpolate_pos_encoding(IJepaEmbeddings.interpolate_pos_encodings!&&q) 0066q9 yy##%%+*F6?++ +22r". __, &}c'9:)11!5G]`a)11!Q1=--33 ( 4 *11!Q1=BB1b#Nr) pixel_valuesbool_masked_posrBc URupEpgURU5nUbXURSn URRXIS5n UR S5R U 5n USU - -X--nU(aXR XU5-nOdX`RS:wdXpRS:wa2[SUSUSURSSURSS3 5eXR-nURU5nU$) Nrr.g?rzInput image size (*z) doesn't match model (z).) r3r mask_tokenexpand unsqueezetype_asrB image_size ValueErrorr$dropout) r%rDrErB batch_size_r+r,r* seq_length mask_tokensmasks r'forwardIJepaEmbeddings.forward=s# (4'9'9$ v**<8  &#))!,J//00LK",,R088ED#sTz2[5GGJ $#&C&CJX]&^^J++u8J/J (%9+,Adooa.@-AE$&>&>>J\\*- r))r$)F)NF)__name__ __module__ __qualname____firstlineno__rboolrr!TensorintrB BoolTensorrT__static_attributes__ __classcell__r&s@r'rrsa{aDaTaa%5<<%%UX%]b]i]i%T48). ll))D0#'   r)rcB\rSrSr\R "5S5rSrg)IJepaPreTrainedModel]cD[R"X5 [U[R[R 45(ac[ R"URSURRS9 URb![ R"UR5 gg[U[5(ac[ R"URSURRS9 URb![ R"UR5 ggg)zInitialize the weightsg)meanstdN)r _init_weights isinstancerLinearConv2dinit trunc_normal_weightrinitializer_rangebiaszeros_rr$rH)r%modules r'rg"IJepaPreTrainedModel._init_weights_s %%d3 fryy"))4 5 5   v}}3DKK[TU]U5 Xl[XS9Ulg)z add_pooling_layer (bool, *optional*, defaults to `True`): Whether to add a pooling layer use_mask_token (`bool`, *optional*, defaults to `False`): Whether to use a mask token for masked image modeling. )rN)rrrrr*)r%rrxrr&s r'rIJepaModel.__init__ns#   )&Pr))rr*)FF) rVrWrXrYrrZrr^r_r`s@r'rvrvms( Q{ Qt Q]a Q Qr)rva IJepa Model transformer with an image classification head on top (a linear layer on top of the final hidden states) e.g. for ImageNet. Note that it's possible to fine-tune IJepa on higher resolution images than the ones it has been trained on, by setting `interpolate_pos_encoding` to `True` in the forward of the model. This will interpolate the pre-trained position embeddings to the higher resolution. ) custom_introc ^\rSrSrS\4U4SjjrS S\RS-S\RS-S\S-S\ \ S \ 4 S jjr S r U=r$) IJepaForImageClassificationzrcb>[TU]U5 [USS9UlUR 5 g)NF)rx)rrrvijepa post_init)r%rr&s r'r$IJepaForImageClassification.__init__s(  %@  r)NrDlabelsrBkwargsrc UR"U4SU0UD6nURnURURSS95nSnUbUR"X'UR 40UD6n[ UUURURS9$)ab labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). rBr)r>N)losslogits hidden_states attentions) rlast_hidden_state classifierre loss_functionrrrr) r%rDrrBroutputssequence_outputrrs r'rT#IJepaForImageClassification.forwards/3jj / %=/ /  "33!5!5!!5!<=  %%fdkkLVLD$!//))   r))r)NNN)rVrWrXrYrrr!r[rZr r rrTr^r_r`s@r'r}r}zsp{-1&*04  llT)  t# #'+  +,     r)r})rbrvr})r!torch.nnr-transformers.models.ijepa.configuration_ijeparrrkmodeling_outputsrrmodeling_utilsrprocessing_utilsr utilsr r r vit.modeling_vitrrrrrrbrvr}__all__rsr)r'rs E&Q-&BBeeLmL^ /- / / Q%x Q  % "68Q%  % P r)