jSSKJr SSKrSSKrSSKJr SSKJr SSKJ r SSK J r SSK Jr SSKJr SS KJr SS KJr SS KJr SS KJr SS KJrJrJr SSKJrJrJ r SSK!J"r" SSK#J$r$J%r%J&r& SSK'J(r(J)r) SSK*J+r+J,r, SSK-J.r. \&R^"\05r1"SS\Rd5r3"SS\Rd5r4"SS\5r5"SS\5r6"SS\5r7"SS \Rd5r8"S!S"\Rd5r9SLS#\RdS$\RtS%\RtS&\RtS'\RtS-S(\;S-S)\;S*\"\$4S+jjr<"S,S-\Rd5r="S.S/\Rd5r>"S0S1\5r?"S2S3\Rd5r@"S4S5\Rd5rA"S6S7\5rB"S8S9\Rd5rC\%"S:S;\55rDSMS<\E\F\F4S=\;S>\FS'\RS-S?\FS@\R4 SAjjrI\%"SBSC\D55rJ\%"SDSE9"SFSG\D55rK\%"SHSE9"SISJ\D55rL/SKQrMg)N)CallableN)CrossEntropyLoss)initialization)ACT2FN)filter_output_hidden_states)is_deepspeed_zero3_enabled)is_fsdp_managed_module)create_bidirectional_mask)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputCausalLMOutputSequenceClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel*get_torch_context_manager_or_global_device)Unpack)TransformersKwargsauto_docstringlogging)can_return_tuplemerge_with_config_defaults)OutputRecordercapture_outputs) HubertConfigc.^\rSrSrU4SjrSrSrU=r$)HubertPositionalConvEmbedding0c4>[TU]5 [R"URURUR UR S-UR S9UlSUl[USS5(a'[R"UR5UlGO[RRn[[RRS5(a$[RRRn[5(Ga%SSKnUR"R%URR&SS9 U"URSSS 9UlSSS5 [URS 5(aUURRR&R(nURRR&R*nO,URR,nURR.nUR"R1X5 UR"R1X5 OU"URSSS 9Ul[3UR 5Ul[6UR8Ulg!,(df  GN,=f) N) kernel_sizepaddinggroupsconv_pos_batch_normF weight_normr modifier_rankweight)namedimparametrizations)super__init__nnConv1d hidden_sizenum_conv_pos_embeddingsnum_conv_pos_embedding_groupsconv batch_normgetattr BatchNorm1dutilsr'hasattrr-r deepspeedzeroGatheredParametersr* original0 original1weight_gweight_vregister_external_parameterHubertSamePadLayerr$rfeat_extract_activation activation)selfconfigr'r;r@rA __class__s ڸC:\Users\johnn\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\transformers/models/hubert/modeling_hubert.pyr/&HubertPositionalConvEmbedding.__init__1s II      6622a777    60% 8 8 nnV-?-?@DO((..Krxx00-@@ hh77CC )++ ^^66tyy7G7GWX6Y +DIIH! LDIZ499&899#yy99@@JJH#yy99@@JJH#yy11H#yy11H::4J::4J' aH )&*H*HI  !?!?@ZYs J JcURSS5nURbURU5nURU5nURU5nUR U5nURSS5nU$)Nrr") transposer6r5r$rErF hidden_statess rIforward%HubertPositionalConvEmbedding.forwardVsn%//15 ?? & OOM:M -0  ]3  6 %//15 )rEr6r5r$__name__ __module__ __qualname____firstlineno__r/rO__static_attributes__ __classcell__rHs@rIrr0s#AJ  rQrc.^\rSrSrU4SjrSrSrU=r$)rCbcR>[TU]5 US-S:XaSUlgSUlg)Nr"rr)r.r/num_pad_remove)rFr3rHs rIr/HubertSamePadLayer.__init__cs) #:Q#>!#CarQcXURS:aUSS2SS2SUR*24nU$Nrr]rMs rIrOHubertSamePadLayer.forwardgs6    ")!Q0F43F3F2F0F*FGMrQrarRrYs@rIrCrCbsKrQrCc2^\rSrSrSU4SjjrSrSrU=r$)HubertNoLayerNormConvLayermcb>[TU]5 US:aURUS- OSUlURUUl[ R "URURURUURUURS9Ul [URUl g)Nrrr#stridebias)r.r/conv_dim in_conv_dim out_conv_dimr0r1 conv_kernel conv_stride conv_biasr5rrDrErFrGlayer_idrHs rIr/#HubertNoLayerNormConvLayer.__init__ns [TU]5 US:aURUS- OSUlURUUl[ R "URURURUURUURS9Ul [ R"URSS9Ul [URUlg)NrrrgT)elementwise_affine)r.r/rjrkrlr0r1rmrnror5 LayerNorm layer_normrrDrErps rIr/!HubertLayerNormConvLayer.__init__s [TU]5 US:aURUS- OSUlURUUl[ R "URURURUURUURS9Ul [URUl [ R"URURSS9Ulg)NrrrgT) num_groups num_channelsaffine)r.r/rjrkrlr0r1rmrnror5rrDrE GroupNormr}rps rIr/!HubertGroupNormConvLayer.__init__s [TU]5 URS:Xa@[USS9/[ UR S- 5Vs/sHn[ XS-S9PM sn-nOVURS:Xa-[ UR 5Vs/sH n[XS9PM nnO[SURS35e[R"U5Ul SUl S Ul gs snfs snf) Ngroupr)rqrlayerz`config.feat_extract_norm` is z), but has to be one of ['group', 'layer']FT)r.r/feat_extract_normrrangenum_feat_extract_layersrdrx ValueErrorr0 ModuleList conv_layersgradient_checkpointing_requires_grad)rFrGirrHs rIr/HubertFeatureEncoder.__init__s   # #w .3FQGHLQRXRpRpstRtLuLLuq*6EBLuLK % % 0QVW]WuWuQvwQvA3FGQvKwK01I1I0JJst ==5&+#"Lxs C C%cNUR5H nSUlM SUlg)NF) parameters requires_gradrrFparams rI_freeze_parameters'HubertFeatureEncoder._freeze_parameterss#__&E"'E '#rQcUSS2S4nUR(aUR(aSUlURH nU"U5nM U$)NT)rtrainingrr)rF input_valuesrN conv_layers rIrOHubertFeatureEncoder.forwardsK$QW-    4==*.M '**J&}5M+rQ)rrr) rSrTrUrV__doc__r/rrOrWrXrYs@rIrrs8#"$  rQrc.^\rSrSrU4SjrSrSrU=r$)HubertFeatureProjectioncx>[TU]5 URUlUR(a1[R"UR SUR S9Ul[R"UR SUR5Ul [R"UR5Ul g)Nreps)r.r/feat_proj_layer_normr0r|rjlayer_norm_epsr}Linearr2 projectionDropoutfeat_proj_dropoutdropoutrFrGrHs rIr/ HubertFeatureProjection.__init__s} $*$?$?!  $ $ ll6??2+>FDYDYZDO))FOOB$79K9KLzz&":":; rQcUR(aURU5nURU5nURU5nU$rt)rr}rrrMs rIrOHubertFeatureProjection.forwards;  $ $ OOM:M 6  ]3 rQ)rrr}rrRrYs@rIrrs<rQrmodulequerykeyvalueattention_maskscalingrkwargsc UcURS5S-n[R"XRSS55U-nUbX-n[R R USS9n[R RXURS9n[R"X5n U RSS5R5n X4$)Nrr"rr,)prr) sizetorchmatmulrLr0 functionalsoftmaxrr contiguous) rrrrrrrr attn_weights attn_outputs rIeager_attention_forwardrs**R.D(<<}}Q':;gEL!#4 ==((2(>L==((6??([L,,|3K''1-88:K  $$rQc^\rSrSrSrSS\S\S\S\S\S \S \S-4U4S jjjr SS \ RS \ RS-S\ RS-S\S-S\ \ S\\ R\ RS-44 SjjrSrU=r$)HubertAttentioni z=Multi-headed attention from 'Attention Is All You Need' paperN embed_dim num_headsr is_decoderri is_causalrGc>[TU]5 XlX lX0lX-UlXplUR U-UR:wa[SURSUS35eUR S-UlX@l X`l [R"XUS9Ul [R"XUS9Ul[R"XUS9Ul[R"XUS9Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).r)ri)r.r/rrrhead_dimrGrrrrr0rk_projv_projq_projout_proj) rFrrrrrirrGrHs rIr/HubertAttention.__init__ s "" !.  MMI %$.. 8MdnnM]$YKr3 }}d* $"ii 4@ ii 4@ ii 4@  )TB rQrNkey_value_statesroutput_attentionsrreturnc USLnURSSn/UQSPURP7nURU5RU5R SS5n U(aUOUn /U RSSQSPURP7n UR U 5RU 5R SS5n UR U 5RU 5R SS5n [R"URR[5nU"UU U U U4UR(dSO URURUS.UD6unnUR"/UQSP76R!5nUR#U5nUU4$)z#Input shape: Batch x Time x ChannelNrrr")rrr)shaperrviewrLrrr get_interfacerG_attn_implementationrrrrreshaperr)rFrNrrrris_cross_attention input_shape hidden_shape query_statescurrent_stateskv_shape key_states value_statesattention_interfacerrs rIrOHubertAttention.forward+s.T9$))#2. 88b8$--8 {{=166|DNNqRST -?)]B^))#2.BBDMMB[[055h?II!QO {{>277AKKAqQ (?(M(M KK , ,.E) %8      % $}}C$,,LL/ %  % ! \"));;;;FFH mmK0 L((rQ) rGrrrrrrrrrrr)rFTFN)NNF)rSrTrUrVrintfloatboolrr/rTensorrr tuplerOrWrXrYs@rIrr sG  &*CCC C  C  CCt#CCD15.2). 0)||0) ,,-0) t+ 0)  $; 0)-.0) u||U\\D00 10)0)rQrc.^\rSrSrU4SjrSrSrU=r$)HubertFeedForwardi^c>[TU]5 [R"UR5Ul[R "URUR5Ul [UR[5(a[URUlOURUl[R "URUR5Ul[R"UR 5Ulgrt)r.r/r0ractivation_dropoutintermediate_dropoutrr2intermediate_sizeintermediate_dense isinstance hidden_actstrrintermediate_act_fn output_densehidden_dropoutoutput_dropoutrs rIr/HubertFeedForward.__init___s $&JJv/H/H$I!"$))F,>,>@X@X"Y f'' - -'-f.?.?'@D $'-'8'8D $IIf&>&>@R@RS jj)>)>?rQcURU5nURU5nURU5nURU5nUR U5nU$rt)rrrrrrMs rIrOHubertFeedForward.forwardlsX// > 00? 11-@ ))-8 ++M: rQ)rrrrrrRrYs@rIrr^s @rQrcx^\rSrSrU4SjrS S\R S\R S-S\\4Sjjr Sr U=r $) HubertEncoderLayerivc>[TU]5 [URURUR SUS9Ul[R"UR5Ul [R"URURS9Ul [U5Ul[R"URURS9Ulg)NFrrrrrGr)r.r/rr2num_attention_headsattention_dropout attentionr0rrrr|rr}r feed_forwardfinal_layer_normrs rIr/HubertEncoderLayer.__init__ws (((00,,  zz&"7"78 ,,v'9'9v?T?TU-f5 " V-?-?VEZEZ [rQNrNrrc UnUR"U4SU0UD6upURU5nXA-nURU5nXRU5-nUR U5nU$Nr)rrr}rrrFrNrr attn_residual_s rIrOHubertEncoderLayer.forwardsr & >>-aaZ`a  ]3 %5  6 %(9(9-(HH --m< rQ)rrrrr}rt rSrTrUrVr/rrrrrOrWrXrYs@rIr r vsG \$/3|| t++, rQr cx^\rSrSrU4SjrS S\R S\RS-S\\ 4Sjjr Sr U=r $) HubertEncoderic>[TU]5 Xl[U5Ul[ R "URURS9Ul [ R"UR5Ul [ R"[UR5Vs/sHn[!U5PM sn5UlSUlgs snfNrF)r.r/rGrpos_conv_embedr0r|r2rr}rrrrrnum_hidden_layersr layersrrFrGrrHs rIr/HubertEncoder.__init__s  ;FC,,v'9'9v?T?TUzz&"7"78 mmvOgOgIh$iIhA%7%?Ih$ij &+#%j C NrNrrc bUb4URS5RSSURS5nSX)'[URUUS9nUR U5nXR UR5-nURU5nURU5n[5=(d [U5nURH_n[R"/5nUR=(a XRR :n U (a U(dMTU"U4SU0UD6nMa [#US9$Nrrr"r)rG inputs_embedsrr)last_hidden_state) unsqueezerepeatrr rGrtodevicer}rr r rrrandr layerdropr rFrNrrexpand_attention_maskposition_embeddings synced_gpusrdropout_probabilityskip_the_layers rIrOHubertEncoder.forwards  %$2$<$(>}?S?S(TT  6  ]3 02R6LT6R [[E"'**R. !]]Z/B[[EZEZ/ZN![[ %m ]N ]V\ ] !??rQrGrrr}rrrt) rSrTrUrVr/rtensorrrrrOrWrXrYs@rIrrsK,/3!@||!@ t+!@+, !@!@rQrcJ^\rSrSrU4SjrS\R 4SjrSrU=r $)HubertAttnAdapterLayeric>[TU]5 URUlURUl[ R"UR 5Ul[ R"UR UR5Ul [ R"5Ul [ R"URUR 5Ul g)z Implements adapter modules directly with 3D tensor weight as parameters and without using ModuleList to speed up training throughput. N)r.r/adapter_attn_dim input_dimr2 hidden_dimr0r|normrlinear_1ReLUact_fnlinear_2rs rIr/HubertAttnAdapterLayer.__init__s 00 ,,LL1  $//4>>B ggi  $..$//B rQrNcURU5nURU5nURU5nURU5nU$rt)r<r=r?r@rMs rIrOHubertAttnAdapterLayer.forwards@ -0  m4  M2  m4 rQ)r?r;r:r=r@r<) rSrTrUrVr/r FloatTensorrOrWrXrYs@rIr7r7s  CU%6%6rQr7cx^\rSrSrU4SjrS S\R S\R S-S\\4Sjjr Sr U=r $) !HubertEncoderLayerStableLayerNormic>[TU]5 [URURUR SUS9Ul[R"UR5Ul [R"URURS9Ul [U5Ul[R"URURS9Ul[#USS5b[%U5UlgSUlg)NFr rr9)r.r/rr2r r rr0rrrr|rr}rrrr7r7 adapter_layerrs rIr/*HubertEncoderLayerStableLayerNorm.__init__s (((00,,  zz&"7"78 ,,v'9'9v?T?TU-f5 " V-?-?VEZEZ [ 6-t 4 @!7!?D !%D rQNrNrrc UnURU5nUR"U4SU0UD6upURU5nXA-nXRUR U55-nUR bXR U5-nU$r)r}rrrrrHrs rIrO)HubertEncoderLayerStableLayerNorm.forwards &  6 >>-aaZ`a  ]3 %5 %(9(9$:O:OP]:^(__    )),>,>},MMMrQ)rHrrrrr}rtrrYs@rIrFrFsF&,/3|| t++, rQrFcB^\rSrSrU4SjrSS\\4SjjrSrU=r $)HubertEncoderStableLayerNormic>[TU]5 Xl[U5Ul[ R "URURS9Ul [ R"UR5Ul [ R"[UR5Vs/sHn[!U5PM sn5UlSUlgs snfr)r.r/rGrrr0r|r2rr}rrrrrrrFrrr s rIr/%HubertEncoderStableLayerNorm.__init__s  ;FC,,v'9'9v?T?TUzz&"7"78 mm@EfF^F^@_ `@_1 .v 6@_ `  ',# ar"rc 0Ub4URS5RSSURS5nSX)'[URUUS9nUR U5nX-nUR U5n[5=(d [U5nURH_n[R"/5nUR=(a XRR:n U (a U(dMTU"U4SU0UD6nMa URU5n[US9$r$)r'r(rr rGrrr r rrr+rr,r}rr-s rIrO$HubertEncoderStableLayerNorm.forwards  %$2$<$[TU]U5 [U[R5(Ga,[ 5(aSSKn[US5(ak[US5(aZURRURUR/SS9 [R"UR5 SSS5 OnURRURSS9 [R"UR5 SSS5 O [R"UR5 URb![R "UR5 gg[U["5(a3[US5(a![R$"UR&5 gg[U[(5(aN[US5(a<[R*"UR,SUR.R0S -- 5 ggg!,(df  N=f!,(df  N=f) zInitialize the weightsrNrAr@r(masked_spec_embed layer_weightsg?r)r. _init_weightsrr0r1r r;r:r<r=rAr@initkaiming_normal_r*rizeros_ HubertModeluniform_r\HubertForSequenceClassification constant_r]rGr)rFrr;rHs rIr^#HubertPreTrainedModel._init_weightsHs f% fbii ( ()++ 6:..76:3N3N"::FOOV__;]mn:o,,V]];po#::6==XY:Z,,V]];[Z$$V]]3{{& FKK('  , ,v233 f6674  ? @ @v//v33SDKK._conv_out_lengthgs 99\7wWZ[[ [rQ)ziprGrmrn)rFrgrnr#rhs rI _get_feat_extract_output_lengths6HubertPreTrainedModel._get_feat_extract_output_lengthsbsG  \ $'t{{'>'> @W@W#X K,]PM$YrQfeature_vector_lengthrcURURS55R[R5nUR Sn[R "XA4URURS9nSU[R"UR SURS9US- 4'URS/5RS5RS/5R5nU$)Nrr)dtyper*r)r*) rqsumr)rlongrzerosrur*arangeflipcumsumr)rFrsroutput_lengths batch_sizes rI"_get_feature_vector_attention_mask8HubertPreTrainedModel._get_feature_vector_attention_maskqs>>~?Q?QRT?UVYYZ_ZdZde#))!,   /~7K7KTbTiTi uv ^%9%9!%<^EZEZ[]kno]opq',,bT299"=BBB4HMMOrQ)rSrTrUrVr__annotations__base_model_prefixmain_input_nameinput_modalities_no_split_modulessupports_gradient_checkpointing_supports_flash_attn_supports_sdpa_supports_flex_attnr rFrr_can_record_outputsrno_gradr^ LongTensorrrqr~rWrXrYs@rIrSrS8s  $O-/CD&*#N,.OP$_A)T  ]]_``2 e>N>NQT>T   ]b]m]m  rQrSr mask_prob mask_length min_masksrc ^^^^^UunmTS:a [S5eTT:a[STSTS35e[RRS5R 5mUUUUU4SjnUb-UR 5R S5R5O[U5Vs/sHnTPM snn[R"UT4[S 9n /n U"T5n U S :XaU $UHn U"U 5n [RR[R"U TS- - 5U S S 9n[U5S :XaTS- nOUS n[R"U[R"X- [R S 9U-/5nU R#U5 M [R$"U 5n [R&"U SS2SS2S4X[T45n U R)X[T-5n [R"T5SSSS24n[R&"UX[T45R)X[T-5nU U-n U R+5TS- :a TS- XTS- :'[R,"XSS5 U $s snf) a2 Computes random mask spans for a given shape. Used to implement [SpecAugment: A Simple Data Augmentation Method for ASR](https://huggingface.co/papers/1904.08779). Note that this method is not optimized to run on TPU and should be run on CPU as part of the preprocessing during training. Args: shape: The shape for which to compute masks. This should be of a tuple of size 2 where the first element is the batch size and the second element is the length of the axis to span. mask_prob: The percentage of the whole axis (between 0 and 1) which will be masked. The number of independently generated mask spans of length `mask_length` is computed by `mask_prob*shape[1]/mask_length`. Note that due to overlaps, `mask_prob` is an upper bound and the actual percentage will be smaller. mask_length: size of the mask min_masks: minimum number of masked spans attention_mask: A (right-padded) attention mask which independently shortens the feature axis of each batch dimension. rz&`mask_length` has to be bigger than 0.zO`mask_length` has to be smaller than `sequence_length`, but got `mask_length`: z and `sequence_length`: `c>[TU-T- T-5n[UT5nUT-T:aTT-nUTS- - U:a[UTS- - S5nU$)z;Given input length, compute how many spans should be maskedrr)rmax)rmnum_masked_spanepsilonrrrsequence_lengths rIcompute_num_masked_span6_compute_mask_indices..compute_num_masked_spansoi,6DwNOoy9 [ (? :-[TU]U5 Xl[U5Ul[ U5UlURS:dURS:aG[R"[R"UR5R55UlUR (a[#U5UlO['U5UlUR)5 gNr)r.r/rGrfeature_extractorrfeature_projectionmask_time_probmask_feature_probr0 Parameterrrr2rcr\do_stable_layer_normrMrWr post_initrs rIr/HubertModel.__init__s   !5f!="9&"A  3 &&*B*BS*H%'\\%,,v?Q?Q2R2[2[2]%^D "  & &7?DL(0DL rQNrNmask_time_indicesrc[URSS5(dU$UR5upEnUb(URR UR 5X'OURR S:aUR(a[XE4URR URRUURRS9n[R"X!R[RS9nURR UR 5X'URRS:aUR(a[XF4URRURR URR"S9n[R"XqR[RS9nUSS2S4R%SUS5nSX'U$) z Masks extracted features along time axis and/or along feature axis according to [SpecAugment](https://huggingface.co/papers/1904.08779). apply_spec_augmentTNr)rrrr)r*ru)rrrr)r7rGrr\r)rurrrmask_time_lengthmask_time_min_masksrr5r*rrmask_feature_lengthmask_feature_min_masksexpand)rFrNrrr}rr2mask_feature_indicess rI_mask_hidden_statesHubertModel._mask_hidden_states st{{$8$?? 4A3E3E3G0 [  (/3/E/E/H/HI\I\/]M , [[ ' '! + 5-++44 KK88-++99 ! !& ->G[G[chcmcm n /3/E/E/H/HI\I\/]M , ;; ( (1 ,#8)++77 KK;;++<< $ $)<<0DMaMainisis#t #74#@#G#GO]_#` 23M /rQF)tie_last_hidden_statesrroutput_hidden_states return_dictrc UbUOURRnUbUOURRnUbUOURRnUR U5nUR SS5nUbUR URSU5nURU5n URXS9n URU UUUUS9n U Sn U(d U 4U SS-$[U U RU RS9$)a mask_time_indices (`torch.BoolTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices to mask extracted features for contrastive loss. When in training mode, model learns to predict masked extracted features in *config.proj_codevector_dim* space. Example: ```python >>> from transformers import AutoProcessor, HubertModel >>> from datasets import load_dataset >>> processor = AutoProcessor.from_pretrained("facebook/hubert-large-ls960-ft") >>> model = HubertModel.from_pretrained("facebook/hubert-large-ls960-ft") >>> def map_to_array(example): ... example["speech"] = example["audio"]["array"] ... return example >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> ds = ds.map(map_to_array) >>> input_values = processor(ds["speech"][0], return_tensors="pt").input_values # Batch size 1 >>> hidden_states = model(input_values).last_hidden_state ```Nrr")r)rrrrr)r&rNrZ)rGrrrrrLr~rrrrWrrNrZ) rFrrrrrrrextract_featuresrNencoder_outputss rIrOHubertModel.forward7s)N2C1N-TXT_T_TqTq$8$D $++JjJj &1%[TU]U5 [U5Ul[R "UR 5UlX lURc[SURS35e[US5(aUR(a URO URn[R "X1R5UlUR%5 g)a target_lang (`str`, *optional*): Language id of adapter weights. Adapter weights are stored in the format adapter..safetensors or adapter..bin. Only relevant when using an instance of [`HubertForCTC`] with adapters. Uses 'eng' by default. NzYou are trying to instantiate z with a configuration that does not define the vocabulary size of the language model head. Please instantiate the model as follows: `HubertForCTC.from_pretrained(..., vocab_size=vocab_size)`. or define `vocab_size` of your model's configuration. add_adapter)r.r/rbrTr0r final_dropoutrr vocab_sizerrHr:routput_hidden_sizer2rlm_headr)rFrGrrrHs rIr/HubertForCTC.__init__s  !&) zz&"6"67 &    $00@AHH *1)G)GFL^L^F % %djdvdv yy!35F5FG  rQc @[5[R"S5:XagURnUb'[ UR SS5c[ SUS35eUc.[ UR SS5b[RS5 gUbURUSS9 gg) a This method overwrites [`~PreTrainedModel.tie_weights`] so that adapter weights can be correctly loaded when passing `target_lang=...` to `from_pretrained(...)`. This method is **not** supposed to be called by the user and is prone to be changed in the future. metaNr9zCannot pass `target_lang`: z- if `config.adapter_attn_dim` is not defined.z)By default `target_lang` is set to 'eng'.T) force_load) rrr*rr7rGrloggerinfo load_adapter)rFrrs rI tie_weightsHubertForCTC.tie_weightss 6 75<<;O O  &&  "wt{{UbJUR5URR:a"[SURR35eUR"U4SU0UD6nUR nUR U5nURU5nSnUGbXUbUO"[R"U[RS9nURURS55R[R5n US:n U RS5n URU 5n [R R#US[R$S9R'SS5n [R(R*R-S S 9 [R R/U U U U URR0URR2URR4S 9nSSS5 [7XUR8UR:S 9$!,(df  N-=f) a labels (`torch.LongTensor` of shape `(batch_size, target_length)`, *optional*): Labels for connectionist temporal classification. Note that `target_length` has to be smaller or equal to the sequence length of the output logits. Indices are selected in `[-100, 0, ..., config.vocab_size - 1]`. All labels set to `-100` are ignored (masked), the loss is only computed for labels in `[0, ..., config.vocab_size - 1]`. Nz$Label values must be <= vocab_size: rrrr)r,rurF)enabled)blank reduction zero_infinitylosslogitsrNrZ)rrGrrrTr&rrr ones_likerwrqrvr) masked_selectr0r log_softmaxfloat32rLbackendscudnnflagsctc_loss pad_token_idctc_loss_reductionctc_zero_infinityrrNrZ)rFrrrroutputsrNrrrg labels_masktarget_lengthsflattened_targets log_probss rIrOHubertForCTC.forwards  &**,$++2H2H"HCDKKDZDZC[\] ]++  )    11  ]3 m,  #1"<%//R^fkfpfpBq !AA.BTBTUWBXY\\]b]g]ghM!A+K(__R0N & 4 4[ A  11&b 1V``abdefI%%++E+:}}--%!"++22"kk<<"&++"?"?.;G4I4IV]VhVh  ;:s A H H)rrTrrrtr)rSrTrUrVrr/rrrrrrrrrrrrOrWrXrYs@rIrrs C$J:<0;(/3&* : llT):  t+:  t# : +, :   : : rQrz Hubert Model with a sequence classification head on top (a linear layer over the pooled output) for tasks like SUPERB Keyword Spotting. c^\rSrSrU4SjrSrSr\\\ S S\ RS-S\ RS-S\ RS-S \ \ S \\-4 S jj555rS rU=r$)rdi c">[TU]U5 [US5(aUR(a [ S5e[ U5UlURS-nUR(a2[R"[R"U5U- 5Ul [R"URUR 5Ul[R"UR UR$5UlUR)5 g)Nrz]Sequence classification does not support the use of Hubert adapters (config.add_adapter=True)r)r.r/r:rrrbrTruse_weighted_layer_sumr0rrrr]rr2classifier_proj_size projector num_labels classifierr)rFrG num_layersrHs rIr/(HubertForSequenceClassification.__init__s  6= ) )f.@.@o "&) --1  ( (!#ejj.Dz.Q!RD 6#5#5v7R7RS))F$?$?ARARS rQcLURRR5 grrrs rIr6HubertForSequenceClassification.freeze_feature_encoder#rrQcTURR5H nSUlM grrrs rIr1HubertForSequenceClassification.freeze_base_model*rrQNrrrrrc URR(aSUS'UR"U4SU0UD6nURR(alURn[R "USS9n[ RRURSS9nXgRSSS5-RSS9nO URnURU5nUcURSS9nOURUR SU5n U R#S5R%SSUR S5n S Xj)'URSS9U RSS9RSS5- nUR'U5n Sn UbF[)5n U "U RSURR*5URS55n [-U U URUR.S 9$) a input_values (`torch.FloatTensor` of shape `(batch_size, sequence_length)`): Float values of input raw speech waveform. Values can be obtained by loading a `.flac` or `.wav` audio file into an array of type `list[float]`, a `numpy.ndarray` or a `torch.Tensor`, *e.g.* via the torchcodec library (`pip install torchcodec`) or the soundfile library (`pip install soundfile`). To prepare the array into `input_values`, the [`AutoProcessor`] should be used for padding and conversion into a tensor of type `torch.FloatTensor`. See [`HubertProcessor.__call__`] for details. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). TrrrrrNr"rr)rGr rTrNrstackr0rrr]rrvr&r meanr~rr'r(rrr rrZ)rFrrrrrrN norm_weights pooled_output padding_maskexpand_padding_maskrrloss_fcts rIrO'HubertForSequenceClassification.forward2s, ;; - --1F) *++  )   ;; - -#11M!KK 1=M==001C1C0LL*->->r1a-HHMMRSMTM#55M}5  !)..1.5MBB=CVCVWXCY[ijL"."8"8"<"C"CAq-J]J]^_J`"a 25M. /)--!-4|7G7GA7G7N7S7STVXY7ZZM/  ')HFKKDKK,B,BCV[[QS_UD'!//))   rQ)rrTr]r r)rSrTrUrVr/rrrrrrrrrrrrOrWrXrYs@rIrdrd s";( /3&* 9 llT)9  t+9  t# 9 +, 9 ) ) 9 !9 rQrd)rrdrbrSrr`)Ncollections.abcrnumpyrrtorch.nnr0rrr_ activationsrbackbone_utilsrintegrations.deepspeedr integrations.fsdpr masking_utilsr modeling_flash_attention_utilsr modeling_layersr modeling_outputsrrrmodeling_utilsrrrprocessing_utilsrr9rrr utils.genericrrutils.output_capturingrrconfiguration_hubertr get_loggerrSrModulerrCrdrxrrrrrrrrr rr7rFrMrSrrrndarrayrrbrrd__all__rrQrIr3s*% %&!9@76B9YYrr&@@IE.   H %/BII/d!;*9690#299#Lbii0!% II% <<% % << % LL4' % T\ %%'(%8R)biiR)j 03D+@BII+@\RYY2$(B$N/@299/@dBOBBR/3 t c?ttt$$t+ t  t ZZ tnI 'I I X A (A  A H ] &;]  ] @ frQ