oMjnSrSSKrSSKJr SSKJr SSKrSSKJr SSK Js J r SSK J r SSKJrJr SSKJr SS KJr S S KJrJr S S KJr S SKJr "SS\R85r"SS\R85r"SS\R85r"SS\R85r "SS\R85r!"SS\R85r"\"SS\55r#\"SS\55r$"SS \\5r%g)!aCosmos3 AVAE Audio Tokenizer. The decoder reuses the Oobleck architecture (Snake1d activations + weight-norm convs + residual units), inlined here instead of imported so the audio module is self-contained. The encoder is the Cosmos3 SpecConvNeXt audio encoder used by AVAE checkpoints; it is intentionally separate from Oobleck's waveform encoder because the tensor layouts and bottleneck semantics are different. N) OrderedDict) dataclass) weight_norm) ConfigMixinregister_to_config) BaseOutput)apply_forward_hook) ModelMixinget_parameter_dtype) FP32LayerNorm)#OobleckDiagonalGaussianDistributionc6^\rSrSrSrSU4SjjrSrSrU=r$)Snake1d*z3 A 1-dimensional Snake activation function module. c6>[TU]5 [R"[R "SUS55Ul[R"[R "SUS55UlSUR lSURlX l g)NrT) super__init__nn Parametertorchzerosalphabeta requires_gradlogscale)self hidden_dimr __class__s C:\Users\johnn\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\diffusers/models/autoencoders/autoencoder_cosmos3_audio.pyrSnake1d.__init__/se \\%++aQ"?@ LLQ A!>? #'  "&  cURnUR(d URO[R"UR5nUR(d UR O[R"UR 5nUR USUSS5nXS-R5[R"X1-5RS5--nUR U5nU$)Nrrg& .>r ) shaperrrexprreshape reciprocalsinpow)r hidden_statesr'rrs r"forwardSnake1d.forward8s##"&-- UYYtzz5J $ tyy599TYY3G%--eAha"E %(@(@(BUYYuOdEeEiEijkEl(ll %--e4 r$)rrr)T __name__ __module__ __qualname____firstlineno____doc__rr.__static_attributes__ __classcell__r!s@r"rr*s!  r$rc ^\rSrSrSrS S\S\S\S\S\4 U4SjjjrS \RS \R4S jr S r U=r $)Cosmos3AudioConvNeXtBlockDz;1D ConvNeXt block used by the Cosmos3 SpecConvNeXt encoder.r intermediate_dim identity_init use_snakecausalc >>[TU]5 XPlU(aF[R"[R "SS5[R "XSUS95UlOE[R"[R "SS5[R "XSUS95Ul[USSS9Ul [R "XS S 9Ul U(a [U5O[R"5Ul [R "X!S S 9UlU(a[RR!URR"5 URR$b4[RR!URR$5 ggg) N)rr) kernel_sizegroups)rrh㈵>F)epsbiasrrC)rrr?r Sequential ConstantPad1dConv1ddwconvrnormpwconv1rGELUactpwconv2initzeros_weightrG)rr r<r=r>r?r!s r"r"Cosmos3AudioConvNeXtBlock.__init__Gs  --  + *a SDK --  + *a SDK "*$UC yy1M 097+,rwwyyy!11M  GGNN4<<.. /||  ,t||001- r$r-returncUnURU5nURURSSS55RSSS5nURU5nUR U5nUR U5nX!-$)Nrr r)rLrMpermuterNrPrQ)rr-residuals r"r.!Cosmos3AudioConvNeXtBlock.forwardfsu  M2  -"7"71a"@AII!QPQR  ]3 /  ]3 ''r$)rPr?rLrMrNrQ)FTF) r1r2r3r4r5intboolrrTensorr.r6r7r8s@r"r:r:DskE $ 222 2  2  22>(U\\(ell((r$r:c^\rSrSrSrS\S\S\S\S\\S4S \\S4S \S \S \S \S\S\S\4U4Sjjr S\ RS\ R4Sjr S\ RS\ R4Sjr SrU=r$)&Cosmos3AudioSpectrogramConvNeXtEncoderpzKCosmos3 waveform-to-latent encoder using STFT features and ConvNeXt blocks.input_channelsstereochannels latent_dimchannel_multiples.strides num_blocksn_fft hop_lengthr=r>r? padding_modecz>[TU]5 U (a [S5e[U5[U5:wa$[ S[U5S[U5S35eX(aSOS-UlX0lX@l[U5Ul [U5Ul Xpl Xl Xl Xl[[ R""US-UR -URSU-SSS 95/n[%UR5HunnURUU-nU[UR5S- :aURUS-U-OURS U-n['U5H"nUR)[+UUS -U U U S 95 M$ UR)[[ R""UUSU-U[,R."US- 5U S 955 M UR)[[ R""URS U-USSS 955 [ R0"U6Ulg)Nz7Cosmos3 AVAE causal audio encoder is not supported yet.z?`enc_c_mults` and `enc_strides` must have the same length, got z and .r rrF)rCrGr&)r r<r=r>r?)rCstridepaddingrj)rrNotImplementedErrorlen ValueErrorrarcrdtuplererfrgrhrir?rrrK enumeraterangeappendr:mathceilrIlayers)rrarbrcrdrerfrgrhrir=r>r?rjryindexrn input_dim output_dim_r!s r"r/Cosmos3AudioSpectrogramConvNeXtEncoder.__init__ss1  %&_` `  !S\ 1Q()*%G ~Q@  -VC $!&'8!9W~ $ $   QY$"5"55**1-8 !   # 't||4ME6..u5@I3t556::&&uqy1H<++B/(:  :& -#,)2Q&3"+% ' MMII!"$%J% $ &1* 5%1   '5@   $"8"8"[TU]5 SU-S-n[U5Ul[ [ R "XSX#S95Ul[U5Ul[ [ R "XSS95Ul g)NrAr rB)rCrrorrH) rrrsnake1rrrKconv1snake2conv2)rrrrr!s r"r!Cosmos3AudioResidualUnit.__init__sf !a'i(  9QYa!op i(  9Q!OP r$cUnURURU55nURURU55nURSURS- S-nUS:a USX3*24nX-nU$)a1 Forward pass through the residual unit. Args: hidden_state (`torch.Tensor` of shape `(batch_size, channels, time_steps)`): Input tensor . Returns: output_tensor (`torch.Tensor` of shape `(batch_size, channels, time_steps)`) Input tensor after passing through the residual unit. r&r r.)rrrrr')r hidden_state output_tensorros r"r. Cosmos3AudioResidualUnit.forwards%  4;;}#=>  4;;}#=> %%b)M,?,?,CCI Q;'WX-=(=>L$4 r$)rrrr)r r1r2r3r4r5r[rr.r6r7r8s@r"rrs.Q#QcQQr$rcB^\rSrSrSrSS\S\4U4SjjjrSrSrU=r $) Cosmos3AudioDecoderBlockiz+Decoder block used in Cosmos3Audio decoder.rnoutput_paddingc >[TU]5 [U5Ul[ [ R "UUSU-U[R"US- 5US95Ul [USS9Ul [USS9Ul [USS9Ul g)Nr )rCrnrorr)rr )rrrrrrConvTranspose1drwrxconv_t1r res_unit1 res_unit2 res_unit3)rr{r|rnrr!s r"r!Cosmos3AudioDecoderBlock.__init__s i( "   J &1*--     2*qI1*qI1*qIr$cURU5nURU5nURU5nURU5nUR U5nU$N)rrrrr)rrs r"r. Cosmos3AudioDecoderBlock.forward/sN{{<0 ||L1 ~~l3 ~~l3 ~~l3 r$)rrrrr)rrrr8s@r"rrs,5JcJsJJ$r$rc2^\rSrSrSrU4SjrSrSrU=r$)Cosmos3AudioDecoderi?zCosmos3Audio Decoderc >[T U]5 UnS/U-n[[R"X!US-SSS95Ul/n[ U5H=upU[X[U5U- -X[U5U- S- -U U S-S9/- nM? [R"U5Ul Un [U 5Ul [[R"XSSSS 95Ul g) Nrr&rBr)rCror )r{r|rnrF)rCrorG)rrrrrKrrtrrq ModuleListblockrrr) rrcraaudio_channelsupsampling_ratiosrerfr stride_indexrnr|r!s r"rCosmos3AudioDecoder.__init__Bs #C"33!>FWXZF[;[ijtu!vw $-g$6 L (&3w<,;V)WW'CL<S\S\ S\ S\ S \ S \ S \ S \ S \ S\ S\ S-S\ S\S\ S\ S\ S\ S\ S\ S\ S\ S\ S\ S\ S\S\ S\ S \ S!\ S"\ S#\S$\ S-S%\S&\ S'\ S(\ S)\ S*\S+\ \\ -S-S,\ \\ -S-S-\ 4RU4S.jjj5rS/rS0\S1S4U4S2jjrS3\R(S1\R(4S4jr\S?S3\R(S5\ S6\ S1\\ \-4S7jj5r\S8\R(S1\R(4S9j5rS@S3\R(S:\ S5\ S;\R6S-S6\ S1\\ \R(-4 S<jjrS=rU=r$)ACosmos3AVAEAudioTokenizerivuDAudio tokenizer for Cosmos3 sound generation. Wraps the Cosmos3 AVAE SpecConvNeXt encoder and Oobleck-style decoder used by the Cosmos3 omni model. The decoder API stays tensor-returning because ``Cosmos3OmniPipeline`` calls it directly when ``enable_sound=True``. Only the shipped AVAE configuration (``model_type="autoencoder_v2"``, waveform input, ``spec_convnext`` encoder, ``vae`` bottleneck, ``oobleck`` decoder, log-scale SnakeBeta, no latent normalization) is supported; any other value raises ``NotImplementedError``. Parameters: model_type (`str`, defaults to `"autoencoder_v2"`): AVAE model variant; only `"autoencoder_v2"` is supported. sampling_rate (`int`, defaults to `48000`): Audio sample rate in Hz. vocoder_input_dim (`int`, defaults to `64`): Latent channel count fed into the decoder (``== transformer sound_dim``). dec_dim (`int`, defaults to `320`): Base decoder channel count. dec_c_mults (`tuple[int, ...]`, defaults to `(1, 2, 4, 8, 16)`): Decoder channel multipliers. dec_strides (`tuple[int, ...]`, defaults to `(2, 4, 5, 6, 8)`): Decoder upsampling strides. dec_out_channels (`int`, defaults to `2`): Output audio channels (2 = stereo). stereo (`bool`, defaults to `True`): Whether the audio is stereo; doubles the encoder's effective channel count. use_wav_as_input (`bool`, defaults to `True`): Whether the encoder consumes raw waveforms; only `True` is supported. normalize_volume (`bool`, defaults to `True`): Whether `encode` peak-normalizes the waveform before encoding. hop_size (`int`, *optional*): Waveform→latent temporal compression factor used for `encode` padding. Defaults to `prod(dec_strides)` when `None`. input_channels (`int`, defaults to `1`): Per-channel encoder input count before the `stereo` doubling. enc_type (`str`, defaults to `"spec_convnext"`): Encoder type; only `"spec_convnext"` is supported. enc_dim (`int`, defaults to `192`): Base encoder channel count. enc_intermediate_dim (`int`, defaults to `768`): Unused; kept for config fidelity (ConvNeXt blocks use ``input_dim * 4``). enc_num_layers (`int`, defaults to `12`): Unused; kept for config fidelity (depth derives from `enc_num_blocks`). enc_num_blocks (`int`, defaults to `2`): ConvNeXt blocks per encoder downsampling stage. enc_n_fft (`int`, defaults to `64`): STFT FFT size for the encoder spectrogram front-end. enc_hop_length (`int`, defaults to `16`): STFT hop length for the encoder spectrogram front-end. enc_latent_dim (`int`, defaults to `128`): Encoder output channels; split into mean/scale by the VAE bottleneck (so ``enc_latent_dim == 2 * vocoder_input_dim``). enc_c_mults (`tuple[int, ...]`, defaults to `(1, 2, 4)`): Encoder channel multipliers per stage. enc_strides (`tuple[int, ...]`, defaults to `(4, 5, 6)`): Encoder downsampling strides per stage. enc_identity_init (`bool`, defaults to `False`): Whether to zero-init the ConvNeXt residual 1x1 convs. enc_use_snake (`bool`, defaults to `True`): Whether ConvNeXt blocks use SnakeBeta (else GELU). dec_type (`str`, defaults to `"oobleck"`): Decoder type; only `"oobleck"` is supported. dec_use_snake (`bool`, defaults to `True`): Whether the decoder uses SnakeBeta; only `True` is supported. dec_final_tanh (`bool`, defaults to `False`): Vestigial decoder tanh flag; only `False` is supported. dec_anti_aliasing (`bool`, defaults to `False`): Decoder anti-aliasing flag; only `False` is supported. dec_use_nearest_upsample (`bool`, defaults to `False`): Decoder upsample mode flag; only `False` is supported. dec_use_tanh_at_final (`bool`, defaults to `False`): Decoder final-tanh flag; only `False` is supported. bottleneck_type (`str`, defaults to `"vae"`): Bottleneck type; only `"vae"` is supported. bottleneck (`dict`, *optional*): Bottleneck config; if given, its `"type"` must be `"vae"`. activation (`str`, defaults to `"snakebeta"`): Activation family; only `"snakebeta"` is supported. snake_logscale (`bool`, defaults to `True`): Whether SnakeBeta parameters are log-scaled; only `True` is supported. anti_aliasing (`bool`, defaults to `False`): Global anti-aliasing flag; only `False` is supported. use_cuda_kernel (`bool`, defaults to `False`): Whether to use fused CUDA kernels; only `False` is supported. causal (`bool`, defaults to `False`): Whether convolutions are causal; only `False` is supported by the encoder. padding_mode (`str`, defaults to `"zeros"`): Convolution padding mode. latent_mean (`float` or `list[float]`, *optional*): Latent normalization mean; latent normalization is not implemented, so a non-`None` value raises ``NotImplementedError``. latent_std (`float` or `list[float]`, *optional*): Latent normalization std; latent normalization is not implemented, so a non-`None` value raises ``NotImplementedError``. encoder_enabled (`bool`, defaults to `True`): Whether to instantiate the encoder. Set to `False` (or auto-disabled on load) for decoder-only checkpoints, which cannot `encode`. FN model_type sampling_ratevocoder_input_dimdec_dim dec_c_mults dec_stridesdec_out_channelsrbuse_wav_as_inputnormalize_volumehop_sizeraenc_typeenc_dimenc_intermediate_dimenc_num_layersenc_num_blocks enc_n_fftenc_hop_lengthenc_latent_dim enc_c_mults enc_stridesenc_identity_init enc_use_snakedec_type dec_use_snakedec_final_tanhdec_anti_aliasingdec_use_nearest_upsampledec_use_tanh_at_finalbottleneck_type bottleneck activationsnake_logscale anti_aliasinguse_cuda_kernelr?rj latent_mean latent_stdencoder_enabledc*F>[T*U]5 US:wa[SU<S35eU (d [S5eU S:wa[SU <S35eU b!U RSU5S:wa [S 5eUS:wa [S 5eUS :wa[S U<S35eU(aU(dU(dU(dU(a [S 5eU!S :wdU"(aU#(dU$(a [S5eU'cU(b [S5eSUlSUlU)(a3[ U UUU[U5[U5UUUUUU%U&S9 UlSUl[UUU[[U55[U5S9Ul U b[U 5Ulg[R"U5Ulg)Nautoencoder_v2zCosmos3 AVAE model type z is not supported.z4Cosmos3 AVAE tokenizer only supports waveform input. spec_convnextzCosmos3 AVAE encoder type typevaez8Cosmos3 AVAE tokenizer only supports the VAE bottleneck.oobleckzCosmos3 AVAE decoder type zMCosmos3 AVAE decoder only supports the shipped Oobleck decoder configuration. snakebetazICosmos3 AVAE tokenizer only supports the shipped SnakeBeta configuration.zdCosmos3 AVAE tokenizer does not apply latent normalization; `latent_mean`/`latent_std` must be None.F) rarbrcrdrerfrgrhrir=r>r?rjT)rcrarrre)rrrpgetencoder_encoder_availabler_rsrlistreverseddecoderr[rwprod _hop_size)+rrrrrrrrrbrrrrarrrrrrrrrrrrrrrrrrrrrrrrr?rjrrrr!s+ r"r"Cosmos3AVAEAudioTokenizer.__init__sZ  ) )%(@N`&ab b%&\] ]  &%(B8,N`&ab b  !jnnV_&MQV&V%&`a a e #%&`a a y %(B8,N`&ab b '$%&uv v  $Nm%&qr r  "j&<%v  "' A- )"' "4k*))/')DL'+D #*,+"8K#89";/   08/Cc(mS^I_r$c>SUlSUlURSS9 g)NF)r)rrr)rs r"_disable_encoder*Cosmos3AVAEAudioTokenizer._disable_encoder$s" "' 6r$ state_dictrVc>[TU]U5 URb)[SU55(dUR 5 ggg)Nc3B# UHoRS5v M g7f)zencoder.N) startswith).0keys r" ICosmos3AVAEAudioTokenizer._fix_state_dict_keys_on_load..+s/aV`sz0J0JV`s)r_fix_state_dict_keys_on_loadranyr)rrr!s r"r6Cosmos3AVAEAudioTokenizer._fix_state_dict_keys_on_load)s@ ,Z8 << #C/aV`/a,a,a  ! ! #-b #r$rcDURU5RSS5$)Nrr )rr)rrs r"_encode!Cosmos3AVAEAudioTokenizer._encode.s||F#--a33r$ return_dict force_padcURS:wa"[S[UR5S35eURbUR (d [S5eUnUR R(a&XDR5R5S-- S-nU(dUR(dVURSnURXPR-- UR-nUS:a[R"USU4S SS 9n[UR5nURUR!US 95n[#U5n U(dU 4$[%U S 9$) aEncode a waveform into a VAE latent distribution. Args: sample: Audio waveform tensor with shape ``[B, C, T]``. return_dict: Whether to return a ``Cosmos3AudioEncoderOutput``. force_pad: Whether to right-pad to ``hop_size`` even when the model is in training mode. rz(`sample` must have shape [B, C, T], got rlzThis Cosmos3 AVAE sound tokenizer was loaded from decoder-only weights and cannot encode audio. Re-convert the AVAE checkpoint with encoder weights to use `encode()`.rEgffffff?r&rconstant)modevalue)r)r)ndimrrrsr'rrconfigrabsmaxtrainingr rrr rrrr) rrrrr- sample_lengthro encoder_dtypemoments posteriors r"encode Cosmos3AVAEAudioTokenizer.encode1s/ ;;! GfllH[G\\]^_ _ << t'>'>Y   ;; ' ')->->-@-D-D-F-MNQUUM DMM)//3M~~)GHDNNZG{ !ma\ Z[ \ +DLL9 ,,}//m/DE7@ < (Y??r$latentscURS:HnU(aURS5nURU5RSS5nU(aUR S5$U$)zDecode sound latents into an audio waveform. Args: latents: ``[B, C, T]`` or ``[C, T]`` tensor of diffusion-model latents. Returns: Waveform tensor ``[B, audio_channels, N]`` or ``[audio_channels, N]``. r rgg?)r$ unsqueezer clampr)rr/rrs r"decode Cosmos3AVAEAudioTokenizer.decode[sV,,!# ''*G W%++D#6#*u}}Q55r$sample_posterior generatorcURXS9RnU(aURUS9OUR5nUR U5nU(dU4$[ US9$)a Encode then decode a waveform. `sample_posterior=False` (default) decodes the distribution mode (mean), whereas the upstream Cosmos3 AVAE always samples; pass `sample_posterior=True` for reference-equivalent behavior. Args: sample (`torch.Tensor`): Input waveform sample with shape `(batch_size, audio_channels, num_samples)`. sample_posterior (`bool`, *optional*, defaults to `False`): Whether to sample from the posterior instead of decoding the distribution mode. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`Cosmos3AudioDecoderOutput`] instead of a plain tuple. generator (`torch.Generator`, *optional*): A [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make sampling deterministic. force_pad (`bool`, *optional*, defaults to `False`): Whether to right-pad the waveform to `hop_size` before encoding even when the model is in training mode. Returns: [`Cosmos3AudioDecoderOutput`] or `tuple`: If `return_dict` is True, a [`Cosmos3AudioDecoderOutput`] is returned, otherwise a plain `tuple` is returned. )r)r6)r)r-rrr"r3r) rrr5rr6rr,r/decodeds r"r.!Cosmos3AVAEAudioTokenizer.forwardks_>KKK<HH ;K)""Y"7QZQ_Q_Qa++g&: (88r$)rr r r))ri@i@)rr rmr)r rmrAr;r TTTNrri r r:r)rr rm)rmr<rAFTrTFFFFrNrTFFFrNNT)TF)FTNF) r1r2r3r4r5 _supports_gradient_checkpointing_supports_group_offloadingrrr[rsr\dictrrrrrrrr]rr rrr-r3 Generatorrr.r6r7r8s@r"rrvs@D(-$!&+"!#-, !!%!%#'$'  !&&"'"!"$"').&+$"&%## %#2615 $Ue`e`e` e`  e`  e`e`e`e`e`e`*e`e`e`e` "!e`"#e`$%e`&'e`()e`*+e`,-e`./e`0 1e`23e`45e`67e`89e`: ;e`<#'=e`> $?e`@Ae`B4KCe`DEe`FGe`HIe`JKe`LMe`NOe`PT%[(4/Qe`RDK'$.Se`TUe`e`N7 ${$t$ 4ell4u||4! '@ '@'@ '@ #U+N%O O '@'@R 6ell 6u|| 6 6$"' ,0 &9 &9&9 &9 ??T) &9  &9 #U5<<%8 8&9&9r$r)&r5rw collectionsr dataclassesrrtorch.nnrtorch.nn.functional functionalrtorch.nn.utilsrconfiguration_utilsrrutilsr utils.accelerate_utilsr modeling_utilsr r normalizationrautoencoder_oobleckrModulerr:r_rrrrrrrr$r"rQs  #! &B8<)Dbii4)( )(X{-RYY{-~"ryy"J ryy> &"))&R 5 5 5     [9 K[9r$