KgjwSSKrSSKrSSKJr SSKJs Jr SSKJr S Sjr "SS\R5r "SS\R5r "SS \R5r g) N) rearrangecURSnSU-n[RRXS9U- nUb2[RRXFS9U- nXxR 5-n[RRUR UR S9US9n [UR5S:aURS5n[RRX-USS9S SU24n X-n U(a[R"U 5n Ub&U [US 5-R UR S9$U R UR S9$) N)ndtypeforward)rnorm.z b H -> b H 1) shapetorchfftrfftconjtor len unsqueezeirfftFgelur) uk dropout_maskrk_revseqlenfft_sizek_fk_rev_fu_fyouts ڨC:\Users\johnn\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\fla/modules/conv/long_conv.pyfft_convr%s WWR[F6zH ))... '( 2C ))...3h>LLN" ))..AGG,. 9C 177|ammA  XI>sGVG|LA %C ffSki n==AAAPPvvAGGv$$cZ^\rSrSrSrS\S\4U4SjjrS\R4Sjr Sr U=r $) LongConvolution&a{ LongConvolution applies a convolution operation on the input tensor using a fixed filter of length max_len. The filter is learned during training and is applied using FFT convolution. Args: hidden_size (int): The number of expected features in the input and output. max_len (int): The maximum sequence length. Returns: y: [batch_size, seq_len, hidden_size] tensor hidden_sizemax_lenc >[TU]5 Xl[R"[ R "URU5SS9Ulg)z Initializes the LongConvolution module. Args: hidden_size (int): The number of expected features in the input and output. max_len (int): The maximum sequence length. T requires_gradN)super__init__r*nn Parameterrrandnfilter)selfr*r+kwargs __class__s r$r0LongConvolution.__init__4s: &ll5;;t/?/?#IY]^ r&xcURSS5n[XRSSS9nURSS5nURURS9$)z Applies the LongConvolution operation on the input tensor. Args: x: [batch_size, seq_len, hidden_size] tensor Returns: y: [batch_size, seq_len, hidden_size] tensor r rNFrrr) transposer%r4rr )r5r9argsr6r"s r$r LongConvolution.forwardDsK KK1  Q $U C KK1 tt!''t""r&)r4r*) __name__ __module__ __qualname____firstlineno____doc__intr0rTensorr __static_attributes__ __classcell__r7s@r$r(r(&s7 ___ # # #r&r(c:^\rSrSrS\S\4U4SjjrSrSrU=r$)PositionalEmbeddingRemb_dimseq_lenc >[T U]5 X l[R"SSUR5SSS2S4nUS:aUS- S-n[R"SUS- U5SSS2S4nS[ R -U-U- n[R"SWS- U5Sn[R"SU-U-5n [R"XIRU R/SS 9n [R"U S S 9Ul g) zPComplex exponential positional embeddings for implicit long convolution filters.rr Nrg-C6?)NNyr)dimFr-)r/r0rMrlinspacemathpiexpcatrealimagr1r2z) r5rLrMr6tbands t_rescaledwfrWr7s r$r0PositionalEmbedding.__init__Ss  NN1a .tQ} = Q;q[Q&E^^Aw{G IIcAgk " IIq&&!&&)r 2au5r&c,URSS2SU24$)N)rW)r5Ls r$r PositionalEmbedding.forwardfsvva!e}r&)rMrW) r?r@rArBrDr0r rFrGrHs@r$rJrJRs 66c6&r&rJc x^\rSrSrSrS S\S\S\S\4U4SjjjrS\4S jrS \R4S jr S r U=r $)ImplicitLongConvolutionja Long convolution with implicit filter parameterized by an MLP. Args: hidden_size (int): The number of expected features in the input and output. max_len (int): The maximum sequence length. d_emb (Optional[int]): The dimension of the positional embeddings. Must be odd and greater or equal to 3 (time, sine and cosine). Defaults to 3. d_hidden (Optional[int]): The number of features in the hidden layer of the MLP. Defaults to 16. Attributes: pos_emb (`PositionalEmbedding`): The positional embedding layer. mlp (`nn.Sequential`): The MLP that parameterizes the implicit filter. r*r+d_embd_hiddenc H>[TU]5 XlX0lUS-S:waUS:dS5e[ X25Ul[ R"[ R"X45[R R5[ R"XA55Ul g)zB Long convolution with implicit filter parameterized by an MLP. rrr zCd_emb must be odd and greater or equal to 3 (time, sine and cosine)N) r/r0r*rdrJpos_embr1 SequentialLinearrReLUmlp)r5r*r+rdrer6r7s r$r0 ImplicitLongConvolution.__init__s &  AINuz Q P Q )*5: == IIe & HHMMO IIh , r&rMcbURURU55RSS5$)Nr r)rkrgr<)r5rMr=r6s r$r4ImplicitLongConvolution.filters'xx W-.88A>>r&r9cURSS5nURURS5n[XSSS9nURSS5nUR UR S9$)zs Args: x: [batch_size, seq_len, hidden_size] tensor Returns: y: [batch_size, seq_len, hidden_size] tensor r rrNFr;r)r<r4rr%rr )r5r9r=r6rr"s r$r ImplicitLongConvolution.forwards] KK1  KK $ Q5 9 KK1 tt!''t""r&)rdr*rkrg)r ) r?r@rArBrCrDr0r4rrEr rFrGrHs@r$rbrbjsa0         :?c? # # #r&rb)TN)rQrtorch.nnr1torch.nn.functional functionalreinopsrr%Moduler(rJrbr&r$rxsQ %,)#bii)#X"))0B#biiB#r&