0ViSSKrSSKrSSKJr SSKJrJr SSKr/SQr Sr Sr Sr Sr SrS \\R\\R4S S4S jrS\ SS4S jrSS\ SS4SS .S \\RS\\\R\\R4S\S\S\\\R&R(S\\\RS S4SjjjrSS \\RS\S S4SjjrSS \\RS\\RS S4Sjjr\ SS4S \\RS\\RS\S S4Sjjrg)N)Sequence)OptionalUnion) all_reducereduce broadcast all_gatherreduce_scatterc^[[RS5(d[R"SSS9 g[ 5nUHhnUR (a gUR5(d gUR(d gUR5nX1;a gURU5 Mj g)N_nccl_all_reducez)PyTorch is not compiled with NCCL support stacklevelFT) hasattrtorch_Cwarningswarnset is_sparse is_contiguousis_cuda get_deviceadd)tensorsdevicestensordevices TD:\a\ltx-desktop-ci\ltx-desktop-ci\python-embed\Lib\site-packages\torch\cuda\nccl.py is_availabler s 588/ 0 0 AaPeG   ##%%~~""$   F c[RR5nUS- nUS- S-nUS-n[RR5R S5nUS:XaXU4$XX44$)a Returns the version of the NCCL. This function returns a tuple containing the major, minor, and patch version numbers of the NCCL. The suffix is also included in the tuple if a version suffix exists. Returns: tuple: The version information of the NCCL. izutf-8)rr _nccl_version_nccl_version_suffixdecode)vermajorminorpatchsuffixs rversionr.$sp (( "C 2IE BY% E %KE XX * * , 3 3G [RR5$N)rr_nccl_unique_idr!r unique_idr39s 88 # # %%r!cB[RRXU5$r0)rr_nccl_init_rank) num_ranksuidranks r init_rankr9=s 88 # #ID 99r!inputsreturnc[U[RR5(a[U[R 5(a [ S5eg)Nz(Inputs should be a collection of tensors) isinstance collectionsabc ContainerrTensor TypeError)r:s r_check_sequence_typerCAsA fkoo77 8 8J ==BCC=r!c|[U5 UcUn[U5 [RRXX#U5 gr0)rCrrr r:outputsopstreamscommss rrrHs2 ! HHfrEBr!)rFoutputrootrGrHrFc[U5 Ub-Ub [S5e[R"S[SS9 XbnOp[ U[ R5(dH[ U[RR5(a[R"S[SS9 XnO UcXOUn[ RRXX#XE5 g)Nz'output' and 'outputs' can not be both specified. 'outputs' is deprecated in favor of 'output', taking in a single output tensor. The signature of reduce is: reduce(inputs, output=None, root=0, op=SUM, streams=None, comms=None).z`nccl.reduce` with an output tensor list is deprecated. Please specify a single output tensor with argument 'output' instead instead.r rz\nccl.reduce with an output tensor list is deprecated. Please specify a single output tensor.) rC ValueErrorrr FutureWarningr=rrAr>r?rr _nccl_reduce)r:rJrKrGrHrIrF_outputs rrrRs   Y  MM`   mG  - -* ((33   5   ,"(.&,f HH&4WDr!cZ[U5 [RRXX#5 gr0)rCrr_nccl_broadcast)r:rKrHrIs rrr}s   HHV7:r!cp[U5 [U5 [RRXX#5 gr0)rCrr_nccl_all_gather)r:rFrHrIs rr r s(  ! HHfw>r!cr[U5 [U5 [RRXX#U5 gr0)rCrr_nccl_reduce_scatterrEs rr r s* ! HH!!&2Fr!)rNN)NN)r>rcollections.abcrtypingrr torch.cudar__all__SUMr r.r3r9rArCrintcudaStreamrrr r r2r!rr_s$" P*-*&:Du||Xell5K'K!LDQUD $TCEI59  (E15(E U\\ "(E U5<<%,,)??@ A(E (E (E huzz001 2 (Ehu||, -(E (EXHL; U\\ ";*-; ;   ? U\\ "? ell #?  ?   G U\\ " G ell # G  G  Gr!