0Vijp%SSKJr SSKrSSKrSSKJr SSKJrJrJrJ r J r SSK J r J r Jr SSKrSSKJr \(aSSKJr SS KJr /S Qr\"S 5r\ "S 5r\"\R2S 5(d]\"S5\R2R4S'\"S5\R2R4S'\"S5\R2R4S'SSKJrJrJr SSjrS Sjr "SS\R2R:5r!"SS5r"\ S\S\#44r$S\%S'\S!S"Sjj5r&\S!S#Sjj5r&S!S$Sjjr&g)%) annotationsN)Callable)Optionaloverload TYPE_CHECKING TypeAliasUnion) ParamSpecSelfTypeVar)Tensor) _POOL_HANDLE) _dummy_type)is_current_stream_capturinggraph_pool_handle CUDAGraphgraphmake_graphed_callables_R_P_CudaStreamBase _CUDAGraph_graph_pool_handle_cuda_isCurrentStreamCapturing)rrrc[5$)zReturn True if CUDA graph capture is underway on the current CUDA stream, False otherwise. If a CUDA context does not exist on the current device, returns False without initializing the context. )rVD:\a\ltx-desktop-ci\ltx-desktop-ci\python-embed\Lib\site-packages\torch\cuda\graphs.pyrr1s * ++rcP[RR[55$)zReturn an opaque token representing the id of a graph memory pool. See :ref:`Graph memory management`. .. warning:: This API is in beta and may change in future releases. )torchcudarrrrrrr:s :: " "#5#7 88rc^\rSrSrSrSSU4SjjjrSSU4SjjjrSU4SjjrSU4SjjrSU4Sjjr SU4Sjjr SU4S jjr SU4S jjr SU4S jjr SU4S jjrSU4S jjrSrU=r$)rFaWrapper around a CUDA graph. Arguments: keep_graph (bool, optional): If ``keep_graph=False``, the cudaGraphExec_t will be instantiated on GPU at the end of ``capture_end`` and the underlying cudaGraph_t will be destroyed. Users who want to query or otherwise modify the underlying cudaGraph_t before instantiation can set ``keep_graph=True`` and access it via ``raw_cuda_graph`` after ``capture_end``. Note that the cudaGraphExec_t will not be instantiated at the end of ``capture_end`` in this case. Instead, it will be instantiated via an explicit called to ``instantiate`` or automatically on the first call to ``replay`` if ``instantiate`` was not already called. Calling ``instantiate`` manually before ``replay`` is recommended to prevent increased latency on the first call to ``replay``. It is allowed to modify the raw cudaGraph_t after first calling ``instantiate``, but the user must call ``instantiate`` again manually to make sure the instantiated graph has these changes. Pytorch has no means of tracking these changes. .. warning:: This API is in beta and may change in future releases. c">[TU]X5$N)super__new__)cls keep_graph __class__s rr(CUDAGraph.__new__asws//rc >[TU]XS9 g)aBegin capturing CUDA work on the current stream. Typically, you shouldn't call ``capture_begin`` yourself. Use :class:`~torch.cuda.graph` or :func:`~torch.cuda.make_graphed_callables`, which call ``capture_begin`` internally. Arguments: pool (optional): Token (returned by :func:`~torch.cuda.graph_pool_handle` or :meth:`other_Graph_instance.pool()`) that hints this graph may share memory with the indicated pool. See :ref:`Graph memory management`. capture_error_mode (str, optional): specifies the cudaStreamCaptureMode for the graph capture stream. Can be "global", "thread_local" or "relaxed". During cuda graph capture, some actions, such as cudaMalloc, may be unsafe. "global" will error on actions in other threads, "thread_local" will only error for actions in the current thread, and "relaxed" will not error on these actions. Do NOT change this setting unless you're familiar with `cudaStreamCaptureMode `_ )poolcapture_error_modeN)r' capture_begin)selfr.r/r+s rr0CUDAGraph.capture_beginds& 4Orc">[TU]5 g)aEnd CUDA graph capture on the current stream. After ``capture_end``, ``replay`` may be called on this instance. Typically, you shouldn't call ``capture_end`` yourself. Use :class:`~torch.cuda.graph` or :func:`~torch.cuda.make_graphed_callables`, which call ``capture_end`` internally. N)r' capture_endr1r+s rr4CUDAGraph.capture_endys rc">[TU]5 g)zInstantiate the CUDA graph. Will be called by ``capture_end`` if ``keep_graph=False``, or by ``replay`` if ``keep_graph=True`` and ``instantiate`` has not already been explicitly called. Does not destroy the cudaGraph_t returned by ``raw_cuda_graph``. N)r' instantiater5s rr8CUDAGraph.instantiates rc">[TU]5 g)z,Replay the CUDA work captured by this graph.N)r'replayr5s rr;CUDAGraph.replays  rc">[TU]5 g)z1Delete the graph currently held by this instance.N)r'resetr5s rr>CUDAGraph.resets   rc >[TU]5$)zReturn an opaque token representing the id of this graph's memory pool. This id can optionally be passed to another graph's ``capture_begin``, which hints the other graph may share the same memory pool. )r'r.r5s rr.CUDAGraph.pools w|~rc >[TU]5$)z/Enable debugging mode for CUDAGraph.debug_dump.)r'enable_debug_moder5s rrCCUDAGraph.enable_debug_modesw(**rc">[TU]U5$)z Arguments: debug_path (required): Path to dump the graph to. Calls a debugging function to dump the graph if the debugging is enabled via CUDAGraph.enable_debug_mode() )r' debug_dump)r1 debug_pathr+s rrFCUDAGraph.debug_dumpsw!*--rc >[TU]5$)amReturns the underlying cudaGraph_t. ``keep_graph`` must be True. See the following for APIs for how to manipulate this object: `Graph Managmement `_ and `cuda-python Graph Management bindings `_ )r'raw_cuda_graphr5s rrJCUDAGraph.raw_cuda_graphs w%''rc >[TU]5$)aReturns the underlying cudaGraphExec_t. ``instantiate`` must have been called if ``keep_graph`` is True, or ``capture_end`` must have been called if ``keep_graph`` is False. If you call ``instantiate()`` after ``raw_cuda_graph_exec()``, the previously returned cudaGraphExec_t will be destroyed. It is your responsibility not to use this object after destruction. See the following for APIs for how to manipulate this object: `Graph Execution `_ and `cuda-python Graph Execution bindings `_ )r'raw_cuda_graph_execr5s rrMCUDAGraph.raw_cuda_graph_execs w*,,rr)F)r*boolreturnr )Nglobal)r.Optional[_POOL_HANDLE]r/strrPNonerPrTrPr)rGrSrPrT)rPint)__name__ __module__ __qualname____firstlineno____doc__r(r0r4r8r;r>r.rCrFrJrM__static_attributes__ __classcell__)r+s@rrrFsr400NVP*PGJP PP* +.(--rrc^\rSrSr%SrSrS\S'S S SjjrS SjrS Sjr S r g)ramContext-manager that captures CUDA work into a :class:`torch.cuda.CUDAGraph` object for later replay. See :ref:`CUDA Graphs ` for a general introduction, detailed use, and constraints. Arguments: cuda_graph (torch.cuda.CUDAGraph): Graph object used for capture. pool (optional): Opaque token (returned by a call to :func:`~torch.cuda.graph_pool_handle()` or :meth:`other_Graph_instance.pool()`) hinting this graph's capture may share memory from the specified pool. See :ref:`Graph memory management`. stream (torch.cuda.Stream, optional): If supplied, will be set as the current stream in the context. If not supplied, ``graph`` sets its own internal side stream as the current stream in the context. capture_error_mode (str, optional): specifies the cudaStreamCaptureMode for the graph capture stream. Can be "global", "thread_local" or "relaxed". During cuda graph capture, some actions, such as cudaMalloc, may be unsafe. "global" will error on actions in other threads, "thread_local" will only error for actions in the current thread, and "relaxed" will not error on actions. Do NOT change this setting unless you're familiar with `cudaStreamCaptureMode `_ .. note:: For effective memory sharing, if you pass a ``pool`` used by a previous capture and the previous capture used an explicit ``stream`` argument, you should pass the same ``stream`` argument to this capture. .. warning:: This API is in beta and may change in future releases. .. _cudaStreamCaptureMode: https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__STREAM.html#group__CUDART__STREAM_1g9d0535d93a214cbf126835257b16ba85 NOptional[torch.cuda.Stream]default_capture_streamcxURRc-[RR 5URlUcSOU4UlUbUOURRUlUR ce[RRUR 5UlXl X@l g)Nr) r+rbr!r"Streamr.capture_streamstream stream_ctx cuda_graphr/)r1rhr.rfr/s r__init__graph.__init__s >> 0 0 849JJ4E4E4GDNN 1,BTG (Fdnn.S.S ""...**++D,?,?@$"4rc[RR5 [RRR (a[ R"5 [RR5 URR5 URR"URSUR06 g)Nr/)r!r" synchronizecompilerconfigforce_cudagraph_gcgccollect empty_cacherg __enter__rhr0r.r/)r1s rrsgraph.__enter__s{  >> 3 3 JJL   !!# %% YY  $66  rcjURR5 URR"U6 gr&)rhr4rg__exit__)r1argss rrvgraph.__exit__ s$ ##%   $'r)r/rerhr.rg)NNrQ)rhrr.rRrfrar/rSrU)rwobjectrPrT) rXrYrZr[r\rb__annotations__rirsrvr]rrrrrsU:;?7> (,.2"* 55%5, 5  50 2(rrtorch.nn.Module.r_ModuleOrCallablecgr&r callables sample_argsnum_warmup_itersallow_unused_inputr.s rrrsrcgr&rr~s rrrs%(rc [R"5(a%[R"5(a [S5eSn[ U[ 5(d+SnU4n[ R"[ [S4U54nO-[ R"[ [ [S4S4U5n/n[X5GH up[ U[RR5(a~[UR5S:Xa2[UR5S:Xa[UR5S:XdS5e[!SUR#555(dS5e[R$R&R("U 6n UR+[ U 55 [!S U 55(aGMS 5e UV s/sHn [U 5PM n n UVs/sHGn[ U[RR5(a[ UR-55OS PMI n n[/[U55V s/sH n X}X-PM nn [/[U55Vs/sH!n[R0R35PM# nn[/[U55Vs/sH!n[R0R35PM# nnUc [55OUn[R0R75 [R0R9[R0R;55 [XU5Hunn nS unnn[/U5Hn[R$R&R=U"U 65n[ SU55n[U5S:dMR[R>RAU[ SU55[ SU55SUS9nM UUU4HnAM M S S S 5 [R0R75 /n/n[XU5Hunn n0RCUUS9 U"U 6nS S S 5 [R$R&REW5unnUR+[ U55 UR+U5 M /n/n [[GU5[GU5[GU55GHunn!n"[ SU!55n#[ SU!55nS n[U5S:ah[R0RCU"US9 [R>RAU[ SU55[ SU#55SUS9nS S S 5 /n$Sn%UHCn&U&RH(aUbU$R+UU%5 U%S- n%M2U$R+S 5 ME [ U$5n$UR+U#5 U R+U$5 GM" URK5 U RK5 SSjn'/n([MU5Hun nU'"UU UU XXUU XUU UU U U 5 n)[ U[RR5(aFSSjn*U*"UURNU)URP5Ul(U(R+U5 MU(R+U)5 M U(aU(S$[ U(5$s sn fs snfs sn fs snfs snf!,(df  GNC=f!,(df  GN=f!,(df  GN=f)a Accept callables (functions or :class:`nn.Module`\ s) and returns graphed versions. Each graphed callable's forward pass runs its source callable's forward CUDA work as a CUDA graph inside a single autograd node. The graphed callable's forward pass also appends a backward node to the autograd graph. During backward, this node runs the callable's backward work as a CUDA graph. Therefore, each graphed callable should be a drop-in replacement for its source callable in an autograd-enabled training loop. See :ref:`Partial-network capture` for detailed use and constraints. If you pass a tuple of several callables, their captures will use the same memory pool. See :ref:`Graph memory management` for when this is appropriate. Arguments: callables (torch.nn.Module or Python function, or tuple of these): Callable or callables to graph. See :ref:`Graph memory management` for when passing a tuple of callables is appropriate. If you pass a tuple of callables, their order in the tuple must be the same order they'll run in the live workload. sample_args (tuple of Tensors, or tuple of tuples of Tensors): Samples args for each callable. If a single callable was passed, ``sample_args`` must be a single tuple of argument Tensors. If a tuple of callables was passed, ``sample_args`` must be tuple of tuples of argument Tensors. num_warmup_iters (int): The number of warmup iterations. Currently, ``DataDistributedParallel`` needs 11 iterations for warm up. Default: ``3``. allow_unused_input (bool): If False, specifying inputs that were not used when computing outputs (and therefore their grad is always zero) is an error. Defaults to False. pool (optional): Token (returned by :func:`~torch.cuda.graph_pool_handle` or :meth:`other_Graph_instance.pool()`) that hints this graph may share memory with the indicated pool. See :ref:`Graph memory management`. .. note:: The ``requires_grad`` state of each Tensor in ``sample_args`` must match the state that's expected for the corresponding real input in the training loop. .. warning:: This API is in beta and may change in future releases. .. warning:: ``sample_args`` for each callable must contain only Tensors. Other types are not allowed. .. warning:: Returned callables do not support higher order differentiation (e.g., double backward). .. warning:: In any :class:`~torch.nn.Module` passed to :func:`~make_graphed_callables`, only parameters may be trainable. Buffers must have ``requires_grad=False``. .. warning:: After you pass a :class:`torch.nn.Module` through :func:`~make_graphed_callables`, you may not add or remove any of that Module's parameters or buffers. .. warning:: :class:`torch.nn.Module`\s passed to :func:`~torch.cuda.make_graphed_callables` must not have module hooks registered on them at the time they are passed. However, registering hooks on modules *after* passing them through :func:`~torch.cuda.make_graphed_callables` is allowed. .. warning:: When running a graphed callable, you must pass its arguments in the same order and format they appeared in that callable's ``sample_args``. .. warning:: The automatic mixed precision is supported in :func:`~torch.cuda.make_graphed_callables` only with disabled caching. The context manager `torch.cuda.amp.autocast()` must have `cache_enabled=False`. z_make_graphed_callables does not support the autocast caching. Please set `cache_enabled=False`.FT.rzModules must not have hooks registered at the time they are passed. However, registering hooks on modules after passing them through make_graphed_callables is allowed.c3<# UHoRSLv M g7f)FN requires_grad.0bs r )make_graphed_callables..sEA%/szIn any :class:`~torch.nn.Module` passed to :func:`~make_graphed_callables`, only parameters may be trainable. All buffers must have ``requires_grad=False``.c3V# UHn[U[R5v M! g7fr&) isinstancer!r )rargs rrrsHKS:c5<<00Ks')zfIn the beta API, sample_args for each callable must contain only Tensors. Other types are not allowed.rN)NNNc3J# UHoR(dMUv M g7fr&rros rrrs$K1??QQ# #c3J# UHoR(dMUv M g7fr&rris rrrs%';!AA';rc3r# UH-oR(dM[R"U5v M/ g7fr&rr! empty_likers rrrs(+9@AOO/E,,Q//s77)outputsinputs grad_outputs only_inputs allow_unused)r.c3r# UH-oR(a[R"U5OSv M/ g7fr&rrs rrrs'$ FT??E  Q  \rSrSr\SUUUU4Sjj5r\\RRRSUUU4Sjj55r Sr g)Omake_graphed_callables..make_graphed_autograd_function..Graphedic>[T5HBnTUR5XR5:wdM,TURX5 MD TR5 [ T[ 5(de[ ST55$)Nc3@# UHoR5v M g7fr&detachrs rrjmake_graphed_callables..make_graphed_autograd_function..Graphed.forward..s@AXXZZs)rangedata_ptrcopy_r;rtuple)ctxrr fwd_graph len_user_argsstatic_input_surfacestatic_outputss rforwardWmake_graphed_callables..make_graphed_autograd_function..Graphed.forwards|}-A+A.779VY=O=O=QQ,Q/55fi@.  "!.%8888@@@@rcH>[U5[T5:Xde[TU5H?up#UcM UR5UR5:wdM.URU5 MA TR 5 [ T[ 5(de[ ST55$)Nc3L# UHnUbUR5OUv M g7fr&rrs rrkmake_graphed_callables..make_graphed_autograd_function..Graphed.backward..*s(0#$-AHHJQ6/s"$)lenziprrr;rr)rgradsggrad bwd_graphstatic_grad_inputsstatic_grad_outputss rbackwardXmake_graphed_callables..make_graphed_autograd_function..Graphed.backwards5zS)<%===="#6>GA}::<4==?:GGDM ?   """4e<<<<0rrN)rryrr rPtuple[Tensor, ...])rryrr rPr) rXrYrZr[ staticmethodrr!autogradfunctiononce_differentiablerr])rrrrrrrsrGraphedrsN  A A A ^^ $ $ 8 8  9 rrc>[RRR"U6nTR"[ U5T-6n[RRR UT5$r&)r!utils_pytreearg_tree_leavesapplyrtree_unflatten) user_argsflatten_user_argsoutr module_paramsoutput_unflatten_specs rfunctionalizedVmake_graphed_callables..make_graphed_autograd_function..functionalized0sW!& 3 3 C CY O --%(9":]"JLC;;&&55c;PQ Qr)rryrPry)r!rFunction) rrrrrrrrrrrs ````````` @rmake_graphed_autograd_function>make_graphed_callables..make_graphed_autograd_functions4  enn-- B R Rrc$^^^^SUUUU4SjjnU$)NcD>TRT:XaT"U0UD6$T"U0UD6$r&)training)r user_kwargsfuncgraph_training_stategraphedorig_fwds rnew_fwdEmake_graphed_callables..make_graphed_forward..new_fwdQs2}}(<<& A[AA'BkBBr)rz_P.argsrz _P.kwargsrPrr)rrrrrs```` rmake_graphed_forward4make_graphed_callables..make_graphed_forwardKs CCr)rrrrrztuple[torch.nn.Parameter, ...]rrWrztorch.utils._pytree.TreeSpecrrrrrztuple[Optional[Tensor], ...]rrrPzCallable[..., object]) rr{rrOrCallable[_P, _R]rrrPr))r!is_autocast_enabledis_autocast_cache_enabled RuntimeErrorrrtypingcastr rnnModuler_backward_hooks_forward_hooks_forward_pre_hooksallbuffersrrrappend parametersrr"rrrlrfrd tree_leavesrrr tree_flattenreversedrreverse enumeraterr)+rrrrr.just_one_callable _sample_argsflatten_sample_argscrw flatten_argper_callable_len_user_argsper_callable_module_paramsr"per_callable_static_input_surfaces_ fwd_graphs bwd_graphsmempoolrr grad_inputsr outputs_gradvper_callable_static_outputs"per_callable_output_unflatten_specr func_outputsflatten_outputsspec per_callable_static_grad_outputsper_callable_static_grad_inputsrrrrgrad_idxrrretrrs+ rrr(sQR   ""u'F'F'H'H m   i ' ' L  E&#+$6 DF {{5vs{);S)@#A;O y/ a ) )A%%&!+(()Q.,,-2  ]  3EEEE - E kk))994@ ""5#56HKHHH  Z H#009L!L8K#d)8K!L"A",Auxx!?!?allnRG" s9~&*&A !;!>>&'* 38I2GH2GQ%**&&(2GJH27I2GH2GQ%**&&(2GJH%)\!tG  JJ   5::,,. /03 %G1 ,D$,2B .K,+,++--99$+F$$K$KK |$q("'.."5"5 ,$%';% &++9@+&%)%7#6 #K -|[9:'1  0. JJ#%)+&!$Yj!IdI ZZ  ig  6;L7!& 3 3 @ @ N#**5+AB*11$7 "J(*$&(#;>34,-<7ni$$ FT$  JJJ  | q !!)'!:#nn11( T,@ TT!&&W2E&W!W $!3 2 ; 'C  [%<"))+h*?@A "))$/ ( ##56(//0CD'../ABK<P%,,.#++-44464 4 < 4 1 4+4:4/4 4n$&CY'40 qM qM & ) & ) .q 1 . 1 ' * ,Q / +A .   dEHHOO , , % &* * +   "   0dmmWdllDL JJt  JJw I(L1v :W"M"* IH 0 /B7 60;:sL\?,A]] =(]=(]A3]A]7]*'A]< ]'* ]9 < ^ )rPrOrV)FN) rr|rrrrWrrOr.rRrPr|) rtuple[_ModuleOrCallable, ...]rztuple[tuple[Tensor, ...], ...]rrWrrOr.rRrPr) r7Union[_ModuleOrCallable, tuple[_ModuleOrCallable, ...]]rz9Union[tuple[Tensor, ...], tuple[tuple[Tensor, ...], ...]]rrWrrOr.rRrPr)' __future__rrprcollections.abcrrrrrr typing_extensionsr r r r!r torch.cudar_utilsr__all__rrhasattr_C__dict__torch._Crrrrrrrryr|rzrrrrrs" $FF66 '   T]t_uxx*++&1,&?EHHl#.9:N.OEHH*+:E(;EHH67,9q-##q-hS(S(n %%6f8M%MN9N $#'  #  !     $#' (,(/(( ( ! ( # ( ($#' }F}J}} } ! } = }r