j<*SrSSKrSSKrSSKrSSKJrJrJr SSKJ r SSK J r J r SSK Jr SSKJrJr SSKJrJrJr SS KJr SSKrSS KJr S r\"S 5SS S:Xa\R:R<rO \R@r\"\"5r#Sr$SSjr%Sr&SSjr'\RP"\&\ RR5 \ RR\ l*\&\ l)\+"\ S5(a\'\ l,\"SS\55r-\"SS55r.g!\!a SrNf=f)a3PyTorch compatible DataLoaders Essentially we extend PyTorch DataLoader by adding the ability to save the data loading state, so that a checkpoint may be saved in the middle of an epoch. Example ------- >>> import torch >>> from speechbrain.utils.checkpoints import Checkpointer >>> # An example "dataset" and its loader >>> dataset = torch.randn(10, 1) >>> dataloader = SaveableDataLoader(dataset, num_workers=3) >>> # Setup the checkpointer: >>> tmpdir = getfixture("tmpdir") >>> checkpointer = Checkpointer(tmpdir, {"dataloader": dataloader}) >>> # Iterate: >>> for i, data_point in enumerate(dataloader): ... # Here you would process the data: ... rainfall_amount_prediction = data_point * 4.0 ... # Now, imagine the experiment gets killed on the fifth batch: ... if i == 4: ... break ... # Luckily, you had just saved a checkpoint: ... if i == 3: ... _ = checkpointer.save_checkpoint(end_of_epoch=False) >>> # So when you restart the experiment: >>> new_dataloader = SaveableDataLoader(dataset, num_workers=3) >>> new_checkpointer = Checkpointer(tmpdir, {"dataloader": new_dataloader}) >>> _ = new_checkpointer.recover_if_possible() >>> # The dataloader fast-forwards to the position where we left off: >>> assert next(iter(new_dataloader)) == dataset[4] Authors: * Aku Rouhe 2020 N) DataLoaderDistributedSamplerIterableDataset)_BaseDataLoaderIter)BatchsizeGuesser PaddedBatch)DynamicItemDataset)DistributedSamplerWrapperReproducibleRandomSampler)mark_as_loader mark_as_saverregister_checkpoint_hooks) get_logger)versionT webdatasetz0.1.FcURSS5nURSS5nU(a[U[5(duURSS5nUb[UUUUS9nSUS'XCS'U$URS5c[ UUUS9nSUS'XCS'U$[URSS5US 9nXCS'U$U(a*[U[5(a[ R S 5 U$) akPrepare loader_kwargs for DDP when necessary. Arguments --------- distributed_launch : bool DDP flag rank : int node rank in DDP dataset : Dataset The dataset to make a DataLoader for. loader_kwargs : dict Keyword args to DataLoader, see PyTorch DataLoader for options. Returns ------- loader_kwargs augmented keyword args to DataLoader samplerNshuffleF drop_last)rankrr batch_sampler)rr)rzDCannot automatically solve distributed sampling for IterableDataset.)get isinstancerr rloggerwarning)distributed_launchrdataset loader_kwargsrrrs ګC:\Users\johnn\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\speechbrain/dataio/dataloader.pydistributed_loader_specificsr!Ls , 40G 51G*Wo"F"F!%%k59   /# G(-M) $'.) $. -   / 7(#G(-M) $'.) $ 0!!/48G.5/ *   7O D D #  c SU;a[U[5(a [US'URSS5(aVURS5b [ S5e[ [ RRSS55n[XS 9nXBS'US [(a [U[5(a S U;aSUS '[U[5(a [U40UD6nO [U40UD6nUb [XQ5nU$) aMakes a basic DataLoader with SpeechBrain defaults. For DynamicItemDatasets (which return dicts), use PaddedBatch as the default collate_fn. Shuffling gets implemented by ReproducibleRandomSampler. If the Dataset is not an IterableDataset, the DataLoader is a SaveableDataLoader. If the Dataset is a webdataset.dataset.Composable, set default batch_size = None. Can also loop over the underlying dataloader continuously, and stop iterations at nominal epoch lengths. Arguments --------- dataset : Dataset The dataset to make a DataLoader for. looped_nominal_epoch : None, int If an integer is given, loop the underlying DataLoader infinitely and set a nominal epoch length in batches (or whatever the DataLoader yields). **loader_kwargs : dict Keyword args to DataLoader, see PyTorch DataLoader for options. Returns ------- DataLoader If looped_nominal_epoch is None LoopedLoader If looped_nominal_epoch is not None collate_fnrFrNz?Cannot specify both shuffle=True and a sampler in loader_kwargsSB_GLOBAL_SEEDi&l!)seed batch_size)rr rr ValueErrorintosenvironr WDS_AVAILABLE WDS_CLASSrrSaveableDataLoader LoopedLoader)rlooped_nominal_epochrr&r dataloaders r make_dataloaderr2sJ=(Z#..'2 l#E**   Y ' 3+ 2::>>"2I>?+G?#*i ) $  w * *  -&* l#'?++9=9 'A=A '!*C r"cvUR"U/UQ70UD6 [US5(aRURbD[UR5H#n[ UR 5 US-Ul M% SUlggg![ a3 SnSURS3 US3 [R"U5  MFf=f)N_speechbrain_recovery_skip_toz/Tried to fast-forward Sampler after checkpoint z recovery by  z! indices. Ignoring this mismatch.) __old_init__hasattrr4rangenext _sampler_iter StopIterationwarningswarn _num_yielded)selfloaderargskwargsiMSGs r __new_initrFsf.t.v.788  0 0 <v;;[TU]"U0UD6 [UR[5(a[ R S5 SUlSUlg)NzkSaveableDataLoader cannot save the position in an IterableDataset. Save the position on the dataset itself.) super__init__rrrrrr4_speechbrain_iterator)r@rBrC __class__s r rRSaveableDataLoader.__init__sJ $)&) dllO 4 4 NNL .2*%)"r"c0>[TU]5nXlU$N)rQ__iter__rS)r@iteratorrTs r rXSaveableDataLoader.__iter__&s7#%&."r"c6[UR[5(a[R S5 UR cSnOUR R n[USSS9nUR[U55 SSS5 g!,(df  g=f)NzWarning again: a checkpoint was requested on SaveableDataLoader, but the dataset is an IterableDataset. Cannot save the position in an IterableDataset. Not raising an error; assuming that you know what you're doing.wutf-8encoding) rrrrrrSr?openwritestr)r@pathto_savefos r _speechbrain_save$SaveableDataLoader._speechbrain_save1ss dllO 4 4 NNF   % % -G00==G $g ." HHS\ "/ . .s &B  BcURb[RS5 gU(ag[USS9nUR 5nU[ S5:Xa SSS5 g[ U5UlSSS5 g!,(df  g=f)NzSaveableDataLoader was requested to load a checkpoint, but the DataLoader has already been iterated. The DataLoader file will be ignored. This is normal in evaluation, when a checkpoint is loaded just to retrieve the best model.r]r^)rSrdebugr`readrbr)r4)r@rc end_of_epochfisaveds r _speechbrain_load$SaveableDataLoader._speechbrain_loadAst  % % 1 LL:     $ )RGGIED ! * ) 69Z2 * ) )s!A: A:: B)rSr4) __name__ __module__ __qualname____firstlineno____doc__rRrXr rfr rn__static_attributes__ __classcell__)rTs@r r.r.s<$*  # #@@r"r.cX\rSrSrSrS SjrSrSrSr\ S5r \ S S j5r S r g) r/iYa\Loops an underlying iterable indefinitely, with nominal epoch lengths This is useful for working with IterableDatasets, and particularly webdataset-style loading. We recommend using ``.repeat()`` on the webdataset IterableDataset instance, so that the underlying dataloader naturally continues for ever. Arguments --------- loader : iterable A DataLoader or other iterable that is looped repeatedly. epoch_length : int The length of the nominal epoch. After this many steps, raises StopIteration batchsize_fn : callable Function for determining batch size, default ``BatchsizeGuesser`` NczXlSUlX lSUlSUlSUlUc[ 5UlggrH)rArY epoch_lengthstep total_steps total_samplesr batchsize_fn)r@rAryr}s r rRLoopedLoader.__init__msA  (    0 2D  r"cTURc[UR5UlU$rW)rYrIrAr@s r rXLoopedLoader.__iter__ws! ==  -DM r"cURUR:afU=RS- slU=RS- sl[UR5nU=RURU5- slU$SUl[ e![ a2 [ UR5Ul[UR5nNnf=f)Nr6r) rzryr{r:rYr<rIrAr|r})r@batchs r __next__LoopedLoader.__next__|s 99t(( ( IINI    !  ,T]]+   $"3"3E": : LDI ! , $T[[ 1 T]]+ ,sB9C  C cUR$rW)ryrs r __len__LoopedLoader.__len__s   r"c[USSS9n[URUS9 [URUS9 [URUS9 SSS5 g!,(df  g=f)zSaves the needed information.r\r]r^)fileN)r`printrzr{r|)r@rcres r saveLoopedLoader.savesL$g ." $))" % $"" , $$$2 ./ . .s =A A c[USS9n[UR5R55Ul[UR5R55Ul[UR5R55UlU(d1URS:Xa!UR S:aURUlSSS5 g!,(df  g=f)zLoads the needed information.r]r^rN)r`r)readlinestriprzr{r|ry)r@rcrkrls r loadLoopedLoader.loads$ )RBKKM//12DI"2;;=#6#6#89D !$R[[]%8%8%:!;D DIINt7G7G!7K!-- * ) )s B=C C)r}ryrYrArzr|r{rW)T)rprqrrrsrtrRrXrrr rr rrur"r r/r/YsC$3  !// . .r"r/rW)F)/rt functoolsr*r=torch.utils.datarrrtorch.utils.data.dataloaderrspeechbrain.dataio.batchrrspeechbrain.dataio.datasetr speechbrain.dataio.samplerr r speechbrain.utils.checkpointsr r rspeechbrain.utils.loggerrrwdsimportlib_metadatarr,r Composabler- DataPipeline ImportErrorrprr!r2rFrMupdate_wrapperrRr7r8rNr.r/rr"r rs:#J LL;B9 0 *M|Qq!V+KK** $$  H ?DHd4,N %8%A%AB#6#?#? ) ))!,M@M@M@`J.J.J.iMs1D8 DDD