ٓ,jaPSrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSK r SSK r SSK r SSK r SSKrSSKJr SSKJr SS\ RR(4Sjr"SS5r"S S 5rS rSS jrS rSrSrSrSrSrSr Sr!Sr"SSjr#SSjr$Sr%Sr&SSjr'g)z~This library gathers utilities for hyperpyyaml loading Authors * Peter Plantinga 2020 * Aku Rouhe 2020 * Jianchen Li 2022 N)StringIO)RepresenterErrorTcj[XU5nURS[S9 [R"S5nUR SUSS9 UR S[5 UR S[5 UR S[5 UR S [5 S [RRRlS [ RRRRl[R""XS 9nS [RRRlS [ RRRRlUR%5Vs/sHofR'S 5(dMUPM nnUHnXX M U$s snf)aThis function implements the HyperPyYAML syntax The purpose for this syntax is a compact, structured hyperparameter and function definition. This function implements a few extensions to the yaml syntax, listed below. **PyYAML complex tag shortcuts** Part of our clean structured hyperparameter interface is being able to specify python objects easily and cleanly. This is possible with native YAML using the following syntax: .. code-block:: yaml alignment_saver: !!python/object/new:speechbrain.data_io.TensorSaver kwargs: {save_dir: results/asr/ali} However, due to the extensive use within speechbrain yaml files, we have added a shortcut for this that has the following syntax: .. code-block:: yaml alignment_saver: !new:speechbrain.data_io.TensorSaver save_dir: results/asr/ali In this example, the alignment_saver will be an instance of the ``TensorSaver`` class, with ``'exp/asr/ali'`` passed to the ``__init__()`` method as a keyword argument. This is equivalent to: .. code-block:: python import speechbrain.data_io.data_io alignment_saver = speechbrain.data_io.TensorSaver( save_dir='exp/asr/ali' ) We have also implemented a few more shortcuts::: !!python/name: => !name: !!python/module: => !module: !!python/object/apply: => !apply: **References and copies** Allows internal references to any node in the file. Any node with tag ``!ref`` will create an object reference to the yaml object at the ```` location within the yaml itself, following reference chains. .. code-block:: yaml output_folder: results/asr alignment_saver: !new:speechbrain.data_io.TensorSaver save_dir: !ref Strings values are handled specially: references are substituted but the rest of the string is left in place, allowing filepaths to be easily extended: .. code-block:: yaml output_folder: results/asr alignment_saver: !new:speechbrain.data_io.TensorSaver save_dir: !ref /ali # results/asr/ali A more complex example for demonstration purposes: .. code-block:: yaml key1: {a: !new:object {arg1: 1}} key2: !ref Here, ``key2`` will contain a reference to the ``a`` object, so changing ``a.arg1`` will also change ``key2.arg1``. If you need a deep copy of the object instead of a shallow reference, you can use a similar syntax with the tag ``!copy``. For example: .. code-block:: yaml key1: {a: !new:object {arg1: 1}} key2: !copy These will also implement very basic arithmetic, so: .. code-block:: yaml key1: 1 key2: !ref + 3 # this is 4 **Tuples** One last minor enhancement is an implicit tuple resolver. Passing a string value of ``(3, 4)`` will be given a tag of ``!tuple`` which is then interpreted as a tuple. Arguments --------- yaml_stream : stream A file-like object or string from which to read. overrides : mapping or str A set of overrides for the values read from the stream. As yaml implements a nested structure, so can the overrides. See `speechbrain.utils.data_utils.recursive_update` overrides_must_match : bool Whether an error will be thrown when an override does not match a corresponding key in the yaml_stream. return_dict : bool Whether to return a dictionary rather than the default namespace. loader : Loader Use to parse the yaml_stream, could be `ruamel.yaml.Loader`, `yaml.Loader`, etc. Returns ------- hparams : dict Reflects the structure of ``yaml_stream``. Example ------- >>> yaml_string = """ ... a: 3 ... thing: !new:collections.Counter ... b: !ref ... """ >>> params = load_hyperpyyaml(yaml_string) >>> params["thing"] Counter({'b': 3}) z!tuple)tag constructorz^\(.*\)$()first!new:!name:!module:!apply:)TLoaderF__)resolve_referencesadd_constructor _make_tuplerecompileadd_implicit_resolveradd_multi_constructor_construct_object_construct_name_construct_module_apply_functionyamlrBaseConstructorconstruct_object __defaults__ruamelloadkeys startswith) yaml_stream overridesoverrides_must_matchloader tuple_patternhparamsk removal_keyskeys ڞC:\Users\johnn\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\hyperpyyaml/core.pyload_hyperpyyamlr/s[F%[=QRK x[AJJ{+M   = D   *;<   ?;   ->?   O<FD$$55BMFKK++<<Iii 3GFD$$55BMFKK++<<I '||~D~!d1CA~LD L N Es >F0F0c2\rSrSrSrSrSr\S5rSr g)RefTagzClass for dumping !ref tags to yaml Arguments --------- ref_str : str String including yaml keys in `` notation Example ------- See ``dump_hyperpyyaml`` !refcXlgNref_str)selfr7s r.__init__RefTag.__init__s cNURURUR5$r5)represent_scalaryaml_tagr7cls representernodes r.to_yamlRefTag.to_yamls++CLL$,,GGr;r6N) __name__ __module__ __qualname____firstlineno____doc__r>r9 classmethodrC__static_attributes__r;r.r1r1s) HHHr;r1c,\rSrSrSrSr\S5rSrg) PlaceholderzVClass for dumping !PLACEHOLDER tags to yaml Example ------- See ``dump_hyperpyyaml`` !PLACEHOLDERc:URURS5$)N)r=r>r?s r.rCPlaceholder.to_yamls++CLL"==r;rLN) rErFrGrHrIr>rJrCrKrLr;r.rNrNs H>>r;rNc$[RR5nURR [ [ R 5 URR [[R 5 UR"X/UQ70UD6 g)aDump yaml including placeholder and reference tags. Arguments --------- yaml_tree : dict An object to dump output_stream : stream A file stream for putting the yaml *args, **kwargs Arguments to forward to ruamel.yaml.YAML().dump() Example ------- >>> to_yaml = {'a': Placeholder(), 'b': RefTag('')} >>> stringio = StringIO() >>> dump_hyperpyyaml(to_yaml, stringio) >>> stringio.getvalue() 'a: !PLACEHOLDER\nb: !ref \n' N) r!rYAMLrAadd_representerr1rCrNdump) yaml_tree output_streamargskwargs ruamel_yamls r.dump_hyperpyyamlr]sd(++""$K++FFNNC++K9L9LMY???r;cSn[US5(aF[RR[RR UR 55n[ RR5nURU5nUb@[U5S:a1[U[5(aURU5n[XQUS9 [!SXUU5 [#5nUR%XP5 UR'S5 U$![a [SU5ef=f![(a8nUR*SS-4UlUR-UR.5eSnAff=f)aResolves inter-document references, a component of HyperPyYAML. Arguments --------- yaml_stream : stream A file-like object or string with the contents of a yaml file written with the HyperPyYAML syntax. overrides : mapping or str Replacement values, either in a yaml-formatted string or a dict. overrides_must_match : bool Whether an error will be thrown when an override does not match a corresponding key in the yaml_stream. This is the opposite default from ``load_hyperpyyaml`` because ``resolve_references`` doesn't need to be as strict by default. Returns ------- stream A yaml-formatted stream with all references and overrides resolved. Example ------- >>> yaml_string = """ ... constants: ... a: 3 ... b: !ref ... """ >>> overrides = {'constants': {'a': 4}} >>> resolve_references(yaml_string, overrides).getvalue() 'constants:\n a: 4\n b: 4\n' Nnamer) must_matchzLThe structure of the overrides doesn't match the structure of the document: rootz$. Please use the !apply tag instead.)hasattrospathdirnamerealpathr_r!rrUr"len isinstancestrrecursive_update TypeError ValueError_walk_tree_and_resolverrWseekrrZwith_traceback __traceback__)r%r&r' file_pathr\previewes r.rrsDBI{F##GGOOBGG$4$4[5E5E$FG ++""$K{+GY!!3 i % %#((3I  W

>> deref('constants[a][b]', {'constants': {'a': {'b': 'c'}}}) 'c' N.r)maxsplitrurvzThe reference "%s" is not validz!apply:getattr)suffix) splitstriprlcopydeepcopyr!rcomments CommentedSeq yaml_set_ctagTag)refr{r|attrbranchpartrBs r.derefr8s0 D czIIcAI. F #zz#  >DE E }}V$$ {{##002   KKOO#3O 4  Mr;c[R"S5n[U[5(aUR U5(dU$[ U5S:aXSS;a [ SU5eURU5(a,[URS5X#5nX/- n[XQX#5$URU5nXVs/sHowSPM sn- nX#4SjnURX5n [XX#5n[U5$s snf)a)Resolve a reference to a value, following chained references Arguments --------- reference : str a string containing '' in it where x[y] refers to a scalar node in the file. reference_list : list list of prior references in the chain, in order to catch circular references. full_tree : dict the dictionary in which to find all references and their values. copy_mode : bool Whether to perform a deep copy of the referenced node, rather than a shallow reference to the same object. Returns ------- scalar The dereferenced value, with possible string interpolation and arithmetic parsing. Example ------- >>> tree = {'a': 3, 'b': 'x', 'c': '', 'd': '/', 'e': '/'} >>> recursive_resolve('', [], tree) 1.0 >>> recursive_resolve('', [], tree) 'x/x' z<[^>]*>rNzCircular reference detected: <>rcL[[USRS5XS95$)Nrr)r{r|)rirr)xrr|s r. replace_fn%recursive_resolve..replace_fns!51D)TOPPr;)rrrhrisearchrgrl fullmatchrrrfindallsubparse_arithmetic) ryrzr{r|reference_finderrmatchesmatchrrs r.rrlsBzz*- i % %-=-D-DY-O-O >Q9qr0B#B8.II!!),,iood+YB+%  MM&&y1GW5WEQxW55N%Q   z 5C!#yLI I &&6s>C?c[[R"USS9R5$![[ [ 4a Us$f=f)aSParses simple arithmetic operations in references Adapted from https://stackoverflow.com/a/9558001/1761970 Arguments --------- reference_string : str A string with references and possible arithmetic operations. Returns ------- str Result of parsing and applying the arithmetic. Example ------- >>> parse_arithmetic('2 * 6') 12 eval)mode) _ast_evalastparsebodyrk SyntaxErrorKeyError)reference_strings r.rrs@( #3&AFFGG {H -  s'*AAc[R[R[R[R [R [R[R[R[R[R[R[R[R[R0n[ R"S:a,[%U[R&5(a UR($O+[%U[R*5(a UR,$[%U[R.5(aEU[1UR5"[3UR45[3UR655$[%U[R85(a1U[1UR5"[3UR:55$[=U5e)N))rAddopaddSubrMultmulDivtruedivFloorDivfloordivPowpowModmodsys version_inforhConstantrNumnBinOptyperleftrightUnaryOpoperandrk)rBopss r.rrs)   "&&  bkk   C 6! dCLL ) )::  * dCGG $ $66M$ ""4=!)DII"6 $**8MNN D#++ & &4=!)DLL"9::or;c 2[U[RR5(d[ SU35e[U[RR5(d[ SU35eUR 5Hup4[U[RR5(a#X0;a[ URU05U5 MQU(a4X0;a/[SUSUR5Vs/sHoUPM sn35eX@U'M gs snf)aSimilar function to `dict.update`, but for a nested `dict`. From: https://stackoverflow.com/a/3233356 If you have to a nested mapping structure, for example: {"a": 1, "b": {"c": 2}} Say you want to update the above structure with: {"b": {"d": 3}} This function will produce: {"a": 1, "b": {"c": 2, "d": 3}} Instead of: {"a": 1, "b": {"d": 3}} Arguments --------- d : dict mapping to be updated u : dict mapping to update with must_match : bool Whether to throw an error if the key in `u` does not exist in `d`. Example ------- >>> d = {'a': 1, 'b': {'c': 2}} >>> recursive_update(d, {'b': {'d': 3}}) >>> d {'a': 1, 'b': {'c': 2, 'd': 3}} z'Expected to update a mapping, but got: z.Expected a mapping to use for update, but got z Override 'z' not found in: N) rh collectionsabcMappingrkrrjgetrr#)dur`r+vr-s r.rjrjsN a00 1 1A!EFF a00 1 1HLMM  a00 1 1af QUU1b\1 - AJZs*:1668;T8CC8;T:UVW WaD rs 4  d6;;CUCUpfHH. > >@4CLOd  0&2 &*1h<'~ 4:4r;