+ WjZ< a0t$Rt^RIHt^RIt^RIt^RIt^RIt^RIt^RI t ^RI t ^RI t ^RI t ^RI t ^RIt^RIt^RIt^RIHt^RIHtHtHtHt^RIHtHt^RIHt^RIHt^RI H!t"H#t#^R I$H%t%H&t&^R I'H(t(H)t)H*t*H+t+H,t,H-t-H.t.H/t/H0t0H1t1H2t2^R I3H4t4^R I5H6t6H7t7H8t8^R I9H:t:H;t;^RIR?ltcR@RRARRBRRCRD/RERFlltdRGRHlteRIR/RJRKlltfRLRMltgRNROlthRPRQltiRRRRSlltjRTRUltkRVtlRWRXltmRYRZltnR[R\ltoR]R^ltpRR_R`lltqRaRbltrRcRdltsReRflttRgRhltuRRiRjlltvRkRlltwRmRnltxRoRpltyRqRrltzRsRtlt{RuRvlt|]APRw4Rx4t~RyRzlt]APR{4R|4tR}R~RR/tRtRRltRRlt]APR4R4tRRltRRlt]AEPR4RRRll4tRRlt]AEPR4RRl4t]AEPR4RRl4t]AEPR4RRl4t]AEPR4RRl4tRRltRRlt]AEPR4RRRll4tRRltRRltRRltR]R]/t]AEPR4RRl4t]AEPR4RRl4t]AEPR4RRRll4t]APR4R4t]APR4RRl4t]APR4RRRll4t]APR4RRl4t]AEPR4RRl4t]AEPR4RRl4tR#)uLocal-models dashboard routes — the desktop's window into the managed llama.cpp runtime. Every payload carries plain-language, pre-formatted facts the UI shows verbatim (what will this model do ON THIS MACHINE, how big is the download, what is the runtime doing), never raw internals. Long jobs follow the repo's job pattern: start-POST -> {job_id} -> GET poll with byte progress. ) annotationsN)Path)AnyCallableDictOptional) APIRouter HTTPException) BaseModel)run_in_threadpool)configweb_deps)_CONFIG_MUTATION_LOCK_config_profile_scope) binaries bootstrapcatalogcontext_policy estimatorgrowthhardware hf_browse load_progresspresets supervisor)spawn_context_thread)DownloadDownloadPausedSource)_state_endpoint child_query) split_partszdict[str, dict[str, Any]]_JOBS_RUNNINGz-\d{5}-of-\d{5}g @g?uCThe local server could not start — check the runtime is installedc&]tRt^At$RtR]R&RtR#)RuntimeInstallBodyN Optional[str]backend)__name__ __module__ __qualname____firstlineno__r'__annotations____static_attributes__r(WC:\Users\johnn\AppData\Local\hermes\hermes-agent\hermes_cli\web_routers\local_models.pyr%r%As !G]!r/r%c"]tRt^Et$R]R&RtR#)ModelDownloadBodystrmodel_idr(Nr)r*r+r,r-r.r(r/r0r2r2EMr/r2c"]tRt^It$R]R&RtR#) JobIdBodyr3job_idr(Nr5r(r/r0r8r8I Kr/r8c&]tRt^Mt$RtR]R&RtR#)QuickstartBodyN str | Noner4r()r)r*r+r,r4r-r.r(r/r0r<r<Ms Hjr/r<c"]tRt^Qt$R]R&RtR#)ServerActionBodyr3actionr(Nr5r(r/r0r?r?Qr:r/r?c"]tRt^Ut$R]R&RtR#)ModelEjectBodyr3r4r(Nr5r(r/r0rBrBUr6r/rBc"]tRt^Yt$R]R&RtR#)ModelActivateBodyr3r4r(Nr5r(r/r0rDrDYr6r/rDc,]tRt^]t$R]R&R]R&RtR#)BrowsedDownloadBodyr3repoz list[str]pathsr(Nr5r(r/r0rFrF]s I r/rFc"]tRt^bt$R]R&RtR#) SideloadBodyr3pathr(Nr5r(r/r0rJrJbs Ir/rJc V^8dQhRRRR/#)nfloatreturnr3r()formats"r0 __annotate__rRfs%%%3%r/cVR, R R2#)z.1fz GBi@r()rNs&r0 _human_gbrUfs7mC  $$r/c V^8dQhRRRR/#)rMtokensintrPr3r()rQs"r0rRrRjs  S S r/cVR, R2#)iKr()rWs&r0_k_labelr[jsn Q r/c V^8dQhRRRR/#)rMstatusrXprefixr3r()rQs"r0rRrRosRRRSRr/c#`"RxR# \dp\Y T 2R7ThRp?ii;i5i)zAMap any exception to ``HTTPException(status, f"{prefix}{exc}")``.N status_codedetail) Exceptionr )r]r^excs&& r0 _http_errorrens5R R#7GHcQRs. . +&+.warndebugc ,V^8dQhRRRRRRRRRR/#) rMfnzCallable[[], Any]defaultrrfr=rgrPr()rQs"r0rRrRws2   3  S] il r/cV!4# \dIpT'd\PY$4T'd\PTRR7TuRp?#Rp?ii;i)u``fn()`` or ``default`` on any exception — for garnish that must never 500. ``warn`` logs a warning with the exception (%s), ``debug`` a debug line with traceback; silent otherwise.T)exc_infoN)rcloggerwarningrg)rirjrfrgrds&&$$ r0_quietrowsEt   NN4 %  LLL . s A$AAAAc(V^8dQhRRRRRRRR/#)rMkindr3targetr4r=rPdict[str, Any]r()rQs"r0rRrRs(  s C :  r/cR\P!4PR,RVRVRVRRRRR R R R R ^R\P!4RR / p\;_uu_4V\ VR,&R R R 4V# +'giT#;i)r9:N Nrqrrr4r]runningphasestartingrb total_bytesN done_bytes started_aterror)uuiduuid4hextime _JOBS_LOCKr")rqrrr4jobs&&& r0_jobrs$**,""3'xH)"mT<L$))+W^`d  C "c(m  J  Js A99 B c(V^8dQhRRRRRRRR/#) rMsamplesz$collections.deque[tuple[float, int]]donerXtotalz int | NonerPztuple[float | None, int | None]r()rQs"r0rRrRs+?s#(Gr/c V^,wr4VR,^,V, pV\8gW8:dR#W, V, pV'd/V\^\\^W!, 4V, 443#VR3#)a_Transfer rate and remaining seconds from a trailing sample window. The rate is the slope across the window, not the last two ticks, so a burst reads as throughput rather than a spike. Anything that cannot be turned into an honest number (one sample, a window too short to divide by, a transfer that hasn't moved) reports unknown rather than a guess. NNN)_RATE_MIN_ELAPSEDmaxround)rrrfirst_at first_doneelapsedrates&&& r0 _rate_and_etarsu#1:Hbk!nx'G""d&8   (D SE#a"6"=>??? :r/c$V^8dQhRRRRRR/#)rMrrsrrXrPNoner()rQs"r0rRrRs!!!n!C!D!r/c\PVR,4pVfR#\P!4pVP R\ P !44pVPW134\V4^8d2W4^,^,, \8dVP4KA\WAVPR44wrVVf'VPRR4VPRR4R#WPR&W`R&R#)zRefresh the job's smoothed rate + ETA from the sample it just reported. Samples live on the running entry, not the job, so the wire payload stays the derived facts and the history dies with the transfer. r9Nrrz bytes_per_sec eta_seconds) r#getr monotonic setdefault collectionsdequeappendlen _RATE_WINDOWpopleftrpop)rrrvnowrretas&& r0 _record_raters ll3x=)G .. C  K,=,=,?@G NNC; g, sQZ]2\AgSWW]-CDID | &  t$#O Mr/c V^8dQhRRRR/#)rMrrsrPr()rQs"r0rRrRs>nr/c\V4p\PVR,/4pVPR4p\VRJ;'d%VP 4;'d VR,R8H4VR&\VRJ;'d;VR,R8H;'d'VR,\ 9;'dVR,'*4VR&VR,R8H;'dR V9VR &VR ,'d5\ ^d\VR ,VR ,, ^d,44VR &VR,R8wd%VPRR4VPRR4V#)r9pauseNr]rvpause_requestedrw can_pausepausedresume can_resumerzr{percentrr) dictr#rboolis_set_DOWNLOAD_PHASESminrr)routrvrs& r0 _job_viewrsE s)Cll3x="-G KK E!%t"3"e"e "e"e3x=\eKefCE-``#h-92L`` #G 0@ @``ILM^I_E_aC H 1IIh'6IC  =S%L(9C ._runs $ F)/v67 !HI $X%'*=IZ) S]D9$*CMMMO  Z F $x-Fs8CL  $ ZZZZ) S]D9$*CMMMO  ZZZ  $X%'*=IZ) S]D9$*CMMMO  ZZZZZ) S]D9$*CMMMO  ZZZs%BC7 :C# C EC 'C (.CEC  E# C4 7E:E E E EH?!G3H?:G ; H? G H?H<&:H) H<) H94H<<H?initialFc V^8dQhRRRR/#)rMrrrPr()rQs"r0rR _spawn_job..__annotate__s$4r/c\<SPRR7'gR#\;_uu_4V'g*SR,R8wdSP4RRR4R#SP4\P SR,4pVeVP RR4RSR&RSR&RRR4\SSR 7P4R # +'giL.;i \d@\P SR,R4SeS!4RSR&SP4hi;i) Fblockingr]rNr9rrvr})rT) acquirerrclearr#rrrstartrc)rrvrrrrrrs$ r0r_spawn_job..starts}}e},, Zs8}8 Z KKMll3x=1G" It,%CMCL  D 1 7 7 9+Z  LLX -" #CM MMO   s(CA C1C! C !A D+rrr9T)rN) threadingLockEventr#) rrrrrrrrrrrs fffddd$ @@@r0 _spawn_jobrs] NN E OO E$$24#+UGU"CX $r/c V^8dQhRRRR/#)rMskip_msgr3rPrr()rQs"r0rRrR!sBBsBtBr/c>\\PRVR7R#)uBounce a running router so it rescans the models dir (it only scans at spawn). Never raises — the file operation already succeeded.Nrg)rorrefresh_local_runtime)rs&r0rr!s 9 * *DAr/payloadc ,V^8dQhRRRRRRRRR R /#) rMendpointrsrKr3timeoutrOrz dict | NonerPrr()rQs"r0rRrR's8 E En EC EU EU` Elo Er/c RRVPRR4 2/pRpVe*RVR&\P!V4P4p\P P VR,PR ^4^,V,YTVeR MRR 7p\P PWbR 7;_uu_4pVeRM#\P!VP44uuRRR4# +'giR#;i) zrCall the local router (base_url minus ``/v1``) with its bearer key; GET (no payload) -> parsed JSON, POST -> None. AuthorizationzBearer api_keyryNzapplication/jsonz Content-Typebase_urlz/v1POST)dataheadersmethodr) rjsondumpsencodeurllibrequestRequestrspliturlopenloadsread)rrKrrrrreqrs&&$$ r0_router_requestr's'(,,y"*E)F GHG D"4zz'"))+ .. *!5!$_eligible_entries..\sQOq?1<<3POs. .)tuplerCATALOGr(r/r0_eligible_entriesr)Zs, 5QGOOQ5Q5QGOOQ QQr/cV^8dQhRR/#rMr4r3r()rQs"r0rRrR_sCr/cv\P!4PV4pVf\RRV 2R7hV#)Nzunknown model r`)r catalog_by_idrr )r4entrys& r0 _entry_or_404r0_s9  ! ! # ' ' 1E }nXJ4OPP Lr/c V^8dQhRRRR/#)rMr r fail_detailr3r()rQs"r0rRrRfs3r/cl\P!VRR7pVf\4f \V4hV#)ziForce-start the local server; raise ``fail_detail`` when neither we nor another process ended up serving.T)force)rensure_local_runtimer RuntimeError)r r2sups&& r0_start_local_serverr8fs1  ( (t .__annotate__ts...D.r/cN<SV9dRSR&\P!4R#R#)zRefreshing the local serverrbN)rr)r>rr4s&r0rescan_if_unknown)_ensure_server..rescan_if_unknownts% 5 9CM  + + - !r/Nc0<S!SP44#r")models)rAr7sr0 _ensure_server..zs(6r/rcV^8dQhRR/#rMrPrsr()rQs"r0rRr?s 2 2~ 2r/c<\SR^ R7;'g/PR.4pVUu/uFqPR4VbK up#uupi)/modelsrrid)rr)rmrs r0 _known_models%_ensure_server.._known_modelssJ#HiDJJOOPVXZ[D,01DqEE$KND1 11sA c<S!S!44#r"r()rMrAsr0rErFs (9r/)rr8ror) rr r4r2rrMrrAr7s f&f$$@@@@r0_ensure_serverrPnsl # "=> f 2C..  6HM H 2 2 94xP r/c$V^8dQhRRRRRR/#)rMrrsr4r3rPrr()rQs"r0rRrRs&|||3|4|r/cd\VRR4\P!RR4!RRVRRR4R#) zJMake ``model_id`` the main model via the same machinery as /api/model/set.setting-defaultMaking it your default_apply_model_assignment_synczhermes_cli.web_server_configmainllamacppryN)rr late)rr4s&&r0_assign_defaultrYs5 # ":; MM02PQRXZdfnprtvxz{r/c(V^8dQhRRRRRRRR/#)rMrGr3rKrevisionrPr()rQs"r0rRrRs.FF#FSFCFSFr/cRV RV RV 2#)zhttps://huggingface.co/z /resolve//r()rGrKr[s&&&r0_hf_urlr^s $TF)H:Qtf EEr/c V^8dQhRRRR/#)rMggufrrPr3r()rQs"r0rRrRs7777r/c\\P!\R,RVP4#)z@Variant model id for a staged file (strips split-part suffixes).$ry)resub_SPLIT_PART_REstem)r`s&r0 _model_id_forrgs 66.3&DII 66r/c V^8dQhRRRR/#)rMr4r3rPz list[Path]r()rQs"r0rRrRsSZr/cH\P!4PR4Uu.uFp\V4V8XgKVNK pp\P !V4pVe9V^,P V^,PV^,P3MRpY$Uu.uFkpVfK \P!4VP, P4'gKE\P!4VP, NKm up, pV#uupiuupi)zvEvery local file of a staged model: all split parts plus catalog-declared assets (mmproj/draft/MTP head) when present.z*.ggufr() r models_dirglobrgrfind_entry_for_modelmmprojdraftmtp_head assets_dir local_nameexists)r4pfileshitassetsas& r0_variant_files_on_diskrxs!,,.33H= ^=1qAQU]A]QQ=E ^  & &x 0C?Bc!fmmSV\\3q6?? ;TVF VUV4$-$8$8$:Q\\$I#Q#Q#S4i""$q||33VUUE L _UsDDD)7D%*DcV^8dQhRR/#)rMrPlistr()rQs"r0rRrRsdr/c ^VPUu.uF_p\VPVPVP4\ P !4VP, VP3NKa ppTVPVPVP3Uu.uFxq"fK\VP;'g VPVPVP4\ P!4VP, VP3NKz up, pV#uupiuupi)zeEverything a variant needs: split parts + mmproj/draft/MTP head assets, as (url, dest, bytes) tuples.) rtr^rGrKr[rrjrq size_bytesrmrnrorp)r/variantrwplans&& r0_download_planrs]] $"UZZ 4i6J6J6Lq||6[]^]i]i j"  $||U[[%..A TAvgaff** AFFAJJ?AUAUAWZ[ZfZfAfhihtht uA TTD K  $ TsA%D%D*&D*>AD*c V^8dQhRRRR/#)rM engine_tagr3rPrzr()rQs"r0rRrRsr/cJa\\P!44o\VP R44p.p\ P EFRpVPpVeVPR8wdK'\;QJd)V3RlVP4F 'gK RM RM!V3RlVP44'gK}VP'd,V\VPP R448dK\P!4VP, pVP4'dKTP\!VP";'g VP"VP$VP&4WTP(34EKU V#)zDownload-plan rows for the DFlash drafts of downloaded models that ``engine_tag`` can load but that aren't on disk: models downloaded before their draft joined the catalog.rz draft-dflashc3@<"TFqPS9xK R#5ir"r4)r#vstageds& r0r%"_missing_drafts..s@A::'sTF)setrstaged_model_idsrXrrr(rndraft_spec_typeanyvariantsdraft_min_enginerprqis_filerr^rGrKr[r|)rbuildr~r/assetdestrs& @r0_missing_draftsrs++- .F  '', -E D  =E11^C s@@sss@@@@   ! ! !ec%2H2H2U2UVY2Z.[&[ ##%(8(88||~~ KK!9!9uzz5::u~~VX\^n^no p! Kr/c(V^8dQhRRRRRRRR/#) rMrrsr~rzlabelr3rPrr()rQs"r0rRrRs) * *N *$ *s *SW *r/c VeCVP'd1\P!4VPP, MRpVUu.uFqU^,V8wgKVNK pp\ VRRV 24\ W4VUu.uFqU^,V8XgKVNK ppV'dF\ ;_uu_4VP^R/R7RRR4\ VRR4\W4R#R#uupiuupi +'giL6;i)zThe downloader counts completed files and every part of this plan. A catalog entry's draft downloads last and only speeds generation up, so a draft that fails costs the model nothing.N downloadingz Downloading r{rzrangesz5Downloading the draft model that speeds up generation) rnrrprqr _download_jobrupdate_download_optional)rr~rr/rnrowrequiredoptionals&&&& r0_run_download_planrs@E?PUZU`U`U`I "U[[%;%; ;fjE#7t1vtH7 #} UG45# #7t1vtH7 Z JJ!bJ A c="YZ3)  88 Zs$ C3!C3C8C8:C== D cV^8dQhRR/#rMrrsr()rQs"r0rRrRsr/caRV3RllpV#)c(V^8dQhRRRRRRRR/#)rMrrXrrrrPrr()rQs"r0rR-_download_progress_hook..__annotate__s($$3$s$D$T$r/c<\;_uu_4SPY;'gRVR7\SV4RRR4R# +'giR#;i)Nr)rrr)rrrrs&&&r0tick%_download_progress_hook..ticks4 Z JJ$MMT&J Q d #ZZZs *A A r()rrsf r0_download_progress_hookrs$$ Kr/c V^8dQhRRRR/#rr()rQs"r0rRrRs22~22r/c \PVR,/4p\VUUUu.uFwr4p\W44NK upppVPR4R7pVP \ V4R7R#uupppi)zBModel bytes use the same resumable transfer as pinned PM archives.r9r) pause_event)progressN)r#rrrrunr)rr~rvurlr_dls&& r0rrs]ll3x="-G >A6#$>GKKX_L` aBFF+C0F1?sA7 c V^8dQhRRRR/#)rMrrsrPrr()rQs"r0rRrRs  N T r/c\W4R# \dh\\3d"p\P RT4Rp?R#Rp?ii;i)zDownload files that only speed generation up, such as a DFlash draft. A failure leaves the model running without them, so it is logged instead of failing the job; a pause still pauses.z3optional download failed; continuing without it: %sNFT)rrr6OSErrorrmrn)rr~rds&& r0rrsHc   ' "LcRs A A AA c V^8dQhRRRR/#)rMrvrsrPz%tuple[dict[str, str], dict[str, Any]]r()rQs"r0rRrRsN/Tr/c aa \SR^R7pVPR.4Uu/uFUpVPR/4PR4R 9gK+VR,VPR/4PRR4bKW pp/p\P!4pVP 4Fwo p/pVPS 4pVe<VP VP \VP 4VPR 7VR 9;'d\V V3R lR4p V 'd/VP \V 4\\V 44R 7V'gKWtS &K W43#uupi)u2Resident models right now, plus how each is placed (granted window from the child, spill facts from the preset decision) — the difference between 'fast' and 'why is my CPU busy', so it must be inspectable. 'loading' is its own state (a 20-GB load in flight is the most important thing the pane can show).rJrrr]valuerKunknownN)window window_labelspilledcv<\SR\S4 2^R7PR/4PR4#)z/props?rdefault_generation_settingsn_ctx)rr r)r4rvsr0rE _loaded_models..s7OGw{87L6M-NXYZ S. 3CCLAr/)granted_windowgranted_window_label)loadedreadyloading)rr) rrrread_preset_decisionsitemsrrr[rrorX) rvrrLr placement decisionsstatefactsr~rr4s f @r0_loaded_modelsrsS 7Iq 9DLPHHU[]_L`UL`qx$((15SSCagquuXr*..w BBL`FU "I--/I!<<>% "}}X&   LL (4;;:OY]YeYeL f,,HH ABF2H  LLE RUV[R\I]L ^ 5"'h *  !Us (E*.E*c V^8dQhRRRR/#)rMr`rrPrsr()rQs"r0rRrRsOOdO~Or/c\V4p\R\V4;'gV.44pRVRVR\V4/#)c3T"TFqP4PxK R#5ir")statst_sizer#rss& r0r%_staged_row..sE)DAvvx)Ds&(rKr| size_label)rgsumr!rU)r`r4sizes& r0 _staged_rowrsET"H ET):)D)Dtf)DE ED (L$ io NNr/cV^8dQhRR/#rMrPr=r()rQs"r0rRrR s  : r/c&RRlp\VR4#)zThe active main model when it is one of ours (config authority: the model.provider + model.default that /api/model/set writes).cV^8dQhRR/#rr()rQs"r0rR/_active_llamacpp_model_id..__annotate__ s*r/c\4;'g/PR4;'g/p\VPRR44P4P 4\ 9dV\VPR4;'gVPR4;'gR4P4;'gR#R#)modelproviderryrjrN)rrr3striplower_LLAMACPP_PROVIDERS) model_sections r0r'_active_llamacpp_model_id..read s%--2227;AAr }  R0 1 7 7 9 ? ? AEX X}((3VV}7H7H7PVVTVW]]_ggcg gr/N)ro)rs r0_active_llamacpp_model_idr s $ r/z/api/local-models/statusca \4p\P!VPRR44pVe VPMRpV'd\P !V4M\ 4^,pVe VPMTp^RIpV'd'VP\PV,4MRp\\P.RR7\P!4p\4o S f//3M\V 3Rl//3RR7wrR\!VPR44R TR TR \!VPR4;'dVRJ;'dVRJ4R VRJR TRS RJRS ;'g/PR4R\#4RTR\\$P&/4RT RVP)4'd/\P*!4U u.uFp \-V 4NK up M.R\/V4/#uup i)zCheap, immediate: config state + installed runtime + staged models + supervisor state (GPU facts live in /hardware). Sync def on purpose: blocking urlopen/scans run in the threadpool.r'rNz legacy model adoption failed: %r)rfc<\S4#r")r)rvsr0rE%local_models_status..)s w'r/zloaded-models read failed: %rr rconfigured_tagupdate_availableruntime_installedruntime_backendserver_runningserver_base_urlractive_model_id loaded_modelsrrrDrj)rrrrr'rrrpminstalled_packageBACKEND_PACKAGESroradopt_legacy_modelsrjrrrrget_loading_progressrr staged_modelsrr3) rrrrrrcurrentmdirrrr`rvs @r0local_models_statusrs G  & &w{{9f'E FF(.(:fnnO=LX((9RaRcdeRfN*&**C Rab""8#<#<_#MNgkG 9 ( ("3UV    !DG%,OR'"b8W:YF 4 I./=M~DY!7!b!bF$