+ Wj< Rt^RIHt^RIHtHtHt^RIHtH t H t H t H t H t HtR0tRtRtRtRtR1t]!R24tRtR tR R lt]!R R 44tRRR^/RRllt]!RR44tRRltRRR]RR/RRlltRRR]/RRllt]!R R!44t RRR"R/R#R$llt!R3R%R&llt"R3R'R(llt#RRRRR)^R*RR+R/R,R-llt$R+R/R.R/llt%R#)4udContext policy — the window ladder for managed local models. One contract: any model runs at any window up to its native max; hardware and session depth only change tokens/s. Constants, not knobs — nothing in this module reads config. The policy encodes behavior measured on real hardware (llama.cpp, discrete NVIDIA on Windows/WDDM, unified-memory). ) annotations) dataclassfieldreplace)HardwareBudget LayerKind ModelProfilePhysicsRefusal ctx_bytesfootprint_bytes physics_check?g333333?g@g333333?c V^8dQhRRRR/#)nativeintreturn list[int])formats"[C:\Users\johnn\AppData\Local\hermes\hermes-agent\hermes_cli\local_runtime\context_policy.py __annotate__r(s39c.p\\4pW 8d*VP\V44V\,pK/VPV4V#)zB64K -> 96K -> 128K -> ... -> native (native always the last rung).)floatFLOORappendr_LADDER_GROWTH)rrungssteps& rladderr"(s@E .__annotate__PsIIIIrc <\SVSR7#)r;)r )rEr;r>s&rkvinitial_window..kvPs$HHrc V^8dQhRRRR/#rDr)rs"rrrFSs>>3>3>rc"<\SVSSR7#)rB)r )rEr;r<r>s&rneedinitial_window..needSswo.<> >rNzlargest zero-spill rung (K)floor held at z4K; weights spill (deliberate price of the guarantee))r%r+r&r() r r n_ctx_trainr"usable_vram_bytesminr_EARLY_COST_CTX_FRACTIONram_available_bytesr$max)r>r?r;r<refusalrr rIrMbest_zero_spillrEr%reasoncapkv_bytessf&dd rinitial_windowr]?sqGVUO+9;G  ) )EF 6NEII>>#'O :00 0  "#eV:L'L ,Vt^,r)r3r4r5r6r7r>r:rrrr_r_ts--#'G 'rr_c(V^8dQhRRRRRRRR/#)rr>r mtp_capabler'rarr)rs"rrr|s/gg\g4gdgWcgrcV'g V'g\M\pV'd^M^p\WPV,V,R7#)z``profile`` at one launch posture: window-scaled compute buffers grow with the microbatch launch_args passes, and MTP runs a second context beside the target.)window_compute_per_token)PREFILL_UBATCHDEFAULT_UBATCHrwindow_compute_bytes)r>rcraubatchcontextss&$$ rposture_profilerk|s5 ++^>FqQH 75Q5QTZ5Z]e5e ffrrcFfixed_overheadrequested_windowNc 0V^8dQhRRRRRRRRR R R R /#) rr>rr?rrcr'rlrrm int | Nonerr_r)rs"rrrs9111~1t1 #1",18B1rcBaaaaaa S'dSPR8Xd\SRR7o/o RVVV VVV3RllpV!R4pS'gV#V!R4p\VP\4'dV#\VP\4'dV#VPP VPP 8dV#VPP 'gV#S R,S R,rS'd\V\4'd\V \4'dqVP 'g_VP V P 8dDVPP VP 8dVPP 'dV#V#)zWindow first, then prefill; price both postures at the effective window. A restored window may fit only under lean MTP. Evaluate it before discarding it because stacked exceeds memory, and keep deliberate spill when neither posture is resident. g?g333333?)kv_scalec V^8dQhRRRR/#)rstackedr'rr_r)rs"rr!plan_launch..__annotate__s@@4@J@rc<\S S VR7pS\S PS VR7,p\VSVR7pVSV&\ V\ 4'dS 'd\ S S P;'gS 4pWCP8dr\VSWBR7fa\WVR7p\ V\^VSP, 4\W4V,SP8*RVR, R2.R7p\W0W!4#))rcra)r<zgrown window restored (rOrPr%r&r(r+)rkub_logits_bytesn_vocabr] isinstancer$rTrRr%r r rWrSr r_) rspostureoverheadr`targetrMr?rlinitialrcr>rms & r candidateplan_launch..candidates!'{PWX!O OO'%KK!'6(K# h / /4D)7+>+>+R+RBRSF'MVV-FIM-N&wxP)!s1dV=U=U6U/V'88CvG_G__6v~6FbIJL(X??rFT)rqrryr`r r%r1r$) r>r?rcrlrmr~leanrsstacked_initial lean_initialr}s ffddd @r plan_launchrs4w''3.'C0;=G@@" U D  oG'""N33 $--00!5!55    # # #$+DM75>\ZHH<88AXAXAX&&,*=*==  ''/*@*@@T]]EZEZEZ Krc 0V^8dQhRRRRRRRRR R R R/#) rplanr_r>rliverrcr'rlrrr)rs"rrrs9==Z=,=n=$(='*=HR=rc 6VPp\V\4'dVP'dV#\ WW4R7p\VP\4'dVPP'gVPP VP 8dV#\ \VPVPP R, RVP R, R2.R7VPVPVP4#\\VP;'g\4pVP V8:dV#\ \V^RRVR, R2.R 7VPVPVP4#) aNarrow a resident capacity plan to the window the card's free memory holds now. Only the window moves, and only down to the floor. A plan that already spills keeps its placement, and so does a card too busy to hold even the floor: moving weights to the CPU on a live reading is how a launch once pinned a fitting model to the CPU, because the reading still counted memory the outgoing server was about to free. )rcrlrOz*K fits beside other programs' GPU memory (z K would not))r+TrQz-K; other programs hold most of the GPU memoryrv)r`ryr$r1rr%r_rrar<r>rTrrR)rr>rrcrlr`nowfloors&&&$$ rfit_to_free_memoryrsQ}}H h / /83C3C3C g \C#,,// 8L8L8L <<  (// 1K'#,,||""d*+,4'(  69789<#J\J\^a^i^ik k w**33e 4E% nEqD $'TU[WX $--t|| ==rc>]tRt^t$R]R&RtR]R&RtR]R&RtR#) GrowthDecisionstractionNro next_windowrZr)r3r4r5r6r7rrZr:rrrrrs K"K"FCrroccupancy_confirmedc<V^8dQhRRRRRRRRRR R R R R R R RR/ #)rr>rr?rcurrent_windowrsession_tokensmeasured_decode_tok_sz float | None server_idler'r;rrrr)rs"rrrsb&\&\\&\>&\$'&\9<&\Ua&\!%&\8<&\*.&\;I&\rc @aV'g VS\,8d\RRR7#VP;'gSpSV8d\RRR7#V'g\RRR7#Ve&V\8d\RRVR R \R R 2R7#\ V3R l\ V44V4p \ W VR 7p VPV ,VPVP,8d\RRR7#\RV RSR, RV R, R2R7#)uOne growth evaluation, END-OF-TURN ONLY (recurrent state cannot rewind mid-sequence). Gate order: occupancy (~85%) → native cap → idleness (growth only on an otherwise-idle server) → speed floor (below it compression is the default) → re-fit against LIVE free memory (the rung must fit NOW, not at launch). ``occupancy_confirmed`` skips gate 1 when the caller's own compression gate already fired, so two edge definitions can't deadlock into compress-before-grow. holdzsession below growth occupancy)rZzcompress-defaultz.at native window; compression is the only movez&server busy; re-grant deferred to idlezdecode z.1fz tok/s below the ~z.0fz: tok/s floor; growth is now an explicit per-session choicec38<"TFqS8gK VxK R#5iNr).0rrs& r "growth_decision..sFA~3Eaas rHz.next rung exceeds physics; compression insteadgrowzrung rOzK -> K)rrZ) _GROW_AT_OCCUPANCYrrRSPEED_FLOOR_TOK_Snextr"r weights_bytesrSrV) r>r?rrrrr;rr next_rungrIs &&d$$$$$ rgrowth_decisionrs+ >NEW4W#Wf-MNN  2 2NF09ijj f-UVV(-BEV-V 3C89)#./224 4 FFOI 7 GBr!F$<$rr&rorrr)rs"rrrs"LzU^rca\SP4UUUu.uF!wpwr4V\P8XgKVNK# ppppVeA\;QJdV3RlV4F 'gK RM RM!V3RlV44'dV#.p^pVF7pWq8dV#VP V4VSP V,, pK9 V#uupppi)abRecurrent (n_head_kv==0) block indices whose FFN weights cover ``spill_bytes``. Takes the fewest blocks, lowest index first (llama.cpp's -ngl also keeps the front blocks on the host). Every recurrent block when ``spill_bytes`` is None, when any recurrent block's FFN size is unknown, or when all of them together still fall short (fit spills the rest). c3@<"TFqSP9xK R#5ir)ffn_block_bytes)rir>s& rr)recurrent_spill_blocks.. s!VIq7+B+B"BIsTF) enumeratelayersr RECURRENTanyrr)r>r&rkind_ recurrentchosencoveredsf& rrecurrent_spill_blocksrs(1'@`'@|q)4DIL_L_D_'@I`cc!VI!Vccc!VI!VVVFG   !  M  a7**1--  Mas CCc$V^8dQhRRRRRR/#)rr>rr&rorr*r)rs"rrrs!  \  i rc VP'dRR.#\W4pV'd1RRPRP\ \ V444.#.#)a1-ot placement for spilled configs: expert/FFN weights to host so attention + KV stay GPU-resident. MoE gets the expert pattern; hybrids push the FFNs of just enough recurrent blocks to cover ``spill_bytes`` (their n_head_kv==0 layers carry no KV worth protecting, and full-attention FFNs stay on the GPU).z-otz!blk\.\d+\.ffn_.*_exps\.weight=CPUzblk\.({})\.ffn_.*\.weight=CPU|)moerrjoinmapr)r>r&blockss&& rspill_overridesrsS {{{;<< #G 9F 7>>sxxCQWHX?YZ[[ Irmtp_draft_depthumarac8V^8dQhRRRRRRRRRR R RR RR R /#)rr>rr`r$r;r'rcrrrrarr*r)rs"rrr%sJUY!r`r;rcrrraargss&&$$$$$ r launch_argsr%s #hoo& 'D  -A3CW%'FH H  T65#n*=> >D s>*E3~3FGG == )=)=>> Krc(V^8dQhRRRRRRRR/#)rrxrrcr'rarr)rs"rrr:s)  S $ T VY rc\^\V44pV'd)V'd!\RV,^,R,4#V'dRV,^,^,#RV,^,#)zGPU logits/compute-buffer cost of the microbatch posture chosen by launch_args, priced from the model's own vocab and calibrated against measured server RSS (Qwen3.8 Q4, both postures, three windows).rr r)rWr)rxrcravs&$$ rrwrw:sT As7|A{4!8a<#%&&Qw{Q !8a<rii@gAr)&__doc__ __future__r dataclassesrrr"hermes_cli.local_runtime.estimatorrrrr r r r rrrrrU TARGET_WINDOWrRUNTIME_OVERHEAD_BYTESrgrfr"r$r]r_rkrrrrrrrrwrrrrsg#11   _- $$ $2[]a2[)*2[j (( (g1V[1&<1/31h=+0=-C=8   &\?C&\16&\R( \`$)BCQV$)* E  r