jdZddlZddlmZdadadZdZdZGddej Z Gd d ej j Z Gd d ZeZd ZdS)aRow views: the projections of a fused linear whose rows interleave them head by head, (heads, slots, head_dim, in_features), as the qkv projection of some attention checkpoints. Splitting such a weight into separate projections at load copies all its rows into private RAM; a row view computes its projection from the fused weight in place instead, with a kernel registered by the application (register_kernel), so the fused weight stays in its checkpoint file and MMGP manages it as one parameter (sizes, pinning, transfers). A split map entry declares this layout with "interleaved": True and "attention_head_dim"; load_model_data keeps such weights fused and installs row views when the registered kernel supports them, and splits them into copies of their rows otherwise.Ntree_mapc<||caaddlm}|tdS)zrows computes columns [start, stop) (multiples of head_dim) of projection slot of an interleaved fused weight for x of shape (tokens, in_features); available(dtype, head_dim) tells at load whether it computes them as fast as separate weights would be.)register_row_projectionN)_kernel _availableoffloadr_ROW_VIEW_PROJECTION)rows availablers 6C:\Users\johnn\OneDrive\Documents\AI\mmgp\row_views.pyregister_kernelrs: GZ000000011111c4tduot||SN)r )dtypehead_dims rr r s T ! Aj&A&AArc|jd||g|jddRdd|fjdg|jddRS)z;The rows of projection slot, copied: a split without views.rN)reshapeshape contiguous)tensorslotslotsrs rgatherrsj R>6>"eX A QRR0@ A A A!!!T' J RSU iX^XdefegegXh i i i t t v vvrcdeZdZdZejjZedZ dZ dZ e ddZ dS) RowViewWeightzThe weight of a row view as code inspecting linear modules sees it (shape, dtype, device); any operation on it works on a copy of the projection's rows (DoRA merges, for instance).c|jj}tj||j|jf|j|jS)N)rdevice) fusedweighttorchTensor_make_wrapper_subclass out_features in_featuresrr!)clsviewr"s r__new__zRowViewWeight.__new__'sK !|2239JDL\8]ejepy~zF2GG Grc||_dSr)r*)selfr*s r__init__zRowViewWeight.__init__,s  rc vd|jjd|jjdt|jd|jd S)NzRowViewWeight(projection  of z, ))r*rrtuplerrr-s r__repr__zRowViewWeight.__repr__/sDt49>tttyttRWX\XbRcRcttgkgqttttrNcTd}|t||it||piS)Ncbt|tr|jn|Sr) isinstancerr* materialize)values rz2RowViewWeight.__torch_dispatch__..4s* 5R_@`@`$kEJ$:$:$<$<$B;)rDr$torch.utils._pytreerrr rr rr%rrMrrrKr_r rr5rrrs-uu ((((((   222BBBwww YYYYYELYYY,UUUUUEHOUUUB++++++++&*)++ M M M M Mr