(HJjzddlZddlmZddlmZmZmZddlmZ ddl m Z ddl m Z ddlmZmZmZmZmZ ddede j*d ed eee j*egeeefffd Zgd Zgd Z d dedededeedeededeededeededeeeee j*egeeeffefdefdZd ej:fdZdZe dk(re!deyy)!N)Path)CallableOptionalUnion)tree_map_with_path)dequantize_modelloadquantize_modelsave upload_to_hubrecipemodel group_sizereturnc d d|dk(rd n(|dk(rd dn|dk(rd n|d k(rd ntd ||jDcgc] \}}d |vs |}}}t|d k(r td t|d j dD]\}|j snt|j  dtdtjdtttff fd }|Scc}}w)Naffine mixed_2_6 mixed_3_4 mixed_3_6 mixed_4_6zInvalid quant recipe down_projrz2Model does not have expected keys for mixed quant..pathmodulerc&t|jdkDrt|jdnd}| dzkxs|d zdzk\xs| dzz dzdk(}d|vsd|vsd |vr|rd Sd |vr|rd Sd |vrd Sd S) a=Implements mixed quantization predicates with similar choices to, for example, llama.cpp's Q4_K_M. Ref: https://github.com/ggerganov/llama.cpp/blob/917786f43d0f29b7c77a0c56767c0fa4df68b1c5/src/llama.cpp#L5265 By Alex Barron: https://gist.github.com/barronalex/84addb8078be21969f1690c1454855f3 rrrrv_projv_a_projv_b_proj)rbitsmoderlm_head)lensplitint) rrindex use_more_bitsr high_bitslayer_locationlow_bitsr' num_layerss X/Users/ahmed/devFolder/claude-voice/.venv/lib/python3.12/site-packages/mlx_lm/convert.pymixed_quant_predicatez.mixed_quant_predicate0s4::c?#n4  3/ 0  J!O # 2J!++ 2 a'1,1    d 2jD6H",iN N $ =",iN N  ",iN N((DII) ValueError named_modulesr) enumerater*isdigitlayersstrnnModulerbooldict) rrrk_ down_keysr3r.r/r0r'r1s ` @@@@@r2mixed_quant_predicate_builderrBs$ DI  ;  ;  ; 09::$224I4tq! q8H4II 9~MNN'y|'9'9#'>? 99; @U\\"JJJ J tTz JJ: ! OJs  C=C=)rrrr)float16bfloat16float32hf_pathmlx_pathquantize q_group_sizeq_bitsq_modedtype upload_reporevision dequantizequant_predicatetrust_remote_codec ft|tr t|}|jrt d|dt dt ||dd| id\} } }t| tr|dk7r t dt| | |} |jd d&|jd dx}r|jd dtvrYt d ttt| d dfd}| jt|| j|r | r t d|rt dt| ||||| \} }| r:t d|j!dd|j!ddt#| } t%||| | || t'||yy)NzCannot save to the path zY as it already exists. Please delete the file/directory or specify a new path to save to.z[INFO] LoadingTrQ)rN return_configtokenizer_configlazyrz4Quant predicates only support 'affine' quantization. torch_dtype text_configrLz[INFO] Using dtype:cast_predicatecy)NT)r@s r2zconvert..sDr4c|r?tj|jtjr|j S|S)N)mx issubdtyperLfloatingastype)r?vrXrLs r2 set_dtypezconvert..set_dtypes3a R]]177BKK%Hxx&r4z/Choose either quantize or dequantize, not both.z[INFO] Quantizing)r'rPz[INFO] Dequantizing quantizationquantization_config) isinstancer:rexistsr5printr rBgetMODEL_CONVERSION_DTYPESgetattrr]updater parametersr popr r r )rFrGrHrIrJrKrLrMrNrOrPrQr tokenizerconfigrWrbrXs ` @r2convertrpUs"(C >&xj1R R   #-/@A   E9f/3' X SU U7      } =$/ }M4)HH+H. '' #U+E" (8.I   ' 53C3C3EFGJJKK !"&    +  v #$ >4( ($/ '  h ,r4c\tjd}|jddtd|jdtdd |jd d d d|jddtd|jddtd|jddtdgd|jddt td|jdd tt d!|jd"d#td|jd$d%d&dd'|jd(d)dd'|S)*z Configures and returns the argument parser for the script. Returns: argparse.ArgumentParser: Configured argument parser. z(Convert Hugging Face model to MLX format) descriptionz --hf-pathz--modelzSPath to the model. This can be a local path or a Hugging Face Hub model identifier.)typehelpz --mlx-path mlx_modelzPath to save the MLX model.)rsdefaultrtz-qz --quantizezGenerate a quantized model. store_true)rtactionz--q-group-sizezGroup size for quantization.N)rtrsrvz--q-bitsz!Bits per weight for quantization.z--q-modezThe quantization mode.r)rmxfp4nvfp4mxfp8)rtrsrvchoicesz--quant-predicatezMixed-bit quantization recipe.F)rtr|rsrequiredz--dtypezvType to save the non-quantized parameters. Defaults to config.json's `torch_dtype` or the current model weights dtype.)rtrsr|rvz --upload-repoz-The Hugging Face repo to upload the model to.z-dz --dequantizezDequantize a quantized model.)rtrxrvz--trust-remote-codez)Trust remote code when loading tokenizer.)argparseArgumentParser add_argumentr:r+ QUANT_RECIPESri)parsers r2configure_parserrs $ $>F   b   3 :W  l!>|  +     0     % 5  -   F '   <      ,   8  Mr4c`t}|j}tdit|y)NrZ)r parse_argsrpvars)rargss r2mainrs'  F    D d4jr4__main__zCalling `python -m mlx_lm.convert ...` directly is deprecated. Use `mlx_lm.convert ...` or `python -m mlx_lm convert ...` instead.)@) ruFNNrNNNFNF)"r~pathlibrtypingrrrmlx.corecorer]mlx.nnr; mlx.utilsrutilsr r r r r r:r<r+r>r=rBrrirprrr__name__rgrZr4r2rs,,(689! 9! 9!/29! sBIIt$eD$J&7789!xE <  "& " #Z- Z-Z-Z-3- Z- SM Z-  Z- C=Z-Z-smZ-Z- hRYY-uT4Z/@@A3FGZ-Z-zK(11K\  z  O F r4