Nj;5~ddlmZddlZddlmZddlmZddlmZddl m Z ddl m Z m Z ddlZddlZddlZddlmZdd lmZmZdd lmZdd lmZdd lmZmZdd lm Z ejBdZ"dZ#e de$e%e$e$e%Z&Gdde%eZ'GddZ( d ddZ)ddZ*ddZ+ d ddZ,y)) annotationsN)Sequence)Enum)Path)TemporaryDirectory)TypeVarcast)classification_report) MLPClassifier MLPRegressor)Pipeline)MultiLabelBinarizer)PathLike StaticModel)create_model_cardz sklearn\..+zpipeline.skops LabelTypeceZdZdZdZdZy)HeadType classifier projector multilabelN)__name__ __module__ __qualname__ CLASSIFIER PROJECTOR MULTILABELj/Users/ahmed/devFolder/Ultron/claude-voice/.venv/lib/python3.12/site-packages/model2vec/inference/model.pyrrsJIJrrc eZdZd dZe d d dZd dZ d ddZ ddZ d ddZ d ddZ d dd Z y)StaticModelPipelinec||_||_|jd}d|_t|trt j |_yt|trR|j}|dk(rt jnt j|_|jj|_yt j|_y)z-Create a pipeline with a StaticModel encoder.Nlogistic) modelheadclasses_ isinstancer rrclassifier_typer out_activation_rr)selfr&r' last_head activations r __init__zStaticModelPipeline.__init__"s  IIbM +/ i .#+#5#5D  = 1"22J:D :R8#6#6X`XkXkD  II..DM$,#6#6D rNc|t|||\}}tj|j|_|||S)aLoad a StaticModel from a local path or huggingface hub path. NOTE: if you load a private model from the huggingface hub, you need to pass a token. :param path: The path to the folder containing the pipeline, or a repository on the Hugging Face Hub :param token: The token to use to download the pipeline from the hub. :param trust_remote_code: Whether to trust the remote code. If this is False, we will only load components coming from `sklearn`. :return: The loaded pipeline. )_load_pipelinenp nan_to_num embedding)clspathtokentrust_remote_coder&r's r from_pretrainedz#StaticModelPipeline.from_pretrained3s8%T52CD t--85$rct||y)zSave the model to a folder.N) save_pipeline)r,r6s r save_pretrainedz#StaticModelPipeline.save_pretrainedEs dD!rcddlm}t5}t|||jj ||t |||||dddy#1swYyxYw)a:Save a model to a folder, and then push that folder to the hf hub. :param repo_id: The id of the repository to push to. :param subfolder: The subfolder to push to. :param token: The token to use to push to the hub. :param private: Whether the repository should be private. r)push_folder_to_hubN)model2vec.persistencer>rr;r&r<r)r,repo_id subfolderr7privater>temp_dirs r push_to_hubzStaticModelPipeline.push_to_hubIsK =  !X $ ) JJ & &x 0 tH~y'7E R" ! !s =AA c|jj||||||}tj|dk(r |dddf}|S)z7Encode the instances and coerce the output to a matrix.show_progress_bar max_length batch_sizeuse_multiprocessingmultiprocessing_thresholdN)r&encoder2ndimr,XrGrHrIrJrKencodeds r _encode_and_coerce_to_2dz,StaticModelPipeline._encode_and_coerce_to_2dZsU**## /!! 3&? $  777 q dAg&Grc||j||||||}|jtjk(rn|jJg} |j j |} | D]#} | j|j| |kD%tj| tS|j j|S)aPredict the labels of the input. :param X: The input data to predict. Can be a list of strings or a single string. :param show_progress_bar: Whether to display a progress bar during prediction. Defaults to False. :param max_length: The maximum length of the input sequences. Defaults to 512. :param batch_size: The batch size for prediction. Defaults to 1024. :param use_multiprocessing: Whether to use multiprocessing for encoding. Defaults to True. :param multiprocessing_threshold: The threshold for the number of samples to use multiprocessing. Defaults to 10,000. :param threshold: The threshold for multilabel classification. Defaults to 0.5. Ignored if not multilabel. :return: The predicted labels or probabilities. rF)dtype) rRr*rrr(r' predict_probaappendr2asarrayobjectpredict) r,rPrGrHrIrJrK thresholdrQ out_labelsprobavectors r rYzStaticModelPipeline.predictqs*// /!! 3&? 0    8#6#6 6==, ,,JII++G4E!!$--0B"CD ::j7 7yy  ))rc|jtjk(r td|j ||||||}|j j |S)aPredict the labels of the input. :param X: The input data to predict. Can be a list of strings or a single string. :param show_progress_bar: Whether to display a progress bar during prediction. Defaults to False. :param max_length: The maximum length of the input sequences. Defaults to 512. :param batch_size: The batch size for prediction. Defaults to 1024. :param use_multiprocessing: Whether to use multiprocessing for encoding. Defaults to True. :param multiprocessing_threshold: The threshold for the number of samples to use multiprocessing. Defaults to 10,000. :return: The predicted labels or probabilities. :raises ValueError: If the classifier type is projector. CYou are using evaluate on a projector model. This is not supported.rF)r*rr ValueErrorrRr'rUrOs r rUz!StaticModelPipeline.predict_probasb(   8#5#5 5bc c// /!! 3&? 0 yy&&w//rc|jtjk(r td|j |d||}t |||}|S)aEvaluate the classifier on a given dataset using scikit-learn's classification report. :param X: The texts to predict on. :param y: The ground truth labels. :param batch_size: The batch size. :param threshold: The threshold for multilabel classification. :param output_dict: Whether to output the classification report as a dictionary. :return: A classification report. :raises ValueError: If the classifier type is projector. r_T)rGrIrZ) predictionsy output_dict)r*rrr`rYevaluate_single_or_multi_label)r,rPrcrIrZrdrbreports r evaluatezStaticModelPipeline.evaluatesO   8#5#5 5bc cll1_hli /K1Zef r)r&rr'r returnNoneNF) r5ztype[StaticModelPipeline]r6rr7 str | Noner8boolrhr")r6strrhri)NNF) r@rmrArkr7rkrBrlrhri)rP Sequence[str]rGrlrH int | NonerIintrJrlrKrprh np.ndarray)FT'?)rPrnrGrlrHrorIrprJrlrKrprZfloatrhrq)FrrrsTrt)rsruF) rPrnrcrrIrprZrvrdrlrh!str | dict[str, dict[str, float]]) rrrr/ classmethodr9r<rDrRrYrUrgrrr r"r"!s7"lq & .6 ?I ei   "" ejSS'1SAKS]aS S"      " $' 4#(!$$()/&* &* &* &*  &* " &*$'&*&* &*V#(!$$()/0 0 0 0  0 " 0$'0 0Dsx#,:=QVko *rr"ct|}t}|jr#||z }|js4td|t j |j ||}tj|}tjj|}|s+|D]&}tj|rtd|dtjj||}||fS)aLoad a model and an sklearn pipeline. This assumes the following files are present in the repo: - `pipeline.skops`: The head of the pipeline. - `config.json`: The configuration of the model. - `model.safetensors`: The weights of the model. - `tokenizer.json`: The tokenizer of the model. :param folder_or_repo_path: The path to the folder containing the pipeline. :param token: The token to use to download the pipeline from the hub. If this is None, you will only be able to load the pipeline from a local folder, public repository, or a repository that you have access to because you are logged in. :param trust_remote_code: Whether to trust the remote code. If this is False, we will only load components coming from `sklearn`. If this is True, we will load all components. If you set this to True, you are responsible for whatever happens. :return: The encoder model and the loaded head :raises FileNotFoundError: If the pipeline file does not exist in the folder. :raises ValueError: If an untrusted type is found in the pipeline, and `trust_remote_code` is False. z Pipeline file does not exist in )r7)filezUntrusted type .)trusted)r_DEFAULT_MODEL_FILENAMEexistsFileNotFoundErrorhuggingface_hubhf_hub_downloadas_posixrr9skopsioget_untrusted_types_DEFAULT_TRUST_PATTERNmatchr`load) folder_or_repo_pathr7r8model_filenamehead_pipeline_pathr& unknown_typestr's r r1r1s,23,N!!#0>A!((*#&FGZF[$\] ],<<  ( ( *N%   ' '(; z1evaluate_single_or_multi_label..3srs" $' 1>%512#N3* KcDcO < sD kk^X]-!-*4-PT-!-`6`!!@!!' !r