Nj /ddlmZddlZddlmZddlmZddlmZm Z m Z ddl Z ddl ZddlZddlmZddlmZddlmZdd lmZdd lmZmZdd lmZej:eZe e!e e e!zZ"Gd d eZ#y)) annotationsN)Counter)chain)AnyLiteralcast) Tokenizer)trange)evaluate_single_or_multi_label)BaseFinetuneable)ClassifierLightningModule#MultiLabelClassifierLightningModule)_DEFAULT_RANDOM_SEEDc HeZdZdZdZdddddddd dd dfd Zedd Z d dd ZdddZ ddddddddddde f ddZ d dZ d! d"dZ d#dZd$dZxZS)%StaticModelForClassification val_accuracymaxirNFT) n_layers hidden_dimout_dimpad_id token_mappingweightsfreeze normalizefreeze_weightsc t|D cgc] } t| c} |_d|_t |||||||| ||| |  ycc} w)z'Initialize a standard classifier model.F) vectorsrr tokenizerrrrrrrrN)rangestrclasses_ multilabelsuper__init__)selfr r!rrrrrrrrrx __class__s k/Users/ahmed/devFolder/Ultron/claude-voice/.venv/lib/python3.12/site-packages/model2vec/train/classifier.pyr'z%StaticModelForClassification.__init__sc"5:'N#CNqCFN#C % '!)  $DsA c@tj|jS)z(Return all clasess in the correct order.)nparrayr$)r(s r+classesz$StaticModelForClassification.classes>sxx &&c g}tdt||| D]}|j||||z}|jrvt j |}||kDj j} |j| D cgc]$} |jtj| &c} |j|jdjD cgc]} |j| c} |jrtj|tStj|Scc} wcc} w)aPredict labels for a set of texts. In single-label mode, each prediction is a single class. In multilabel mode, each prediction is a list of classes. :param X: The texts to predict on. :param show_progress_bar: Whether to show a progress bar. :param batch_size: The batch size. :param threshold: The threshold for multilabel classification. :return: The predictions. rdisablerdimdtype)r len_encode_single_batchr%torchsigmoidcpunumpyextendr/r- flatnonzeroargmaxtolistr.object) r(Xshow_progress_bar batch_size thresholdpredbatchlogitsprobsmaskrowidxs r+predictz$StaticModelForClassification.predictCs As1vz?P;PQE..q9K/LMF f- )..0668 $O$3T\\"..*=>$OP &--A-:N:U:U:WX:W3T\\#.:WXYR ??88D/ /88D> !PXs )E %E cg}tdt||| D]}|j||||z}|jrA|j t j |jjg|j t j|djjtj|dS)zPredict probabilities for each class. In single-label mode, returns softmax probabilities. In multilabel mode, returns sigmoid probabilities. rr2rr4)axis) r r8r9r%appendr:r;r<r=softmaxr- concatenate)r(rCrDrErGrHrIs r+ predict_probaz*StaticModelForClassification.predict_proba`s As1vz?P;PQE..q9K/LMF EMM&1557==?@ EMM&a8<<>DDFG R ~~d++r0gMbP?g?autoc tj|tjd|j ||j | t | tjrztjdt| t|jk7r tdt| jDcic]\}}|j||} }}|j| |}nd}|j!||| | |\}}|j#|t|}|j$rt'|||}nt)|||}|j+|||||||| |  |Scc}}w)aFit a model. This function creates a Lightning Trainer object and fits the model to the data. It supports both single-label and multi-label classification. We use early stopping. After training, the weights of the best model are loaded back into the model. This function seeds everything with a seed of 42, so the results are reproducible. It also splits the data into a train and validation set, again with a random seed. If `X_val` and `y_val` are not provided, the function will automatically split the training data into a train and validation set using `test_size`. :param X: The texts to train on. :param y: The labels to train on. If the first element is a list, multi-label classification is assumed. :param learning_rate: The learning rate. :param batch_size: The batch size. If None, a good batch size is chosen automatically. :param min_epochs: The minimum number of epochs to train for. :param max_epochs: The maximum number of epochs to train for. If this is -1, the model trains until early stopping is triggered. :param early_stopping_patience: The patience for early stopping. If this is None, early stopping is disabled. :param test_size: The test size for the train-test split. :param device: The device to train on. If this is "auto", the device is chosen automatically. :param X_val: The texts to be used for validation. :param y_val: The labels to be used for validation. :param class_weight: The weight of the classes. If None, all classes are weighted equally. Must have the same length as the number of classes. :param validation_steps: The number of steps to run validation for. If None, validation steps are estimated from the data. :param random_seed: The random seed to use. Defaults to 42. :return: The fitted model. :raises ValueError: If either X_val or y_val are provided, but not both. zRe-initializing model.NzVYou are passing a tensor as class weight. This will be removed in an upcoming version.z@class_weight must have the same length as the number of classes.) learning_rate class_weight) module train_dataset val_datasetrEearly_stopping_patience min_epochs max_epochsdevicevalidation_steps)plseed_everythingloggerinfo_initialize_on_labels _initialize isinstancer:Tensorwarningr8r$ ValueError enumeraterA_determine_class_weight_create_datasets_determine_batch_sizer%rr _train)r(rCyrYrEr_r`r^ test_sizeraX_valy_valrZrb random_seedrMwresolved_class_weightr\r]cs r+fitz StaticModelForClassification.fitosgb ;' ,- ""1%   #, 5wx|$DMM(::$%ghhDMlNaNaNcDdeDd&#q c 2A 5Dd e$($@$@q$Q !$( !%)%:%:1ay%Y" {// C 1|A G>||G5==99O>s #C9 C>cl|j|j|d||}t|||}|S)aEvaluate the classifier on a given dataset using scikit-learn's classification report. :param X: The texts to predict on. :param y: The ground truth labels. :param batch_size: The batch size. :param threshold: The threshold for multilabel classification. :param output_dict: Whether to output the classification report as a dictionary. :return: A classification report. T)rDrErF) predictionsrr output_dict)evalrNr )r(rCrrrErFrrreports r+evaluatez%StaticModelForClassification.evaluates8 ll1_hli /K1Zef r0ct|dttfrPttt|}t d|Ds t dd|_tt|}nitttt|}t d|Ds t dd|_tttj|}||_ t|j|_y) zSets the output dimensionality, the classes, and initializes the head. :param y: The labels. :raises ValueError: If the labels are inconsistent. rc3HK|]}t|ttfywN)rir#int.0labels r+ zEStaticModelForClassification._initialize_on_labels..sD!z%#s4! "zFInconsistent label types in y. All labels must be strings or integers.Fc3HK|]}t|ttfywr)rir}tuplers r+rzEStaticModelForClassification._initialize_on_labels..sGQEz%$7QrzBInconsistent label types in y. All labels must be lists or tuples.TN)rir#rrr}allrlr%sortedsetrr~r$r8r)r(rrr/s r+rgz2StaticModelForClassification._initialize_on_labelss adS#J 'T#Y"AD!DD !ijj#DOSVnGT$s)_a(AGQGG !eff"DOS!4!4Q!789G 4==) r0c4|jrt|j}tjt||tj }t |jDcic]\}}|| }}}t |D]\}}|Dcgc]}|| } }d||| f< |Stjttt|Dcgc]}|jj|c}tj}|Scc}}wcc}wcc}w)z5Convert a list or list of list of labels to a tensor.r6g?) r%r8r$r:zerosfloatrmrrr}r#indexlong) r(labels num_classes labels_tensorrMrmappingi sample_labelsindicess r+_labels_to_tensorz.StaticModelForClassification._labels_to_tensors ??dmm,K!KKF [ TM4=dmm4LM4Ljc5ucz4LGM$-f$5 =7DE}e75>}E,/ aj)%6 "LL9=d3i9PQ9P$$U+9PQY^YcYcMNERs) D  D"D)r torch.Tensorr!r rrrrrrrrrzlist[int] | Nonerztorch.Tensor | NonerboolrrrrreturnNone)r np.ndarray)F?) rC list[str]rDrrErrFrrr)Fr)rCrrDrrErrr)rCrrr LabelTyperYrrE int | Noner_rr`rr^rrsrrar#rtzlist[str] | NoneruzLabelType | NonerZzrsi"%% >1l6   8 $ IT#Y ' ~#3~r0