^NjddlmZmZmZmZddlmZddlmZddl m Z ddl m Z ddl mZddlmZddlmZmZdd lmZdd lZdd lmZGd d eZy ))AnyIterableSequenceType)asdict) OnnxProvider)Device)Bm25)Bm42)MiniCOIL)SparseEmbeddingSparseTextEmbeddingBase)SpladePPN)SparseModelDescriptionczeZdZUeeeegZee e e d<e dee eeffdZe deefdZdddej(ddfdededzd edzd eedzd eezd eedzd edeffd Z ddeeezdededzdedeef dZdeeezdedeefdZ ddeeezdededefdZxZS)SparseTextEmbeddingEMBEDDINGS_REGISTRYreturncZ|jDcgc] }t|c}Scc}w)a Lists the supported models. Returns: list[dict[str, Any]]: A list of dictionaries containing the model information. Example: ``` [ { "model": "prithvida/SPLADE_PP_en_v1", "vocab_size": 30522, "description": "Independent Implementation of SPLADE++ Model for English", "license": "apache-2.0", "size_in_GB": 0.532, "sources": { "hf": "qdrant/SPLADE_PP_en_v1", }, } ] ``` )_list_supported_modelsr)clsmodels w/Users/ahmed/devFolder/Ultron/claude-voice/.venv/lib/python3.12/site-packages/fastembed/sparse/sparse_text_embedding.pylist_supported_modelsz)SparseTextEmbedding.list_supported_modelss+0,/+E+E+GH+G%u +GHHHs(cjg}|jD]!}|j|j#|SN)rextendr)rresult embeddings rrz*SparseTextEmbedding._list_supported_models/s1/100I MM)::< =1 NF model_name cache_dirthreads providerscuda device_ids lazy_loadkwargsc ^t |||fi|jdjk(rtjdt dd|j D]=} | j} tfd| Ds(| |f|||||d||_ ytdd ) Nzprithvida/Splade_PP_en_v1zvThe right spelling is prithivida/Splade_PP_en_v1. Support of this name will be removed soon, please fix the model_name) stacklevelzprithivida/Splade_PP_en_v1c3tK|]/}j|jjk(1ywr)lowerr).0rr!s r z/SparseTextEmbedding.__init__..Ms/[JZ:##%):):)< 1, data-parallel encoding will be used, recommended for offline encoding of large datasets. If 0, use all available cores. If None, don't use data-parallel processing, use default onnxruntime threading instead. Returns: List of embeddings, one per document N)rembed)r7r;r<r=r(s rr?zSparseTextEmbedding.embed_s+,$4::##Iz8NvNNNs #-+-queryc+ZK|jj|fi|Ed{y7w)z Embeds queries Args: query (Union[str, Iterable[str]]): The query to embed, or an iterable e.g. list of queries. Returns: Iterable[SparseEmbedding]: The sparse embeddings. N)r query_embed)r7r@r(s rrBzSparseTextEmbedding.query_embedws'*4::))%:6:::s !+)+textsc @|jj|fd|i|S)aReturns the number of tokens in the texts. Args: texts (str | Iterable[str]): The list of texts to embed. batch_size (int): Batch size for encoding Returns: int: Sum of number of tokens in the texts. r<)r token_count)r7rCr<r(s rrEzSparseTextEmbedding.token_counts%&tzz%%eM MfMMr )N)i) __name__ __module__ __qualname__rr r r rlistrr__annotations__ classmethoddictstrrrrrr AUTOintrrboolr1rr r?rBrE __classcell__)r:s@rrrs@H$PTV^?_d#:;<_Id4S>&:II2t,B'C!%"37$kk'+' ' :' t ' L)D0 ' Vm ' I$' ' ' X# O#&OO* O  O / " O0 ;x}!4 ; ;Q`Ha ;=A N8C=( N69 NLO N  Nr r)typingrrrr dataclassesrfastembed.commonrfastembed.common.typesr fastembed.sparse.bm25r fastembed.sparse.bm42r fastembed.sparse.minicoilr &fastembed.sparse.sparse_embedding_baser rfastembed.sparse.splade_pprr2"fastembed.common.model_descriptionrrr rr^s>00))&&.0E}N1}Nr