o JjgB@sdZddlmZddlmZmZmZmZddlZ ddl m Z m Z m Z ddlmZddlmZddlmZmZdd lmZdd lmZmZdd lmZd efd dZGdddeeZGdddeZGdddeeZ GdddeeZ!dS)z@A chain for comparing the output of two models using embeddings.)Enum)AnyDictListOptionalN)AsyncCallbackManagerForChainRunCallbackManagerForChainRun Callbacks) Embeddings)pre_init) ConfigDictField)Chain)PairwiseStringEvaluatorStringEvaluatorRUN_KEYreturnc CsRz ddlm}W|Sty(z ddlm}WY|Sty'tdww)zaCreate an Embeddings object. Returns: Embeddings: The created Embeddings object. rOpenAIEmbeddingstCould not import OpenAIEmbeddings. Please install the OpenAIEmbeddings package using `pip install langchain-openai`.)langchain_openair ImportError%langchain_community.embeddings.openairrd/var/www/html/zoom/venv/lib/python3.10/site-packages/langchain/evaluation/embedding_distance/base.py_embedding_factorys  rc@s$eZdZdZdZdZdZdZdZdS)EmbeddingDistancea Embedding Distance Metric. Attributes: COSINE: Cosine distance metric. EUCLIDEAN: Euclidean distance metric. MANHATTAN: Manhattan distance metric. CHEBYSHEV: Chebyshev distance metric. HAMMING: Hamming distance metric. cosine euclidean manhattan chebyshevhammingN) __name__ __module__ __qualname____doc__COSINE EUCLIDEAN MANHATTAN CHEBYSHEVHAMMINGrrrrr*s rc@sVeZdZUdZeedZeed<ee j dZ e ed<e de eefde eeffdd Zed d Zedeefd d ZdedefddZde defddZedejdejdejfddZedejdejdejfddZedejdejdejfddZedejdejdejfddZ edejdejdejfddZ!d ejde"fd!d"Z#d#S)$_EmbeddingDistanceChainMixina0Shared functionality for embedding distance evaluators. Attributes: embeddings (Embeddings): The embedding objects to vectorize the outputs. distance_metric (EmbeddingDistance): The distance metric to use for comparing the embeddings. )default_factory embeddings)defaultdistance_metricvaluesrcCs|d}g}z ddlm}||Wn tyYnwz ddlm}||Wn ty4Ynw|s;tdt|t|rUzddl}W|StyTtdw|S)zValidate that the TikTok library is installed. Args: values (Dict[str, Any]): The values to validate. Returns: Dict[str, Any]: The validated values. r.rrrNzThe tiktoken library is required to use the default OpenAI embeddings with embedding distance evaluators. Please either manually select a different Embeddings object or install tiktoken using `pip install tiktoken`.) getrrappendrr isinstancetupletiktoken)clsr1r.types_rr6rrr_validate_tiktoken_installedHs8       z9_EmbeddingDistanceChainMixin._validate_tiktoken_installedT)arbitrary_types_allowedcCsdgS)zgReturn the output keys of the chain. Returns: List[str]: The output keys. scorerselfrrr output_keysxsz(_EmbeddingDistanceChainMixin.output_keysresultcCs$d|di}t|vr|t|t<|S)Nr;r)r=r?parsedrrr_prepare_outputs  z,_EmbeddingDistanceChainMixin._prepare_outputmetricc CsJtj|jtj|jtj|jtj|jtj |j i}||vr||St d|)zGet the metric function for the given metric name. Args: metric (EmbeddingDistance): The metric name. Returns: Any: The metric function. zInvalid metric: ) rr'_cosine_distancer(_euclidean_distancer)_manhattan_distancer*_chebyshev_distancer+_hamming_distance ValueError)r=rBmetricsrrr _get_metrics z(_EmbeddingDistanceChainMixin._get_metricabcCs6zddlm}Wn tytdwd|||S)zCompute the cosine distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.ndarray: The cosine distance. r)cosine_similarityzThe cosine_similarity function is required to compute cosine distance. Please install the langchain-community package using `pip install langchain-community`.g?)langchain_community.utils.mathrMr)rKrLrMrrrrCs  z-_EmbeddingDistanceChainMixin._cosine_distancecCstj||S)zCompute the Euclidean distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Euclidean distance. )nplinalgnormrKrLrrrrDs z0_EmbeddingDistanceChainMixin._euclidean_distancecCtt||S)zCompute the Manhattan distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Manhattan distance. )rOsumabsrRrrrrE z0_EmbeddingDistanceChainMixin._manhattan_distancecCrS)zCompute the Chebyshev distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Chebyshev distance. )rOmaxrUrRrrrrFrVz0_EmbeddingDistanceChainMixin._chebyshev_distancecCst||kS)zCompute the Hamming distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Hamming distance. )rOmeanrRrrrrGs z._EmbeddingDistanceChainMixin._hamming_distancevectorscCs6||j}||ddd|ddd}|S)zCompute the score based on the distance metric. Args: vectors (np.ndarray): The input vectors. Returns: float: The computed score. r)rJr0reshapeitem)r=rYrBr;rrr_compute_scores &z+_EmbeddingDistanceChainMixin._compute_scoreN)$r#r$r%r&r rr.r __annotations__rr'r0r rstrrr9r model_configpropertyrr>dictrArJ staticmethodrOndarrayrCfloatingrDrErFrGfloatr^rrrrr,<s0 $+    r,c@sFeZdZdZedefddZedefddZede efddZ dd e ee fd e ede ee ffd d Z dd e ee fd e ede ee ffddZd d d d dddede edede e ede e ee fdede defddZd d d d dddede edede e ede e ee fdede defddZd S)EmbeddingDistanceEvalChaina"Use embedding distances to score semantic difference between a prediction and reference. Examples: >>> chain = EmbeddingDistanceEvalChain() >>> result = chain.evaluate_strings(prediction="Hello", reference="Hi") >>> print(result) {'score': 0.5} rcCsdS)zReturn whether the chain requires a reference. Returns: bool: True if a reference is required, False otherwise. Trr<rrrrequires_referencesz-EmbeddingDistanceEvalChain.requires_referencecCd|jjdS)N embedding_ _distancer0valuer<rrrevaluation_namez*EmbeddingDistanceEvalChain.evaluation_namecCddgS)eReturn the input keys of the chain. Returns: List[str]: The input keys. prediction referencerr<rrr input_keys z%EmbeddingDistanceEvalChain.input_keysNinputs run_managercC0t|j|d|dg}||}d|iS)a0Compute the score for a prediction and reference. Args: inputs (Dict[str, Any]): The input data. run_manager (Optional[CallbackManagerForChainRun], optional): The callback manager. Returns: Dict[str, Any]: The computed score. rsrtr;rOarrayr.embed_documentsr^r=rwrxrYr;rrr_calls  z EmbeddingDistanceEvalChain._callc<|j|d|dgIdH}t|}||}d|iS)a:Asynchronously compute the score for a prediction and reference. Args: inputs (Dict[str, Any]): The input data. run_manager (AsyncCallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. rsrtNr;r.aembed_documentsrOr{r^r=rwrxembeddedrYr;rrr_acall)  z!EmbeddingDistanceEvalChain._acallF)rt callbackstagsmetadatainclude_run_inforsrtrrrrkwargsc K"|||d||||d}||S)aEvaluate the embedding distance between a prediction and reference. Args: prediction (str): The output string from the first model. reference (str): The reference string (required) callbacks (Callbacks, optional): The callbacks to use. **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rsrtrwrrrrrA r=rsrtrrrrrr?rrr_evaluate_strings?s z,EmbeddingDistanceEvalChain._evaluate_stringsc ,|j||d||||dIdH}||S)aAsynchronously evaluate the embedding distance between a prediction and reference. Args: prediction (str): The output string from the first model. reference (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rrNacallrArrrr_aevaluate_stringsas z-EmbeddingDistanceEvalChain._aevaluate_stringsN)r#r$r%r&rbboolrir`rorrurrrrr~rrr rcrrrrrrrhs           &   rhc@s(eZdZdZedeefddZedefddZ dde ee fd e e de ee ffd d Z dde ee fd e ede ee ffd d Zddddddededede eede e ee fdede defddZddddddededede eede e ee fdede defddZdS)"PairwiseEmbeddingDistanceEvalChainaUse embedding distances to score semantic difference between two predictions. Examples: >>> chain = PairwiseEmbeddingDistanceEvalChain() >>> result = chain.evaluate_string_pairs(prediction="Hello", prediction_b="Hi") >>> print(result) {'score': 0.5} rcCrq)rrrs prediction_brr<rrrrurvz-PairwiseEmbeddingDistanceEvalChain.input_keyscCrj)Npairwise_embedding_rlrmr<rrrrorpz2PairwiseEmbeddingDistanceEvalChain.evaluation_nameNrwrxcCry)aCompute the score for two predictions. Args: inputs (Dict[str, Any]): The input data. run_manager (CallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. rsrr;rzr}rrrr~s z(PairwiseEmbeddingDistanceEvalChain._callcr)a/Asynchronously compute the score for two predictions. Args: inputs (Dict[str, Any]): The input data. run_manager (AsyncCallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. rsrNr;rrrrrrrz)PairwiseEmbeddingDistanceEvalChain._acallF)rrrrrsrrrrrrc Kr)aEvaluate the embedding distance between two predictions. Args: prediction (str): The output string from the first model. prediction_b (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. tags (List[str], optional): Tags to apply to traces metadata (Dict[str, Any], optional): metadata to apply to **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rsrrr r=rsrrrrrrr?rrr_evaluate_string_pairss z9PairwiseEmbeddingDistanceEvalChain._evaluate_string_pairsc r)aAsynchronously evaluate the embedding distance between two predictions. Args: prediction (str): The output string from the first model. prediction_b (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. tags (List[str], optional): Tags to apply to traces metadata (Dict[str, Any], optional): metadata to apply to traces **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rrNrrrrr_aevaluate_string_pairss z:PairwiseEmbeddingDistanceEvalChain._aevaluate_string_pairsr)r#r$r%r&rbrr`rurorrrrr~rrr rrcrrrrrrrs          (   r)"r&enumrtypingrrrrnumpyrO langchain_core.callbacks.managerrrr langchain_core.embeddingsr langchain_core.utilsr pydanticr r langchain.chains.baserlangchain.evaluation.schemarrlangchain.schemarrr`rr,rhrrrrrs(     8