ChatGPT 嵌入能够将文本转换为固定长度的连续向量，允许对文本数据执行分类 、主题聚类 、新官细中新技搜索和推荐等功能 。口嵌这样，以前很难被处理的量式零基文本数据可以轻松地被处理了 。
使用 ChatGPT 嵌入可以极大地改善用户体验 。文本握全它能够帮助聊天机器人更准确地处理文本信息，并实现更有效的全网文本搜索与推荐 、体验更为流畅的英文源码交互式会话，从而更好地满足用户需求。
OpenAI’s text embeddings measure the relatedness of text strings. Embeddings are commonly used for:
OpenAI的文本嵌入测量文本字符串的相关性 。嵌入通常用于：
An embedding is a vector (list) of floating point numbers. The distance between two vectors measures their relatedness. Small distances suggest high relatedness and large distances suggest low relatedness.
嵌入是浮点数的向量（列表） 。两个向量之间的程助础快距离测量它们的相关性
。小的最转换最详指南距离表示高相关性，大的距离表示低相关性
。
Visit our pricing page to learn about Embeddings pricing. Requests are billed based on the number of tokens in the input sent.
请访问我们的新官细中新技定价页面了解嵌入定价
。请求根据发送的口嵌输入中的标记数计费 。
To see embeddings in action,量式零基 check out our code samples
要查看嵌入实际使用，请查看我们的代码示例，详见博客下部分 Use Cases
To get an embedding, send your text string to the embeddings API endpoint along with a choice of embedding model ID (e.g., text-embedding-ada-002). The response will contain an embedding, which you can extract, save, and use.
要获取嵌入，请将文本字符串发送到嵌入API端点沿着选择嵌入模型ID（例如， text-embedding-ada-002）。响应将包含一个嵌入，您可以提取 、文本握全保存和使用它 。
Example requests: 请求的示例
response = openai.Embedding.create(input="Your text string goes here",model="text-embedding-ada-002")embeddings = response['data'][0]['embedding']curl https://api.openai.com/v1/embeddings \-H "Content-Type: application/json" \-H "Authorization: Bearer $OPENAI_API_KEY" \-d '{"input": "Your text string goes here","model": "text-embedding-ada-002"}'Example response: 返回的示例
{1:'negative', 2:'negative', 4:'positive', 5:'positive'})labels = ['negative', 'positive']label_embeddings = [get_embedding(label, model=model) for label in labels]def label_score(review_embedding, label_embeddings):return cosine_similarity(review_embedding, label_embeddings[1]) - cosine_similarity(review_embedding, label_embeddings[0])prediction = 'positive' if label_score('Sample Review', label_embeddings) > 0 else 'negative'User_and_product_embeddings.ipynb
We can obtain a user embedding by averaging over all of their reviews. Similarly, we can obtain a product embedding by averaging over all the reviews about that product. In order to showcase the usefulness of this approach we use a subset of 50k reviews to cover more reviews per user and per product.
我们可以通过对所有评论进行平均来获得用户嵌入
。类似地，我们可以通过对关于该产品的所有评论进行平均来获得产品嵌入。为了展示这种方法的有用性，我们使用了50k评论的子集来覆盖每个用户和每个产品的更多评论 。
We evaluate the usefulness of these embeddings on a separate test set, where we plot similarity of the user and product embedding as a function of the rating. Interestingly, based on this approach, even before the user receives the product we can predict better than random whether they would like the product.
我们在一个单独的测试集上评估这些嵌入的有用性，在那里我们绘制用户和产品嵌入的相似性作为评级的函数 。有趣的是，基于这种方法，即使在用户收到产品之前，我们也可以比随机预测更好地预测他们是否喜欢该产品 。
Boxplot grouped by Score 箱线图按分数分组
user_embeddings = df.groupby('UserId').ada_embedding.apply(np.mean)prod_embeddings = df.groupby('ProductId').ada_embedding.apply(np.mean)Clustering.ipynb
Clustering is one way of making sense of a large volume of textual data. Embeddings are useful for this task, as they provide semantically meaningful vector representations of each text. Thus, in an unsupervised way, clustering will uncover hidden groupings in our dataset.
聚类是理解大量文本数据的一种方法 。嵌入对于这项任务很有用，因为它们提供了每个文本的语义上有意义的向量表示 。因此，以无监督的方式，聚类将揭示数据集中隐藏的分组 。
In this example, we discover four distinct clusters: one focusing on dog food, one on negative reviews, and two on positive reviews.
在这个例子中，我们发现了四个不同的集群：一个关注狗粮，一个关注负面评价，两个关注正面评价 。
Clusters identified visualized in language 2d using t-SNE 使用t-SNE在语言2d中可视化地识别集群
import numpy as npfrom sklearn.cluster import KMeansmatrix = np.vstack(df.ada_embedding.values)n_clusters = 4kmeans = KMeans(n_clusters = n_clusters, init='k-means++', random_state=42)kmeans.fit(matrix)df['Cluster'] = kmeans.labels_Semantic_text_search_using_embeddings.ipynb
To retrieve the most relevant documents we use the cosine similarity between the embedding vectors of the query and each document, and return the highest scored documents.
为了检索最相关的文档，我们使用查询和每个文档的嵌入向量之间的余弦相似度，并返回得分最高的文档。
from openai.embeddings_utils import get_embedding, cosine_similaritydef search_reviews(df, product_description, n=3, pprint=True):embedding = get_embedding(product_description, model='text-embedding-ada-002')df['similarities'] = df.ada_embedding.apply(lambda x: cosine_similarity(x, embedding))res = df.sort_values('similarities', ascending=False).head(n)return resres = search_reviews(df, 'delicious beans', n=3)Code_search.ipynb
Code search works similarly to embedding-based text search. We provide a method to extract Python functions from all the Python files in a given repository. Each function is then indexed by the text-embedding-ada-002model.
代码搜索的工作原理与基于嵌入的文本搜索类似。我们提供了一种方法，可以从给定存储库中的所有Python文件中提取Python函数 。然后每个函数由 text-embedding-ada-002模型索引 。
To perform a code search, we embed the query in natural language using the same model. Then we calculate cosine similarity between the resulting query embedding and each of the function embeddings. The highest cosine similarity results are most relevant.
为了执行代码搜索，我们使用相同的模型将查询嵌入到自然语言中 。然后，我们计算得到的查询嵌入和每个函数嵌入之间的余弦相似度 。最高余弦相似性结果是最相关的 。
from openai.embeddings_utils import get_embedding, cosine_similaritydf['code_embedding'] = df['code'].apply(lambda x: get_embedding(x, model='text-embedding-ada-002'))def search_functions(df, code_query, n=3, pprint=True, n_lines=7):embedding = get_embedding(code_query, model='text-embedding-ada-002')df['similarities'] = df.code_embedding.apply(lambda x: cosine_similarity(x, embedding))res = df.sort_values('similarities', ascending=False).head(n)return resres = search_functions(df, 'Completions API tests', n=3)Recommendation_using_embeddings.ipynb
Because shorter distances between embedding vectors represent greater similarity, embeddings can be useful for recommendation.
因为嵌入向量之间的较短距离表示较大的相似性，所以嵌入对于推荐是有用的
。
Below, we illustrate a basic recommender. It takes in a list of strings and one ‘source’ string, computes their embeddings, and then returns a ranking of the strings, ranked from most similar to least similar. As a concrete example, the linked notebook below applies a version of this function to the AG news dataset (sampled down to 2,000 news article descriptions) to return the top 5 most similar articles to any given source article.
下面，我们展示一个基本的推荐器 。它接受一个字符串列表和一个“源”字符串，计算它们的嵌入，然后返回字符串的排序，从最相似到最不相似排序 。作为一个具体的例子，下面的链接笔记本将此函数的一个版本应用于AG新闻数据集（采样到2,000篇新闻文章描述），以返回与任何给定源文章最相似的前5篇文章 。
def recommendations_from_strings(strings: List[str],index_of_source_string: int,model="text-embedding-ada-002",) -> List[int]:"""Return nearest neighbors of a given string."""# get embeddings for all stringsembeddings = [embedding_from_string(string, model=model) for string in strings]# get the embedding of the source stringquery_embedding = embeddings[index_of_source_string]# get distances between the source embedding and other embeddings (function from embeddings_utils.py)distances = distances_from_embeddings(query_embedding, embeddings, distance_metric="cosine")# get indices of nearest neighbors (function from embeddings_utils.py)indices_of_nearest_neighbors = indices_of_nearest_neighbors_from_distances(distances)return indices_of_nearest_neighborsOur embedding models may be unreliable or pose social risks in certain cases, and may cause harm in the absence of mitigations.
我们的嵌入模型在某些情况下可能不可靠或构成社会风险，并且在缺乏缓解措施的情况下可能造成伤害
。
Limitation: The models encode social biases, e.g. via stereotypes or negative sentiment towards certain groups.
局限性：这些模型对社会偏差进行了编码，例如通过对某些群体的刻板印象或负面情绪。
We found evidence of bias in our models via running the SEAT (May et al, 2019) and the Winogender (Rudinger et al, 2018) benchmarks. Together, these benchmarks consist of 7 tests that measure whether models contain implicit biases when applied to gendered names, regional names, and some stereotypes.
我们通过运行SEAT（May et al，2019）和Winogender（Rudinger et al，2018）基准测试发现了模型中存在偏差的证据。这些基准包括7个测试，用于衡量模型在应用于性别名称，地区名称和一些刻板印象时是否包含隐含的偏见 。
For example, we found that our models more strongly associate (a) European American names with positive sentiment, when compared to African American names, and (b) negative stereotypes with black women.
例如，我们发现，我们的模型更强烈地将（a）欧洲裔美国人的名字与积极情绪相关联，与非洲裔美国人的名字相比，以及（B）与黑人女性的负面刻板印象 。
These benchmarks are limited in several ways: (a) they may not generalize to your particular use case, and (b) they only test for a very small slice of possible social bias.
这些基准在几个方面受到限制：（a）它们可能不会推广到您的特定用例，（B）它们只测试可能的社会偏见的一小部分 。
These tests are preliminary, and we recommend running tests for your specific use cases.These results should be taken as evidence of the existence of the phenomenon, not a definitive characterization of it for your use case. Please see our usage policies for more details and guidance.
这些测试是初步的，我们建议您针对特定用例运行测试 。这些结果应该被视为现象存在的证据，而不是对您的用例的明确描述 。请参阅我们的使用政策了解更多详情和指导 。
Please contact our support team via chat if you have any questions; we are happy to advise on this.
如果您有任何问题，请通过聊天联系我们的支持团队;我们很乐意就此提供意见 。
Limitation: Models lack knowledge of events that occurred after August 2020.
局限性：模型缺乏对2020年8月之后发生的事件的了解 。
Our models are trained on datasets that contain some information about real world events up until 8/2020. If you rely on the models representing recent events, then they may not perform well.
我们的模型是在包含一些真实的世界事件信息的数据集上训练的，直到2020年8月。如果你依赖于代表最近事件的模型，那么它们可能表现不佳 。
In Python, you can split a string into tokens with OpenAI’s tokenizertiktoken.
在Python中，你可以使用OpenAI的标记器 tiktoken将一个字符串分割成多个标记 。
Example code:
import tiktokendef num_tokens_from_string(string: str, encoding_name: str) -> int:"""Returns the number of tokens in a text string."""encoding = tiktoken.get_encoding(encoding_name)num_tokens = len(encoding.encode(string))return num_tokensnum_tokens_from_string("tiktoken is great!", "cl100k_base")For second-generation embedding models like text-embedding-ada-002, use the cl100k_baseencoding.
对于像 text-embedding-ada-002这样的第二代嵌入模型，请使用 cl100k_base编码。
More details and example code are in the OpenAI Cookbook guide how to count tokens with tiktoken.
更多细节和示例代码在OpenAI Cookbook指南中如何使用tiktoken进行标记计数
。
For searching over many vectors quickly, we recommend using a vector database. You can find examples of working with vector databases and the OpenAI API in our Cookbook on GitHub.
为了快速搜索多个矢量，我们建议使用矢量数据库 。您可以在GitHub上的Cookbook中找到使用矢量数据库和OpenAI API的示例。
Vector database options include: 矢量数据库选项包括：
We recommend cosine similarity. The choice of distance function typically doesn’t matter much.
我们推荐余弦相似性
。距离函数的选择通常并不重要。
OpenAI embeddings are normalized to length 1, which means that:
OpenAI嵌入规范化为长度1，这意味着：
Customers own their input and output from our models, including in the case of embeddings. You are responsible for ensuring that the content you input to our API does not violate any applicable law or our Terms of Use.
客户拥有他们的输入和我们模型的输出，包括嵌入的情况。您有责任确保您输入我们API的内容不违反任何适用法律或我们的使用条款
。
如果大家想继续了解人工智能相关学习路线和知识体系，欢迎大家翻阅我的另外一篇博客《重磅 | 完备的人工智能AI 学习——基础知识学习路线，所有资料免关注免套路直接网盘下载》
这篇博客参考了Github知名开源平台，AI技术平台以及相关领域专家：Datawhale，ApacheCN，AI有道和黄海广博士等约有近100G相关资料，希望能帮助到所有小伙伴们
。
chatgpt免费软件,chatgpt api 免费接口,chatgpt 聊天机器人教程,chatgpt 指令大全,chatgpt app
CHATGPT不仅是一个强大的工具 ，还是创意的源泉 。百度的CHATGPT可以帮助用户生成有趣、富有创意的文本内容 ，为内容创作者提供了全新的创作灵感 。