GraphRagV2.1.0整合DeepSeek(附Python代码)

📅 2026/7/29 18:37:20 👁️ 阅读次数 📝 编程学习
GraphRagV2.1.0整合DeepSeek(附Python代码)

参考资料:

GraphRagV1.1.0整合DeepSeek(附Python代码)

GraphRag官方GitHub

个人网盘资源,包含各种课件,源码等(密码是本人的身份证号后四位)

2025最新版微软GraphRAG 2.0.0本地部署教程:基于Ollama快速构建知识图谱


官方GitHub说明:

在官方的GitHub结构中,有一个docs文件夹,说明文档,关于部署以及代码的相关编写里面都有详细的说明


GraphRag最新版部署步骤:

环境准备

需要准备python环境为:Python 3.10-3.12(推荐3.12.4,兼容性强)

关于python虚拟环境的部署,可以参考这两个文章:

Python入门------基础环境配置(Windows)

Python入门------多个版本--虚拟环境的创建(非anaconda方式)

GraphRag的部署步骤

1. 首先进入到创建的虚拟环境(这里选择python3.11)

2. 激活环境(activate)

3. 安装GraphRag(版本为最新版2.1.0)

pip install graphrag

4. 创建GraphRag的工作目录

这里为了方便,就创建在虚拟环境里面的,也可以创建在指定的目录下

mkdir "openl\input"

注意:这个openl是可以改的,但是input一定是这个名字

5. 然后用这个目录作为工作空间,初始化环境

graphrag init --root ./openl

这样就生成了GraphRag必要的文件,打开openl就可以看到

6. 配置相关配置文件(这里是整合deepSeek,embeding用的是AzureOpenAI)

(1) 首先配置deepseek的API-KEY到.env文件中

(2) 然后配置settings.yaml文件

(其实跟1.1.0的配置方式差不多)

(只需要改models的default_chat_model和default_embedding_model这两块)

models: default_chat_model: type: openai_chat # or azure_openai_chat api_base: https://api.deepseek.com # api_version: 2024-05-01-preview auth_type: api_key # or azure_managed_identity api_key: ${GRAPHRAG_API_KEY} # set this in the generated .env file # audience: "https://cognitiveservices.azure.com/.default" # organization: <organization_id> model: deepseek-chat # deployment_name: <azure_model_deployment_name> encoding_model: cl100k_base # automatically set by tiktoken if left undefined model_supports_json: true # recommended if this is available for your model. concurrent_requests: 25 # max number of simultaneous LLM requests allowed async_mode: threaded # or asyncio retry_strategy: native max_retries: -1 # set to -1 for dynamic retry logic (most optimal setting based on server response) tokens_per_minute: 0 # set to 0 to disable rate limiting requests_per_minute: 0 # set to 0 to disable rate limiting default_embedding_model: type: azure_openai_embedding # or azure_openai_embedding api_base: https://gpt4-xxxx-yyyy.openai.azure.com api_version: {申请azure_openai_key的version} auth_type: api_key # or azure_managed_identity api_key: {填写azure_openai_key} # audience: "https://cognitiveservices.azure.com/.default" # organization: <organization_id> model: text-embedding-3-large deployment_name: text-embedding-3-large encoding_model: cl100k_base # automatically set by tiktoken if left undefined model_supports_json: true # recommended if this is available for your model. concurrent_requests: 25 # max number of simultaneous LLM requests allowed async_mode: threaded # or asyncio retry_strategy: native max_retries: -1 # set to -1 for dynamic retry logic (most optimal setting based on server response) tokens_per_minute: 0 # set to 0 to disable rate limiting requests_per_minute: 0 # set to 0 to disable rate limiting

(3) 最好使用代码确认下你的azure_openAI_key是否正确,有时间的话最好再核对下Deepseek的api_key

我这边使用ChatGpt帮忙写了如下代码

import openai def check_azure_openai_key(api_key: str, endpoint: str, deployment_name: str): """ 检查 Azure OpenAI API Key 是否有效。 :param api_key: Azure OpenAI 的 API Key :param endpoint: Azure OpenAI 端点 (例如 https://your-resource-name.openai.azure.com) :param deployment_name: 部署的模型名称 :return: 是否可访问 """ try: openai.api_type = "azure" openai.api_key = "084d862fa66c4c82886a4b0bb9214ab1" openai.api_base = endpoint openai.api_version = "2024-02-15-preview" # 根据 Azure 版本调整 client = openai.AzureOpenAI(api_key=api_key, api_version="2024-02-15-preview", azure_endpoint=endpoint) response = client.chat.completions.create( model=deployment_name, messages=[{"role": "system", "content": "You are an AI assistant."}, {"role": "user", "content": "Hello"}] ) print("访问成功,API Key 有效!") return True except openai.AuthenticationError: print("访问失败,API Key 无效!") return False except openai.OpenAIError as e: print(f"访问失败,错误信息: {e}") return False # 示例用法 API_KEY = "084d862fa66c4c82886a4b0bb9214ab1" ENDPOINT = "https://gpt4-my-link.openai.azure.com" DEPLOYMENT_NAME = "text-embedding-3-large" # 请替换为你的部署名称 check_azure_openai_key(API_KEY, ENDPOINT, DEPLOYMENT_NAME)

7. 在确定好配置文件都配置正确的情况下,那么就上传数据集,就是你需要让AI帮你总结训练的文档,放置于上述创建的input目录中

8. 然后就围绕这个文件生成问答的知识图谱(可能比较慢,出错了大概率是配置文件的问题)

graphrag index --root ./openl

9. 生成知识图谱后,就可以基于这个知识图谱进行问答了

(1) 回答方式分为两种

  • 本地搜索(LocalSearch)

通过结合 AI 提取的知识图谱中的相关数据和原始文档的文本块来生成答案。这种方法适用于需要理解文档中提到的特定实体的问题(例如,洋甘菊的治愈属性是什么?)。

  • 全局搜索(GlobalSearch)

搜索所有 AI 生成的社区报告来生成答案。这是一种资源密集型方法,但通常对于需要整体理解数据集的问题能给出很好的响应(例如,这个笔记本中提到的草药最重要的价值是什么?)。

(2) 控制台的方式进行问答

  • 本地搜索(LocalSearch)
graphrag query --method local --query "请帮我介绍下ID3算法" --config ./openl/settings.yaml --data ./openl/output --root ./openl

相关参数解释:

graphrag query : graphrag的查询命令

--method local : 表示使用本地搜索

--query "请帮我介绍下ID3算法" :表示查询

--config ./openl/settings.yaml :加载我们配置好的yml文件

--data ./openl/output :加载知识图谱

--root ./openl :工作目录

  • 全局搜索(GlobalSearch)
graphrag query --method global --query "请帮我介绍下ID3算法" --config ./openl/settings.yaml --data ./openl/output --root ./openl

python代码访问方式因为篇幅原因,另起一个篇幅说


Python访问Graphrag+DeepSeek+azureAi:

下面将结合官方的GitHub进行代码的编写和说明

GitHub上的文件说明

下面就是基于这两个文件,编写的代码,直接贴代码

准备环境
  • python版本为 :3.10~3.12
  • pip安装的graphrag版本为:2.1.0

pip install graphrag==2.1.0

本地查询的相关代码

# Copyright (c) 2024 Microsoft Corporation. # Licensed under the MIT License. import asyncio import os import pandas as pd import tiktoken from graphrag.query.context_builder.entity_extraction import EntityVectorStoreKey from graphrag.query.indexer_adapters import ( read_indexer_covariates, read_indexer_entities, read_indexer_relationships, read_indexer_reports, read_indexer_text_units, ) from graphrag.query.question_gen.local_gen import LocalQuestionGen from graphrag.query.structured_search.local_search.mixed_context import ( LocalSearchMixedContext, ) from graphrag.query.structured_search.local_search.search import LocalSearch from graphrag.vector_stores.lancedb import LanceDBVectorStore from graphrag.config.enums import ModelType from graphrag.config.models.language_model_config import LanguageModelConfig from graphrag.language_model.manager import ModelManager # index命令执行后的outut目录 INPUT_DIR = "E:\python_envs\my_env_3.11_graphrag_new_version\Scripts\openl\output" LANCEDB_URI = f"{INPUT_DIR}/lancedb" COMMUNITY_REPORT_TABLE = "community_reports" ENTITY_TABLE = "entities" COMMUNITY_TABLE = "communities" RELATIONSHIP_TABLE = "relationships" COVARIATE_TABLE = "covariates" TEXT_UNIT_TABLE = "text_units" COMMUNITY_LEVEL = 2 # read nodes table to get community and degree data entity_df = pd.read_parquet(f"{INPUT_DIR}/{ENTITY_TABLE}.parquet") community_df = pd.read_parquet(f"{INPUT_DIR}/{COMMUNITY_TABLE}.parquet") entities = read_indexer_entities(entity_df, community_df, COMMUNITY_LEVEL) # load description embeddings to an in-memory lancedb vectorstore # to connect to a remote db, specify url and port values. description_embedding_store = LanceDBVectorStore( collection_name="default-entity-description", ) description_embedding_store.connect(db_uri=LANCEDB_URI) print(f"Entity count: {len(entity_df)}") entity_df.head() relationship_df = pd.read_parquet(f"{INPUT_DIR}/{RELATIONSHIP_TABLE}.parquet") relationships = read_indexer_relationships(relationship_df) print(f"Relationship count: {len(relationship_df)}") relationship_df.head() # NOTE: covariates are turned off by default, because they generally need prompt tuning to be valuable # Please see the GRAPHRAG_CLAIM_* settings # covariate_df = pd.read_parquet(f"{INPUT_DIR}/{COVARIATE_TABLE}.parquet") # claims = read_indexer_covariates(covariate_df) # print(f"Claim records: {len(claims)}") # covariates = {"claims": claims} report_df = pd.read_parquet(f"{INPUT_DIR}/{COMMUNITY_REPORT_TABLE}.parquet") reports = read_indexer_reports(report_df, community_df, COMMUNITY_LEVEL) print(f"Report records: {len(report_df)}") report_df.head() text_unit_df = pd.read_parquet(f"{INPUT_DIR}/{TEXT_UNIT_TABLE}.parquet") text_units = read_indexer_text_units(text_unit_df) print(f"Text unit records: {len(text_unit_df)}") text_unit_df.head() # 下面的这些配置 和yaml中的配置要基本一致 sd_api_key = "{这里填写deepseek的api_key}" openai_api_key = "{这里填写azure_openai_key}" llm_model = "deepseek-chat" embedding_model = "text-embedding-3-large" llm_api_base = "https://api.deepseek.com" embedding_model_api_base = "https://gpt4-xxxx-yyyy.openai.azure.com" # 上面的这些配置 要和 yaml 中的配置要基本一致 chat_config = LanguageModelConfig( api_key=sd_api_key, type=ModelType.OpenAIChat, model=llm_model, api_base=llm_api_base, max_retries=20, ) chat_model = ModelManager().get_or_create_chat_model( name="local_search", model_type=ModelType.OpenAIChat, config=chat_config, ) token_encoder = tiktoken.encoding_for_model(llm_model) embedding_config = LanguageModelConfig( type=ModelType.AzureOpenAIEmbedding, api_base=embedding_model_api_base, api_version="2024-02-15-preview", auth_type='api_key', api_key=openai_api_key, model=embedding_model, deployment_name=embedding_model, encoding_model='cl100k_base', max_retries=20, ) text_embedder = ModelManager().get_or_create_embedding_model( name="local_search_embedding", model_type=ModelType.AzureOpenAIEmbedding, config=embedding_config, ) context_builder = LocalSearchMixedContext( community_reports=reports, text_units=text_units, entities=entities, relationships=relationships, # if you did not run covariates during indexing, set this to None # covariates=covariates, entity_text_embeddings=description_embedding_store, embedding_vectorstore_key=EntityVectorStoreKey.ID, # if the vectorstore uses entity title as ids, set this to EntityVectorStoreKey.TITLE text_embedder=text_embedder, token_encoder=token_encoder, ) # text_unit_prop: proportion of context window dedicated to related text units # community_prop: proportion of context window dedicated to community reports. # The remaining proportion is dedicated to entities and relationships. Sum of text_unit_prop and community_prop should be <= 1 # conversation_history_max_turns: maximum number of turns to include in the conversation history. # conversation_history_user_turns_only: if True, only include user queries in the conversation history. # top_k_mapped_entities: number of related entities to retrieve from the entity description embedding store. # top_k_relationships: control the number of out-of-network relationships to pull into the context window. # include_entity_rank: if True, include the entity rank in the entity table in the context window. Default entity rank = node degree. # include_relationship_weight: if True, include the relationship weight in the context window. # include_community_rank: if True, include the community rank in the context window. # return_candidate_context: if True, return a set of dataframes containing all candidate entity/relationship/covariate records that # could be relevant. Note that not all of these records will be included in the context window. The "in_context" column in these # dataframes indicates whether the record is included in the context window. # max_tokens: maximum number of tokens to use for the context window. local_context_params = { "text_unit_prop": 0.5, "community_prop": 0.1, "conversation_history_max_turns": 5, "conversation_history_user_turns_only": True, "top_k_mapped_entities": 10, "top_k_relationships": 10, "include_entity_rank": True, "include_relationship_weight": True, "include_community_rank": False, "return_candidate_context": False, "embedding_vectorstore_key": EntityVectorStoreKey.ID, # set this to EntityVectorStoreKey.TITLE if the vectorstore uses entity title as ids "max_tokens": 12_000, # change this based on the token limit you have on your model (if you are using a model with 8k limit, a good setting could be 5000) } model_params = { "max_tokens": 2_000, # change this based on the token limit you have on your model (if you are using a model with 8k limit, a good setting could be 1000=1500) "temperature": 0.0, } search_engine = LocalSearch( model=chat_model, context_builder=context_builder, token_encoder=token_encoder, model_params=model_params, context_builder_params=local_context_params, response_type="multiple paragraphs", # free form text describing the response type and format, can be anything, e.g. prioritized list, single paragraph, multiple paragraphs, multiple-page report ) async def main(): print('开始进行查询') result = await search_engine.search("请用中文帮我介绍下ID3决策树算法") print(result.response) # inspect the data used to build the context for the LLM responses result.context_data["reports"] # inspect number of LLM calls and tokens print( f"LLM calls: {result.llm_calls}. Prompt tokens: {result.prompt_tokens}. Output tokens: {result.output_tokens}." ) if __name__ == "__main__": asyncio.run(main())

全局查询的相关代码

import asyncio import os import pandas as pd import tiktoken from graphrag.config.enums import ModelType from graphrag.config.models.language_model_config import LanguageModelConfig from graphrag.language_model.manager import ModelManager from graphrag.query.indexer_adapters import ( read_indexer_communities, read_indexer_entities, read_indexer_reports, ) from graphrag.query.structured_search.global_search.community_context import ( GlobalCommunityContext, ) from graphrag.query.structured_search.global_search.search import GlobalSearch api_key = "{这里填写deepseek的api-key}" llm_model = "deepseek-chat" llm_api_base = "https://api.deepseek.com" config = LanguageModelConfig( api_key=api_key, type=ModelType.OpenAIChat, api_base=llm_api_base, model=llm_model, max_retries=20, ) model = ModelManager().get_or_create_chat_model( name="global_search", model_type=ModelType.OpenAIChat, config=config, ) token_encoder = tiktoken.encoding_for_model(llm_model) # parquet files generated from indexing pipeline INPUT_DIR = "E:\python_envs\my_env_3.11_graphrag_new_version\Scripts\openl\output" COMMUNITY_TABLE = "communities" COMMUNITY_REPORT_TABLE = "community_reports" ENTITY_TABLE = "entities" # community level in the Leiden community hierarchy from which we will load the community reports # higher value means we use reports from more fine-grained communities (at the cost of higher computation cost) COMMUNITY_LEVEL = 2 community_df = pd.read_parquet(f"{INPUT_DIR}/{COMMUNITY_TABLE}.parquet") entity_df = pd.read_parquet(f"{INPUT_DIR}/{ENTITY_TABLE}.parquet") report_df = pd.read_parquet(f"{INPUT_DIR}/{COMMUNITY_REPORT_TABLE}.parquet") communities = read_indexer_communities(community_df, report_df) reports = read_indexer_reports(report_df, community_df, COMMUNITY_LEVEL) entities = read_indexer_entities(entity_df, community_df, COMMUNITY_LEVEL) print(f"Total report count: {len(report_df)}") print( f"Report count after filtering by community level {COMMUNITY_LEVEL}: {len(reports)}" ) report_df.head() context_builder = GlobalCommunityContext( community_reports=reports, communities=communities, entities=entities, # default to None if you don't want to use community weights for ranking token_encoder=token_encoder, ) context_builder_params = { "use_community_summary": False, # False means using full community reports. True means using community short summaries. "shuffle_data": True, "include_community_rank": True, "min_community_rank": 0, "community_rank_name": "rank", "include_community_weight": True, "community_weight_name": "occurrence weight", "normalize_community_weight": True, "max_tokens": 12_000, # change this based on the token limit you have on your model (if you are using a model with 8k limit, a good setting could be 5000) "context_name": "Reports", } map_llm_params = { "max_tokens": 1000, "temperature": 0.0, "response_format": {"type": "json_object"}, } reduce_llm_params = { "max_tokens": 2000, # change this based on the token limit you have on your model (if you are using a model with 8k limit, a good setting could be 1000-1500) "temperature": 0.0, } search_engine = GlobalSearch( model=model, context_builder=context_builder, token_encoder=token_encoder, max_data_tokens=12_000, # change this based on the token limit you have on your model (if you are using a model with 8k limit, a good setting could be 5000) map_llm_params=map_llm_params, reduce_llm_params=reduce_llm_params, allow_general_knowledge=False, # set this to True will add instruction to encourage the LLM to incorporate general knowledge in the response, which may increase hallucinations, but could be useful in some use cases. json_mode=True, # set this to False if your LLM model does not support JSON mode. context_builder_params=context_builder_params, concurrent_coroutines=32, response_type="multiple paragraphs", # free form text describing the response type and format, can be anything, e.g. prioritized list, single paragraph, multiple paragraphs, multiple-page report ) async def main(): print('开始进行查询') result = await search_engine.search("请用中文帮我介绍下ID3决策树算法") print(result.response) # inspect the data used to build the context for the LLM responses result.context_data["reports"] # inspect number of LLM calls and tokens print( f"LLM calls: {result.llm_calls}. Prompt tokens: {result.prompt_tokens}. Output tokens: {result.output_tokens}." ) if __name__ == "__main__": asyncio.run(main())