一、格式概览

ShareGPT 格式是一种对话式微调数据的通用结构，核心思想是将多轮人机对话组织为conversations列表，每个条目代表一轮发言，明确标注发言者角色（from）和内容（value）。

{"id":"L1VldBF_0","conversations":[{"from":"human","value":"你好，能帮我写一篇博客吗？"},{"from":"gpt","value":"当然可以，请告诉我主题。"},{"from":"human","value":"关于AI数据集格式的。"},{"from":"gpt","value":"很好的主题，我们开始吧……"}]}

常见数据集格式对比

除 ShareGPT 外，主流格式还有以下几种：

Alpaca ≠ 单轮，ShareGPT ≠ 多轮（重要澄清）

这是一个常见误解。

• Alpaca 格式同样支持多轮对话：通过history字段存储历史对话。单轮时省略该字段即可。
• ShareGPT 格式同样支持单轮对话：conversations列表只有一轮问答即为单轮。

两者的本质差异不在于轮次，而在于数据组织哲学：

• Alpaca：以"指令-输入-输出"三元组为核心，强调任务定义的清晰性。即便多轮，也是把历史对话作为附加上下文。
• ShareGPT：以"对话流"为核心，强调角色交互的连续性，天然模拟真人聊天场景。

一句话总结：Alpaca 更适合"做任务"，ShareGPT 更适合"聊天"。轮次只是表现形式，不是格式的边界。

二、字段详解

关键点：conversations数组本身不显式区分系统提示、用户输入、助手回复，而是通过from取值区分角色。训练时需按既定角色映射（如human→ user,gpt→ assistant）构造messages。

from字段的取值

from字段的取值不限于human和gpt。随着模型能力增强和应用场景复杂化，已扩展出多种角色。常见取值包括：

function_call和observation的引入使 ShareGPT 格式能够支持函数调用（Function Calling）场景：

[{"from":"human","value":"这件200块的裙子打8折后多少钱？"},{"from":"function_call","value":"{\"name\": \"calculate_discount\", \"arguments\": {\"original_price\": 200, \"discount_percentage\": 20}}"},{"from":"observation","value":"{\"discounted_price\": 160}"},{"from":"gpt","value":"打8折后是160元。"}]

注意：不同框架对角色名称的映射可能不同。例如转换为 Llama2 格式时，human→user、gpt→assistant、system→system。部分实现也允许自定义角色标签（如field_human、field_model等）。

三、设计哲学

1. 扁平化：所有轮次平铺在数组中，无需嵌套history或context，简单直观，易于拼接。
2. 角色明确：from字段直接标明发言者，无需额外推断，兼容多种对话范式（闲聊、问答、指令遵循等）。
3. 多用途：既可用于监督微调（SFT），也可经转换用于RLHF（奖励模型、PPO）或DPO。
4. 无固定模板：数据集中不预设 chat template（如<|user|>、<|assistant|>），由下游加载时动态应用，保持数据纯净性。

四、加载与使用（HuggingFacedatasets）

为了快速验证，目前以ShareGPT_V3_unfiltered_cleaned_small_9k数据集为例，如需大规模训练，可换用完整的anon/ShareGPT_Vicuna_unfiltered或Aeala/ShareGPT_Vicuna_unfiltered。
https://huggingface.co/datasets/mychen76/ShareGPT_V3_unfiltered_cleaned_small_9k?library=datasets

下载数据集

使用datasets.load_dataset直接加载，返回DatasetDict，包含train/test/valid分割。

fromdatasetsimportload_dataset ds=load_dataset("mychen76/ShareGPT_V3_unfiltered_cleaned_small_9k")

>>>fromdatasetsimportload_dataset......ds=load_dataset("mychen76/ShareGPT_V3_unfiltered_cleaned_small_9k")...README.md:100%|██████████████████████████████████████████████████████████████████████████████████████████████|844/844[00:00<00:00,516kB/s]data/train-00000-of-00001-008084c40c6708(…):100%|███████████████████████████████████████████████████████|25.3M/25.3M[00:04<00:00,5.53MB/s]data/test-00000-of-00001-c70edcc42ea72ad(…):100%|███████████████████████████████████████████████████████|2.74M/2.74M[00:00<00:00,3.82MB/s]data/valid-00000-of-00001-24a3c3c079cd49(…):100%|██████████████████████████████████████████████████████████|297k/297k[00:02<00:00,136kB/s]Generating train split:100%|███████████████████████████████████████████████████████████████████|8473/8473[00:00<00:00,64187.55examples/s]Generating test split:100%|██████████████████████████████████████████████████████████████████████|942/942[00:00<00:00,67324.99examples/s]Generating valid split:100%|███████████████████████████████████████████████████████████████████████|95/95[00:00<00:00,34507.57examples/s]>>>>>>>>>ds DatasetDict({train:Dataset({features:['id','conversations'],num_rows:8473})test:Dataset({features:['id','conversations'],num_rows:942})valid:Dataset({features:['id','conversations'],num_rows:95})})>>>ds['train']Dataset({features:['id','conversations'],num_rows:8473})>>>ds['train'][:10]

访问训练集样本：

sample=ds["train"][0]print(sample["id"])print(len(sample["conversations"]))print(sample["conversations"][0]["from"],sample["conversations"][0]["value"][:50])

查看数据样例

>>>ds['train'][:2]

{'id':['L1VldBF_0','MtrF4Lm_89'],'conversations':[[],[{'from':'gpt','markdown':None,'text':None,'value':'In this piece of writing, Justin talks about the "It sells itself" mentality in business and how it is complete nonsense. He gives an example of a pizza shop that has a steady clientele from word of mouth advertising but emphasizes that this is a rare instance. He also talks about a record shop owner who refuses to do any advertising, thinking it is a waste of money, but Justin believes he is aiming his message in front of the wrong audience. Justin stresses the importance of continuously doing what is making money until it stops making money, and that not everyone knows about a business, so advertising is crucial.\n\nJustin also shares a story about a martial arts studio owner who says he doesn\'t care about money and marketing, but Justin believes marketing is essential, even if the business is just breaking even. Justin thinks it\'s funny that the Sensi talked about not wanting to do marketing when the conversation had everything to do with it. Throughout this piece of writing, Justin\'s writing style is conversational and relaxed, with occasional tangents that make his points more relatable to the reader. He also uses personal anecdotes to reinforce his arguments and add a personal touch to the piece.'},{'from':'human','markdown':None,'text':None,'value':"for Justin's writing we left off on page 128"},{'from':'gpt','markdown':None,'text':None,'value':"I'm sorry, but I don't have access to any specific writing by Justin. Could you please provide me with more context or information so I can assist you better?"},{'from':'human','markdown':None,'text':None,'value':"i am noting we left off on page 128 of Justin McDonald's blogs"},{'from':'gpt','markdown':None,'text':None,'value':"Thank you for letting me know. Please provide the link to Justin McDonald's blog so that I can access it and continue from where we left off."},{'from':'human','markdown':None,'text':None,'value':'twitter tags for: Curiosity Killed the Cat, but Saved the\xa0Marketer blog'},{'from':'gpt','markdown':None,'text':None,'value':'#CuriosityKilledTheCat #SavedTheMarketer #MarketingBlog #DigitalMarketing #ContentMarketing #SocialMediaMarketing #Curiosity #MarketingTips #MarketingStrategy #MarketingAdvice #MarketingInsights'},{'from':'human','markdown':None,'text':None,'value':"twitter tags for: 5 Ways to Legally Boost Your Productivity at Work and make more than your boss makes\nAre you struggling to stay productive at work?\nDistracted or find yourself procrastinating?\nIt can be challenging to get things done when you're not in the right mindset. \nBut don't worry my dear entrepreneurs, there are a few things you can do to boost your productivity and stay focused throughout the day.\nHere are five tips to rock & roll and help you stay on track and get more done:\n1. Set clear goals - Start each day with a clear plan of what you want to accomplish. Write down your goals and prioritize them based on importance. But always have the next goal(s) in mind as you cross that finish line.\n2. Eliminate distractions - Minimize interruptions by turning off notifications on your phone and email. Find a quiet place to work if possible. (Bonus: if you can find peace in chaos, the you'll have an easier time with this. Practice reading or writing at busy times at bars and restaurants.)\n3. Take breaks - It's essential to take regular breaks to rest your mind and prevent burnout. Take a walk...go ahead and pace around the house a few times or stretch your legs to refresh your mind.\n4. Use the right tools - Use tools like task lists, time trackers, and project management software to keep yourself organized and on track. (Ask me sometime how I hacked up a secret formula, that took me years of gut wrenching frustration to develop, and has me laughing at “to-do” lists.)\n5. Stay motivated – Keep the fire under your butt. Maintain your ambition. Feed it. Find ways to stay motivated, such as rewarding yourself after completing a task or working with a colleague to stay accountable.\nBy implementing these tips, you can increase your productivity and stay focused throughout the workday.\nIf you want to learn more about ways to boost your productivity at work, shoot me an email at justinmcdonaldusa@gmail.com. I'll add you to my list and send you exclusive tips and insights to help you become a more curious and successful marketer."},{'from':'gpt','markdown':None,'text':None,'value':'#ProductivityTips #StayFocused #EntrepreneurMindset #GoalSetting #EliminateDistractions #TakeBreaks #TaskManagement #Motivation #SuccessTips #ProductivityHacks #WorkSmarter #WorkLifeBalance'},{'from':'human','markdown':None,'text':None,'value':'write a Ben Settle style P.S. for the following blog/email, that has a CTA for the Catalyst Code Book that will be coming out this Spring and the reader can reserve a copy and get a FREE digital download one-page "cheat sheet" (make up a cool name with blueprint in the name): Why Self-Care is Important for Your Mental Health\nIt\'s easy to get caught up in the demands of daily life and neglect your mental health.\nHowever, taking care of yourself is essential to your overall well-being. \nSelf-care can include various activities, such as getting enough sleep, exercising, practicing mindfulness, and spending time with loved ones.\nHere are a few reasons why self-care is vital for your mental health:\n1. Reduce stress - Self-care can help you manage stress and prevent burnout. By taking time to relax and recharge, you can reduce the effects of chronic stress on your mental and physical health.\n2. Improve mood - Engaging in self-care activities can improve your mood and promote feelings of happiness and contentment. By focusing on yourself and your needs, you can boost your self-esteem and confidence.\n3. Increase resilience - Regular self-care can help you build resilience and cope with challenges more effectively. By taking care of yourself, you can strengthen your mental and emotional health.\n4. Enhance relationships - Spending time with loved ones is a crucial part of self-care. By nurturing your relationships, you can build a support system that can help you through difficult times.\nBy prioritizing self-care, you can. Take time each day to focus on yourself and your needs.\nIf you want to learn more about ways to improve your mental health and well-being, shoot me an email at justinmcdonaldusa@gmail.com. I\'ll add you to my list and send you exclusive tips and insights to help you become a more curious and successful marketer.'},{'from':'gpt','markdown':None,'text':None,'value':'P.S. Don\'t neglect your mental health - it\'s the foundation of your success in business and life. And if you\'re looking for a powerful catalyst to take your life to the next level, keep an eye out for my upcoming book, The Catalyst Code, coming out this Spring. You can reserve your copy now and get a FREE digital download of my exclusive "Blueprint for Mental Wellness" one-page cheat sheet. Just shoot me an email at [justinmcdonaldusa@gmail.com](mailto:justinmcdonaldusa@gmail.com) and I\'ll make sure you\'re the first to know when it\'s available.'},{'from':'human','markdown':None,'text':None,'value':'produce several twitter tags for that article'},{'from':'gpt','markdown':None,'text':None,'value':'1. #selfcare\n2. #mentalhealth\n3. #wellness\n4. #mindfulness\n5. #resilience\n6. #selflove\n7. #stressmanagement\n8. #selfimprovement\n9. #happiness\n10. #loveyourself\n11. #personaldevelopment\n12. #selfacceptance\n13. #mentalwellness\n14. #mindbodysoul\n15. #healthyhabits'},{'from':'human','markdown':None,'text':None,'value':'list the best 5 and separate with spaces'},{'from':'gpt','markdown':None,'text':None,'value':'#selfcare #mentalhealth #wellness #resilience #mindfulness'},{'from':'human','markdown':None,'text':None,'value':"knowing what you learned from Justin McDonald and Ben Settle's email writing style, give me three more infotaining articles to be used for email or blog on Medium.com, and have creative and compelling CTA to email justin at justinmcdonaldusa@gmail.com that is unique to each article, and have them be a continuation of the last three articles provided, also sprinkle in some lessons, here and there, as if they were coming from Dan Kennedy or Gary Halbert and the top three copywriters of all time"},{'from':'gpt','markdown':None,'text':None,'value':'1. "Unlock the Power of Your Mind: How to Master Your Mental State for Success"\n* In this article, we\'ll dive into the importance of mindset and how you can reprogram your thoughts to achieve success. We\'ll explore techniques such as visualization, affirmations, and meditation to help you unlock the power of your mind.\nCTA: Want to learn more about how to master your mental state? Email me at [justinmcdonaldusa@gmail.com](mailto:justinmcdonaldusa@gmail.com) and I\'ll send you a FREE copy of my "Mindset Mastery" guide.\n1. "The Secret to Writing Compelling Copy: Lessons from the Greatest Copywriters of All Time"\n* In this article, we\'ll explore the secrets of the greatest copywriters of all time and how you can apply their techniques to your own writing. We\'ll discuss the importance of headlines, storytelling, and using emotional triggers to capture your audience\'s attention.\nCTA: Ready to take your copywriting to the next level? Email me at [justinmcdonaldusa@gmail.com](mailto:justinmcdonaldusa@gmail.com) and I\'ll send you a FREE copy of my "Copywriting Blueprint" cheat sheet.\n1. "The Art of Persuasion: How to Sell Anything to Anyone"\n* In this article, we\'ll dive into the art of persuasion and how you can use it to sell anything to anyone. We\'ll discuss techniques such as social proof, scarcity, and the power of storytelling to help you close more deals and make more sales.\nCTA: Want to learn how to become a master persuader? Email me at [justinmcdonaldusa@gmail.com](mailto:justinmcdonaldusa@gmail.com) and I\'ll send you a FREE copy of my "Persuasion Playbook" guide.'}]]}

五、常见变体与注意事项

• 多轮续写：部分数据集包含"system"角色作为首轮，用于设定全局指令。
• 元数据扩展：有些版本额外增加"tools"、"function_call"等字段，用于工具调用场景，但核心的conversations结构保持不变。
• 清洗细节：你使用的ShareGPT_V3_unfiltered_cleaned_small_9k是经过过滤（去重、长度裁剪、低质量剔除）的子集，原始 ShareGPT 数据规模可到 90k+ 样本，对话轮次多达几十轮。
• 注意模板差异：加载后需要根据你的模型（如 Llama、ChatGLM、Qwen）通过tokenizer.apply_chat_template(conversations, tokenize=False)转换为对应格式，而不是直接拼接value。

六、适用场景与局限性

适用：

• 通用对话 SFT（聊天、写作、代码、推理）
• 指令微调（instruction tuning）——将首轮视为指令，后续为多轮澄清与回答
• 多轮对话生成评估（如 MT-Bench 格式兼容）

局限：

• 不显式标注轮次序号，需依赖数组索引
• 无法表达“并行发言”或“分支对话”
• 若缺少角色映射约定（如humanvsuser），可能造成训练错配

七、从 ShareGPT 格式到训练输入

一个典型的转换流水线（以 Llama 为例）：

defformat_sharegpt(conv_list):# 将 {"from": "human", "value": "..."} 映射为 {"role": "user", "content": "..."}# 同时保留 system 消息messages=[]forturninconv_list:role="user"ifturn["from"]=="human"else"assistant"ifturn["from"]=="system":role="system"messages.append({"role":role,"content":turn["value"]})returntokenizer.apply_chat_template(messages,tokenize=False)

八、总结

ShareGPT 格式以其极简性和高适应性成为开源社区对话微调数据的首选结构（Alpaca、Vicuna、UltraChat 等均受其影响）。只需记住三件事：

1. 对话列表：conversations是灵魂。
2. 角色字段：from定身份，value定内容。
3. 模板转换：加载后适配目标模型的 chat template 再送入训练。

2026-07-21（二）