Model Mechanism and Training Methods of DeepSeek

DeepSeek · Official agreement

Read the agreement

23 of 23 cards

Legal Rights & Agreement Rules · Context

Who provides this model disclosure

Hangzhou DeepSeek Artificial Intelligence Co., Ltd. identifies itself as DeepSeek. This document explains its models and training methods to help users understand their operation and risks; its stated openness and security objectives are aims.

Supporting quotations

Hangzhou DeepSeek Artificial Intelligence Co., Ltd. (hereinafter "we" or "DeepSeek") is a research team dedicated to exploring AGI, focusing on fundamental model technology research and adhering to an open-source approach. We aim to promote technological inclusivity through openness, transparency, and security.

Data Access & Collection · Context

Personal-information rules are in a separate policy

DeepSeek directs users to its Privacy Policy for the specific rules on collecting, protecting, and using personal information. That linked policy is not part of this supplied document, so this disclosure alone does not establish all those rules.

Supporting quotations

For specific rules on how we collect, protect, and use personal information, please carefully read the DeepSeek Privacy Policy.

Not stated here

  • The linked Privacy Policy is not supplied in this assignment.

Use & Processing · Company practice

The disclosed models use training and inference

DeepSeek describes its current foundational models as large language models based on deep neural networks, with separate training and inference stages. This is its description at the time of the disclosure.

Supporting quotations

Currently, the foundational models provided by DeepSeek are all large-scale language models based on deep neural networks. These models operate in two main stages: the training phase and the inference phase. Additionally, DeepSeek models are open-source, so this section will also introduce our open-source efforts.

Use & Processing · Company practice

Training adjusts model parameters in two steps

DeepSeek describes development of deployable neural-network models with billions to trillions of parameters, optimized through gradient descent. It generally divides training into pre-training and optimization training.

Supporting quotations

The model training phase is the development stage, where developers create deployable models using designed training methods. The models consist of multi-layered neural networks with parameters ranging from billions to trillions. These parameters are continuously optimized during training through gradient descent algorithms. Model training can generally be divided into two steps: pre-training and optimization training.

Use & Processing · Company practice

Pre-training builds language ability, with further adjustments needed

Pre-training uses large-scale self-supervised learning on text to learn language patterns and knowledge associations. DeepSeek says this can produce coherent text, but may not yet yield accurate answers or task performance, so further training adjustments are needed.

Supporting quotations

Pre-training: The goal of pre-training is to train the model with datasets, enabling it to
acquire general language understanding and generation capabilities. During this phase, the model learns language patterns and knowledge associations from text data through large-scale self-supervised learning. After pre-training, the model can understand and generate coherent text but may not yet answer questions or perform tasks accurately. Further training adjustments are required.

Use & Processing · Company practice

Fine-tuning adapts the model to practical tasks

Optimization training, also called fine-tuning, adjusts the pre-trained model with task-specific data. DeepSeek says it typically uses supervised fine-tuning or reinforcement learning to follow instructions, align with human preferences, and develop domain expertise. Better practical performance is not a promise of accurate answers in every case.

Supporting quotations

Optimization Training: Also known as fine-tuning, optimization training further adjusts
the model parameters based on the pre-trained model using task-specific data to adapt it to real-world application scenarios. During this phase, the model typically employs supervised fine-tuning (SFT) or reinforcement learning (RL) methods to learn how to answer questions according to instructions, align with human preferences and needs, and unlock expertise in specific domains. After optimization training, the model better meets practical requirements and can be deployed.

Use & Processing · Company practice

The deployed model generates responses from user input

During inference, DeepSeek models compute over input and predict the next token. In its product services, this produces responses such as text, tables, and code. The disclosure also describes use in text-based tasks and downstream applications.

Supporting quotations

The inference phase is when the model is deployed to provide services. Once trained and deployed, the model can encode and compute input information to predict the next token, thereby enabling text generation, conversation, and other capabilities. It can proficiently perform a wide range of text-based tasks and be integrated into various downstream systems or applications. Specifically, for DeepSeek's product services, the model computes and infers based on user input to generate corresponding responses, including text, tables, and code.

Use & Processing · Company practice

DeepSeek describes probabilistic generation rather than retrieval of training copies

DeepSeek says its model predicts token sequences probabilistically from the input context, rather than simply retrieving or copying training text. It states that the model does not store copies of the original training data. This describes the model, not a promise that the company retains no training or user data elsewhere.

Supporting quotations

It is important to note that the model uses an autoregressive generation method, predicting the most likely subsequent sequence of tokens based on the input context through probabilistic calculations. This process is not a simple retrieval or "copy-paste" of text from the original training data. The model does not store copies of the original training data but dynamically generates contextually appropriate responses based on its deep understanding of language structure and semantic relationships.

Content & IP Rights · Company commitment

DeepSeek says it releases model components under MIT terms

DeepSeek commits to open-sourcing its models and says it publicly releases all model weights, parameters, and inference tool code under the permissive MIT License for download and deployment. It also publishes technical reports for each model. This disclosure does not supply the license text or independently establish the terms of a particular download.

Supporting quotations

DeepSeek is committed to open-sourcing its models. To this end, we publicly release all model weights, parameters, and inference tool code on open-source platforms under the permissive MIT License, allowing users to freely download and deploy them. Additionally, DeepSeek publishes comprehensive technical reports for each model, serving as references for the community and researchers and helping the public gain a deeper understanding of each model's technical principles and details.

Not stated here

  • The MIT License text and individual model release terms are not supplied here.

Security & Safeguards · Context

The disclosure distinguishes technology limitations from misuse

DeepSeek describes two sources of AI risk: limitations from immature technology and risks from how the technology is used. Its examples of misuse concerns include privacy, copyright, data and content safety, bias, and discrimination; it says these risks must be considered in the particular usage scenario and purpose.

Supporting quotations

Risks associated with AI models may arise from two causes:
Limitations due to the immaturity of AI technology.
Risks due to the misuse of AI technology.
The risks of AI technology misuse are widely recognized globally, including concerns about privacy protection, copyright, data security, content safety, bias, and discrimination. The technology itself is neutral; risks arise from its practical application and must be considered in the context of usage scenarios and intended purposes.

Use & Processing · Restriction or exception

DeepSeek cannot guarantee that model outputs are accurate

DeepSeek warns that AI may produce incorrect, omitted, or non-factual content, described as hallucination. It commits to reducing hallucinations through research, including higher-quality training sources, alignment strategies, and retrieval-augmented generation, but expressly cannot guarantee hallucination-free output. It says warning labels appear on the welcome page, at the end of generated text, and at the bottom of the interactive interface.

Supporting quotations

Currently, AI is still in its early stages, and the technology is not yet mature. Due to the limitations of current model principles, AI may generate incorrect, omitted, or non-factual content, a phenomenon known as "hallucination." Hallucination is a challenge faced by the entire AI industry. DeepSeek is committed to reducing hallucination rates through research, including but not limited to selecting high-quality training data sources, optimizing alignment strategies, and employing retrieval-augmented generation (RAG) techniques. However, at this stage, we cannot guarantee that the model will not produce hallucinations. To further mitigate the potential adverse effects caused by hallucinations, we have added prominent warning labels on DeepSeek's welcome page, at the end of generated text, and at the bottom of the interactive interface, specifically reminding users that the content is AI-generated and may be inaccurate.

Cross-references

Legal Rights & Agreement Rules · Restriction or exception

Outputs are for reference and should not be the basis for action or inaction

DeepSeek says generated content is for reference only and is not professional advice. For medical, legal, financial, or other professional inquiries, it says the service supplies no professional advice, commitment, or opinion. Users needing professional services should consult experts and decide under their guidance; software output should not be the basis for further action or inaction.

Supporting quotations

The content generated by the model is for reference only and should not be treated as professional advice. Specifically, when using this service for medical, legal, financial, or other professional inquiries, please note that the service does not constitute any advice or commitment or represent opinions in any professional field. If you require professional services, consult experts and make decisions under their guidance. The output of this software should not serve as the basis for further actions or inactions.

Security & Safeguards · Company commitment

DeepSeek describes safety measures throughout model development and deployment

DeepSeek states that it strictly complies with legal and regulatory requirements and takes reasonable measures to continually enhance safety throughout development, training, and deployment. Its non-exhaustive examples include internal risk management, model safety assessments, red-team testing, and improved transparency. These measures do not remove the disclosed risk of hallucinations.

Supporting quotations

DeepSeek takes the potential risks of AI technology applications very seriously. We strictly comply with legal and regulatory requirements and take reasonable measures to continuously enhance model safety throughout the entire lifecycle of model development, training, and deployment. These measures include, but are not limited to, establishing internal risk management systems, conducting model safety assessments, performing red team testing, and improving model and service transparency.
However, at this stage, we cannot guarantee that the model will not produce hallucinations.

Account & Service Control · User right or choice

Users can seek service information, opt out of training, and delete historical data

DeepSeek says it safeguards users’ legal rights to know, choose, and control model technology and services. It specifically lists querying basic service information, opting out of data use for model training, and deleting historical data. For requests or questions about these rights, it refers users to its Privacy Policy or privacy@deepseek.com. This paragraph does not specify the full procedures or promise deletion of every model effect derived from earlier data.

Supporting quotations

At the same time, we respect and safeguard the rights granted to users by law, including but not limited to the right to know, choose, and control model technology and services. Users can query basic service information, opt out of data usage for model training, delete their historical data, and more. If you have any claims, requests, or questions regarding the exercise of these rights, please refer to our [Privacy Policy] or contact us at [privacy@deepseek.com].

Not stated here

  • The separate Privacy Policy text and detailed request procedures are not supplied here.

Legal Rights & Agreement Rules · Company commitment

DeepSeek states that training-data acquisition and use comply with law

DeepSeek says it uses large, diverse data sources and strictly complies with laws concerning intellectual property, trade secrets, and personal privacy in acquiring and using data. This records the company’s representation, not independent verification of compliance.

Supporting quotations

The capabilities of DeepSeek models are built on high-quality, large-scale, and diverse data sources. We place great emphasis on and strictly comply with laws and regulations related to intellectual property, trade secrets, and personal privacy, ensuring that all data acquisition and usage occur within a legal and compliant framework. To enhance transparency, DeepSeek hereby publishes Training Data Summaries:

Use & Processing · Context

Separate training-data summaries are linked for named model versions

The disclosure links training-data summaries for DeepSeek-V3.1 and V3.2, V4, and V4.1. Those PDFs are not supplied here, so this card identifies the references without claiming what they contain.

Supporting quotations

DeepSeek-V3.1&V3.2 Training Data Summary
DeepSeek-V4 Training Data Summary
DeepSeek-V4.1 Training Data Summary

Not stated here

  • The linked training-data summaries’ text is not supplied here; their contents cannot be analyzed from these links.

Data Access & Collection · Company practice

Pre-training primarily uses public internet data and licensed datasets

DeepSeek says pre-training primarily uses two categories: freely accessible internet information, acquired and filtered with technical methods to build broad world knowledge; and proprietary datasets obtained from third-party providers through agreements based on lawful authorization. “Primarily” does not state these are the only possible sources.

Supporting quotations

During the pre-training phase, corpus data is required for training. This phase primarily uses the following two categories of data:
Public Data: We use publicly
available information on the internet to build the model's broad understanding of world knowledge. We employ technical methods to acquire and filter these freely accessible data to enrich the model's knowledge base.
Licensed Data: We collaborate with
third-party data providers to obtain proprietary datasets through legally signed agreements. We ensure all collaborations are based on lawful authorization.

Data Access & Collection · Company practice

Pre-training may incidentally include personal information despite screening

DeepSeek says pre-training does not require personal information, that it does not intentionally collect it to associate with a specific account or individual, and that it does not proactively use it for model training. It excludes sensitive information, credit card numbers, and unique identification information from training sources. However, public or licensed data may incidentally contain personal information; DeepSeek says it screens and removes it as much as possible and tests the data before training. This is not an assurance that every item is removed.

Supporting quotations

The pre-training phase does not require personal information for training. Therefore, we do not intentionally collect personal information to associate with any specific account or individual, nor do we proactively use it to train our models. We exclude sensitive information, credit card numbers, or unique identification information from our training data sources to minimize the risk of collecting any personal information. However, due to the vast scale of pre-training data, some publicly available online content or licensed data from other providers may incidentally contain personal information. We employ technical measures to screen and remove such information from the training data as much as possible and conduct tests before using the data for training.

Cross-references

Security & Safeguards · Company practice

Training-data filters and review address harmful content and bias

DeepSeek says its data governance process automatically filters raw data containing hate speech, pornography, violence, spam, or potential infringement. It also uses algorithmic and manual review to identify and mitigate statistical biases in large datasets. These are described controls and aims, not a guarantee that outputs will be free of harm or bias.

Supporting quotations

Additionally, to ensure data quality, safety, and diversity, we have established a rigorous data governance process. First, we use filters to automatically screen and remove raw data containing hate speech, pornography, violence, spam, or potential infringement. Second, recognizing that large-scale datasets may inherently contain statistical biases, we combine algorithmic and manual review methods to identify and mitigate the impact of these biases on the model's values, thereby enhancing fairness.

Use & Processing · Company practice

A small portion of fine-tuning data may be based on user input

DeepSeek says its research team produces question-answer data for optimization training, manually or automatically, with a small portion potentially based on user input. If it uses user input, it applies encryption, strict de-identification, and anonymization to prevent linkage to a specific individual. Users have a right to opt out of AI training, with instructions referred to the separate Privacy Policy.

Supporting quotations

During the optimization training phase, we typically need to construct or annotate a set of question-answer pair data manually or automatically to train the model. These question-answer pairs are produced by our research team, with a small portion potentially based on user input. If user input is used to construct training data, we apply secure encryption, strict de-identification, and anonymization to make it cannot be linked to any specific individual. We also deploy measures to make personal information not appear in the model's outputs to other users and we will not use it for user profiling or personalized recommendations. To be clear, we do not offer services involving user profiling or personalized recommendations. Users are also given the right to opt out. For information on how to opt out of AI training, please refer to the DeepSeek Privacy Policy.

Not stated here

  • The linked policy’s opt-out procedure is not supplied here.

Inference & Profiling · Company commitment

DeepSeek says it does not offer profiling or personalized recommendations

In its optimization-training disclosure, DeepSeek says it deploys measures to keep personal information out of outputs to other users and will not use that information for user profiling or personalized recommendations. It also states that it does not offer services involving profiling or personalized recommendations. This does not promise that its protection measures can never fail.

Supporting quotations

We also deploy measures to make personal information not appear in the model's outputs to other users and we will not use it for user profiling or personalized recommendations. To be clear, we do not offer services involving user profiling or personalized recommendations.

Use & Processing · User right or choice

Users can opt out of AI training

DeepSeek gives users the right to opt out of AI training and directs them to its separate Privacy Policy for instructions. That policy also supplies specific personal-information rules; the current document does not provide the full opt-out process.

Supporting quotations

Users are also given the right to opt out. For information on how to opt out of AI training, please refer to the DeepSeek Privacy Policy.
For specific rules on how we collect, protect, and use personal information, please carefully read the DeepSeek Privacy Policy.

Not stated here

  • Opt-out steps, timing, and any further conditions in the linked policy are not supplied.

Cross-references

Security & Safeguards · Company practice

Optimization training includes specialized safety data

DeepSeek says it constructs specialized safety data during optimization training to align the model with human values and enhance its inherent safety capabilities. This describes a safety measure, not a promise of risk-free operation.

Supporting quotations

To ensure model safety, during the optimization training phase, we construct specialized safety data to align the model with human values, enhancing its inherent safety capabilities.

Cross-references

  • contrast (model-proposed): DeepSeek cannot guarantee that model outputs are accurate — contrast: Specialized safety training aims to improve the model, while DeepSeek expressly acknowledges that hallucinations may still occur. The measures and remaining risk coexist; this is not a finding that the provisions contradict.