Chatgpt - Seriously good potential (or just some Internet fun)

[NickK]

Ok so V3 on GitHub they have:

The DeepSeek-V3 weight file consists of two main components: Main Model Weights and MTP Modules.

1. Main Model Weights​

• Composition:
  • Input/output embedding layers and a complete set of 61 Transformer hidden layers.
• Parameter Count:
  • Total parameters: 671B
  • Activation parameters: 36.7B (including 0.9B for Embedding and 0.9B for the output Head).

Structural Details​

• Embedding Layer:
  • model.embed_tokens.weight
• Transformer Hidden Layers:
  • model.layers.0 to model.layers.60, totaling num_hidden_layers layers.
• Output Layer:
  • model.norm.weight
  • lm_head.weight

Source: https://github.com/deepseek-ai/DeepSeek-V3/blob/main/README_WEIGHTS.md

So they supply the full network.

The R1 GitHub entry is just documentation and no code from what I can see.

 

[RxR]

Ok so V3 on GitHub they have:


Source: https://github.com/deepseek-ai/DeepSeek-V3/blob/main/README_WEIGHTS.md

So they supply the full network.

The R1 GitHub entry is just documentation and no code from what I can see.

I cant see the code download either. Yet it must be available somewhere - MS are offering it as a model on Azure.

 

[D.P.]

The model weights are also available. I expect the while model to be on AWS bedrock within a few months znd probably z guide to deploy it on Sagemaker within dzyd

yep,

now on AWS Bedrock

DeepSeek-R1 model now available in Amazon Bedrock Marketplace and Amazon SageMaker JumpStart | Amazon Web Services

Today, we are announcing that DeepSeek AI’s first-generation frontier model, DeepSeek-R1, is available through Amazon SageMaker JumpStart and Amazon Bedrock Marketplace to deploy for inference. You can now use DeepSeek-R1 to build, experiment, and responsibly scale your generative AI ideas on...

aws.amazon.com

will go and play

 

[RxR]

Im guessing any uni IT dept worth its salt will have downloaded a copy for staff / student use.

 

[NickK]

GitHub - huggingface/open-r1: Fully open reproduction of DeepSeek-R1

Fully open reproduction of DeepSeek-R1. Contribute to huggingface/open-r1 development by creating an account on GitHub.

github.com

The training commands below are configured for a node of 8 x H100s (80GB). For different hardware and topologies, you may need to tune the batch size and number of gradient accumulation steps.

So that's eight £20K rack mounts then..

Here {model} and {dataset} refer to the model and dataset IDs on the Hugging Face Hub, while {accelerator} refers to the choice of an Accelerate config file in configs.

So you'll need to get those from here: https://huggingface.co/docs/hub/en/datasets-overview you can then filter with 'R1' in the search box and you'll find a load of R1 based models (with no control or indication of quality or source).