Search results

SMP2020-EWECT的数据集开源使用问题

Status: Open.

#133 In Tencent/TencentPretrain;

· pxaklbe opened on Oct 31, 2024

请问这里的中文模型支持的最大输入序列长度是512tokens吗？超过512tokens就会被截断嘛？可不可以在微调的时候扩大模型的位置编码数量？

Status: Open.

#130 In Tencent/TencentPretrain;

· chengzi-big opened on Jul 21, 2024

单机2卡预训练LLAMA-7B报错TypeError: an integer is required (got type NoneType)

Status: Open.

#112 In Tencent/TencentPretrain;

· smallYellowCat opened on Nov 29, 2023

【问题】deepspeed如何对不同显存大小分配数据，我有32G和16G两种大小的GPU

Status: Open.

#98 In Tencent/TencentPretrain;

· 18liumin opened on Sep 26, 2023

LLaMA2-70B格式转换

Status: Open.

#96 In Tencent/TencentPretrain;

· Double-bear opened on Sep 5, 2023

size mismatch for classifier.weight: copying a param with shape torch.Size([7, 768]) from checkpoint, the shape in current model is torch.Size([2, 768]).

Status: Open.

#93 In Tencent/TencentPretrain;

· byte4tide opened on Aug 19, 2023

KeyError: 'd'

Status: Open.

#92 In Tencent/TencentPretrain;

· byte4tide opened on Aug 19, 2023

Resume from checkpoint

Status: Open.

#81 In Tencent/TencentPretrain;

· mohammadaminabbasi opened on Aug 15, 2023

发现几个bug，dynamic_masking 意思是不是写反了，多了not

Status: Open.

#80 In Tencent/TencentPretrain;

· boluoyu opened on Aug 10, 2023

lora推理

Status: Open.

#71 In Tencent/TencentPretrain;

· aSmallsheep opened on Jun 6, 2023

DeepSpeedZeRoOffload initialization failed (can't allocate memory)

Status: Open.

#70 In Tencent/TencentPretrain;

· treya-lin opened on May 25, 2023

关于扩充词表再增量预训练的疑问

Status: Open.

#68 In Tencent/TencentPretrain;

· qiancheng99 opened on May 19, 2023