Trang chủ / Tin tức / Khoa học & công nghệ
Khoa học & công nghệ

GoiBERT: Compact Vietnamese Language Models via Two-Stage Knowledge Distillation

21/10/2026 · Thành viên: TS. TẠ THỊ MINH NGỌC · Nguồn: OpenAlex

Vietnamese pre-trained encoders have improved rapidly, but the strongest ones are large, multilingual-vocabulary models or depend on external word segmentation, which makes them costly to deploy. W...

<p><strong>Tác giả:</strong> Quang Anh Kiet Tran, Nam Anh Tran, Trinh Manh Hung, Thanh Danh Le, Khang Nhat Hoang Vo, Huynh Dai Nhan Tran</p>

<p>Vietnamese pre-trained encoders have improved rapidly, but the strongest ones are large, multilingual-vocabulary models or depend on external word segmentation, which makes them costly to deploy. We present GoiBERT, a family of two compact Vietnamese encoders with 26M and 49M parameters that operate directly on raw text. GoiBERT learns from two complementary teachers. A same-tokenizer Vietnamese encoder first teaches general-purpose representations. A much larger teacher then provides task-le...</p>

<p>📄 DOI: <a href="https://doi.org/https://doi.org/10.48550/arxiv.2610.30067" target="_blank" rel="noopener">https://doi.org/10.48550/arxiv.2610.30067</a></p>

<p>📖 <a href="https://doi.org/https://doi.org/10.48550/arxiv.2610.30067" target="_blank" rel="noopener">Xem bài báo đầy đủ</a></p>

Bài viết được biên soạn lại bởi LIFVietnam. Nguồn tham khảo: OpenAlex