TCD_IWSLT25_ModelCompression_en-de_Bin0_unconstrained_contrastive

This model is a pruned version of Qwen2-Audio-7B-Instruct. It was created in multiple stages, (a) full fine-tuning of the baseline model with the ACL 60/60 dataset, (b) layer pruning of the decoder into 24 layers, while the encoder 32 layers were kept intact, (c) full fine-tuning of the pruned model, and (d) QLoRA fine-tuning with the ACL 60/60 dataset augmented with data knowledge distillation from the fully fine-tuned model, as well as a portion of the CoVoST2 dataset. This process has achieved almost 50% compression, as the resulted model has only 4.12B parameters and consumes approx. 8.7 GB of storage.  It is worth noting that in order to fully restore quality, a larger volume of relevant training data is needed.

Language Pair: English-German

From\To de zh
en 0.693