Arcee AI Research
Check out the latest publications by the Arcee AI team.
Arcee AI Mergekit: A toolkit for merging Large Language Models
Learn more about MergeKit, an open-source library enabling efficient model merging to create multitask models without additional training, preserving original capabilities while enhancing AI performance and versatility.
Domain Adaptation of Llama3-70B-Instruct through Continual Pre-Training and Model Merging: A Comprehensive Evaluation
This is a preprint technical report with thorough evaluations to understand the entire process of domain adaptation, continual pre-training, and model merging for enhancing language model performance on financial regulatory data.
Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation
This paper examines model merging techniques, from simple averaging methods like Model Soups to advanced approaches like DARE and TIES-Merging, and introduces Differentiable Adaptive Merging (DAM) for efficient model integration.