Fine-tuning a separate model for every task means storing and serving many models, and fine-tuning can erase general capabilities. Model merging avoids both problems. It combines the parameters of existing checkpoints into one model, with no extra training.

MergeKit is Arcee’s open-source library for doing this. It has powered thousands of community merges, including some of the strongest open-source checkpoints available.

What it does

• Supports the major merge methods: linear averaging, SLERP, Task Arithmetic, TIES, DARE, and Model Breadcrumbs. It also adds passthrough “Frankenmerging” for layer stacking (used for Goliath-120B and SOLAR-10.7B’s depth up-scaling) and a Mixture-of-Experts builder that assembles an MoE from dense models.
• Runs on any hardware: an out-of-core, graph-scheduled design loads only the tensors each step needs. Merges run on anything from a research cluster to a laptop with no GPU.
• Requires no code: merge recipes are simple YAML files that are easy to share and remix. A hosted GUI on Hugging Face runs merges in a few clicks.

Proof points

Arcee merged Meditron-7B, a medical model, with Llama2-7B-Chat. Every merged variant beat Meditron-7B on all three medical benchmarks. The SLERP merge, for example, raised MedMCQA from 24.07 to 36.91 and PubMedQA from 71.40 to 75.60. The LERP and SLERP merges also beat Llama2-7B-Chat on general benchmarks: MMLU rose from 46.37 to 48.44 with LERP. The result was one model that is stronger at both specialist and general tasks.

More information

Read the paper: https://arxiv.org/abs/2403.13257
Get the code: https://github.com/arcee-ai/mergekit