Community
MergeKit & Arcee.AI:
Our Commitment to Open Source
.png)
About MergeKit
MergeKit is an open source toolkit designed by software engineer Charles Goddard for the blending of pre-trained language models, allowing for the execution of intricate merges even within resource-constrained environments. Notably, Mergekit enables merges to be executed solely on the CPU, or – if desired – to be accelerated with as little as 8 GB of VRAM.
Arcee.AI and Mergekit joined forces with a two-fold commitment:
• To advance the research and development of model merging, which we believe to be a revolutionary technique in LLM development – broadening access to the most performant LLMs
• To keep MergeKit open source.
You can read more about the Arcee.AI + MergeKit merger here.
MergeKit Spaces on Hugging Face
The MergeKit GUI on Hugging Face is the easiest way to get started with model merging.
Just specify a YAML configuration, and a Hugging Face token and the app will perform the merge then upload the merged model to your Hugging Face user profile.
We also have a Hugging Face space that allows you to easily generate the YAML configuration for your merges.
.png)
.png)
Arcee AI Research
Check out our latest publications:
Arcee’s MergeKit: a Toolkit for Leveraging Large Language Models, on Arxiv.
Domain Adaptation of Llama3-70B-Instruct through Continual Pre-Training and Model Merging: A Comprehensive Evaluation
Case Study: How Arcee is Innovating Domain Adaptation, through Continual Pre-Training and Model Merging (download here).
Arcee AI Partners
We’re proud to partner with the following companies:
