Model Merging LLM: Mergekit SLERP vs a Third Fine-Tune

You have two fine-tunes of the same base model, each good at something the other isn't, and a decision to make: train a third model on the combined data, or merge the two checkpoints you already have. Model merging LLM tooling like mergekit turns the second option into a five-minute YAML edit and a CPU job. The catch is that "cheap" only matters if the merged model actually holds up on eval, so this post gives you the config, a way to read the eval table before you ship, and the GPU-hour math that makes the decision defensible rather than a guess.

TL;DR: Should You Merge Two Fine-Tunes or Train a Third?

Merge two fine-tunes from the same base checkpoint with mergekit's SLERP method instead of training a third model; only retrain if the merge fails your eval bar against both parents.

• Cost: mergekit runs on CPU or a GPU with as little as 8GB VRAM, near-free.
• Retrain cost: Spheron H100 on-demand is $2.39/hr as of 08 Sep 2026, GPU-hours a working merge skips.
• Track record: merged models are 20% of the Hugging Face Open LLM Leaderboard's top 50 and 34% of its top 100, per Arcee's mergekit paper.

This post covers only the two-fine-tune SLERP case, and the retrain fallback it compares against runs on H100 GPU pricing like any other SFT job.

Why Model Merging LLM Beats a Third Fine-Tune

The scenario this post is written for is specific: you already ran SFT twice. Maybe one checkpoint learned your support tool-calling format and the other learned your product's tone of voice from a separate dataset. Both jobs are done, both cost real GPU-hours, and now you want one model that does both. The instinct is to combine the two datasets and run SFT a third time. Model merging LLM tooling exists because that third run is usually unnecessary.

This only works under one condition worth stating plainly: both fine-tunes have to share the same base architecture and tokenizer. A Llama 3.1 8B tool-use fine-tune merges cleanly with a Llama 3.1 8B tone fine-tune. It does not merge with a Qwen 3 checkpoint, because the weight tensor shapes don't match. If your two candidate models come from different base families, merging is off the table and the third fine-tune (or LoRA adapters served side by side instead of collapsed into one checkpoint) is the real choice.

SLERP vs TIES vs DARE: What Each Actually Does to the Weight Deltas

These three methods solve different problems, and picking the wrong one for your model count is the most common way a merge underperforms. Here's what each does mechanically, not just what it's for.

SLERP: Interpolating on the Sphere, Not the Line

Plain linear averaging (W_merged = alpha * W_A + (1 - alpha) * W_B) treats weight vectors like points on a straight line. The problem is that in high-dimensional space, that straight line cuts through lower-magnitude territory than either endpoint. As Maxime Labonne wrote in his walkthrough of merging large language models with mergekit, "In high-dimensional spaces, linear interpolation can lead to a decrease in the magnitude of the interpolated vector." In practice, that shows up as a merged model that feels muted or loses sharpness relative to either parent.

SLERP (spherical linear interpolation) fixes this by moving along the arc of the unit hypersphere between the two weight vectors instead of the chord connecting them. This preserves vector norms through the interpolation, which is why it's the standard default for exactly two models that share a base. Because SLERP is defined for two vectors at a time, it caps out at two source models; past that, you're back to TIES or task arithmetic.

TIES: Trim, Elect Sign, Merge

TIES runs three steps in order: trim each model's delta down to its highest-magnitude parameters, elect a single sign per parameter by majority vote across models, then merge only the parameters that agree with the elected sign. It's built for three or more models, which is where sign conflicts get common enough to matter; for exactly two fine-tunes, SLERP is usually the simpler and equally effective choice.

DARE: Pruning a Mostly Redundant Delta

DARE is typically applied as a pre-processing step before TIES (the combination is often written dare_ties in configs), pruning each model's delta down to its load-bearing parameters before the sign-election step runs. It's most useful when one of your two fine-tunes was trained aggressively and you suspect a lot of its delta is noise rather than signal.

A Cloneable Model Merging LLM Config and the GPU-Hours It Replaces

The config.yaml (SLERP, Two Fine-Tunes, One Base)

This is the config for the two-fine-tune case described above: one base model, two SFT checkpoints from that base, merged with SLERP at an even 0.5 blend. Save it as config.yaml.

slices:
  - sources:
      - model: your-org/llama-3.1-8b-tool-use-sft
        layer_range: [0, 32]
      - model: your-org/llama-3.1-8b-tone-sft
        layer_range: [0, 32]
merge_method: slerp
base_model: your-org/llama-3.1-8b-tool-use-sft
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16

The t parameter is where the actual tuning happens. A flat value: 0.5 blends every layer evenly. The filter-based entries above are optional and let you weight attention layers and MLP layers differently across the depth of the network, which is the knob to turn if an even blend under- or over-weights one parent's behavior. Run it with:

pip install mergekit
mergekit-yaml config.yaml ./merged-model --cuda --lazy-unpickle

Drop --cuda if you're merging on CPU only. The --lazy-unpickle flag is what keeps peak memory manageable: it streams tensors layer by layer instead of loading both full checkpoints into memory at once, which is also why mergekit can merge on hardware nowhere near large enough to hold both source models simultaneously.

Eval Table: Parent A, Parent B, the Merge, and a Real Second Fine-Tune

Before you trust a merged checkpoint, you need it sitting in a table next to both parents and, ideally, a real retrained baseline. The shape of that table matters more than any single number in it. Run the same benchmark suite (MMLU or GSM8K through lm-evaluation-harness for general regressions, plus your own domain eval) across all four checkpoints and read the table like this:

This is a template to fill in with your own harness output, not a set of fixed numbers, because every base model and dataset pair lands differently. What you're checking for is a specific pattern: a healthy merge should land close to Parent A on Task A and close to Parent B on Task B, without falling meaningfully below either parent on the general eval. If the merge sits well below both parents on both task scores, that's catastrophic interference, and the fix is either a lower merge coefficient toward the stronger parent, a switch to TIES, or in the worst case, retraining. Only the retrained baseline tells you the ceiling: how much of Task A and Task B performance is actually recoverable in one checkpoint at all, since even a from-scratch combined SFT run can trade some of one skill for the other.

For a harder eval than benchmark scores, especially on open-ended tone or style tasks where there's no single correct answer, an LLM-as-judge pipeline that compares the merge's outputs against both parents pairwise catches regressions that a multiple-choice benchmark misses.

GPU-Hours and Dollar Cost: Merge vs Re-Running SFT

Here's the actual arithmetic, using Spheron's on-demand rates as of 08 Sep 2026: H100 SXM5 at $2.39/hr, A100 at $1.43/hr, and RTX 4090 at $0.72/hr, with H100 spot at $2.05/hr for workloads that can tolerate interruption. Spheron aggregates these rates across 5+ providers on its network, and prices move with availability, so treat the numbers below as the shape of the comparison rather than a fixed quote.

The merge step itself is close to free regardless of which GPU tier you'd otherwise reach for. Mergekit needs only 8 GB of VRAM per its own docs. Even if you run the merge on a rented RTX 4090 out of convenience, at $0.72/hr a 10-to-20-minute job costs a small fraction of that hourly rate (worked example, based on the rate as of 08 Sep 2026). That's the entire GPU-hour cost of testing whether merging works.

The comparison that actually matters is what you'd spend if the merge doesn't hold up and you fall back to training a third model on the combined dataset. QLoRA SFT runs take hours, not minutes, and scale up with model size. At the current on-demand H100 rate of $2.39/hr, every added hour of training time is that same rate you weren't going to spend if the merge had worked, and a multi-hour SFT job adds up to real dollars before you count the time spent preparing the combined dataset and iterating on hyperparameters. Spot pricing at $2.05/hr is available for workloads that can tolerate interruption, but a live SFT job usually can't take a mid-run preemption gracefully the way a batch eval sweep can.

Put those two numbers side by side and the case for trying the merge first is arithmetic, not opinion: a merge attempt costs cents to low single-digit dollars and minutes to an hour of wall-clock time. A full retraining fallback costs tens of dollars in GPU time alone and hours you can't parallelize away. The eval sweep across four checkpoints (base, both parents, the merge) is the only real expense in the "try merging first" path, and it's the same eval work you'd need to validate a retrained model anyway. If you do go the H100 route for a comparison run or eval sweep, per-minute billing with no minimum means a 45-minute merge validation job or a multi-hour eval sweep only costs for the time actually used, not a rounded-up block.

Pricing fluctuates based on GPU availability. Spheron rates above are live as of 08 Sep 2026. Check current GPU pricing → for live rates.

Where Merging Breaks: Eval Regressions to Check Before Shipping

Catastrophic Interference and How to Catch It Before Deploy

Catastrophic interference is what happens when the two fine-tuning directions partially cancel instead of combining. The TIES-Merging paper traces it to two mechanisms: interference from redundant, low-magnitude parameters that add noise without adding signal, and outright sign disagreement, where Parent A moved a parameter up and Parent B moved the same parameter down, so a naive average lands somewhere that serves neither direction. The tell in your eval table is a merge that scores below both parents on both task-specific evals at once, not just below one.

Three checks catch this before it reaches production. First, compare the merge against both parents on each parent's own strong task, not just an aggregate score; averaging can hide a real regression in one task behind an unrelated gain in another. Second, spot-check actual generations, not just benchmark numbers, since a model can hold its MMLU score while producing outputs that read as neither parent's voice, which matters a lot if Task B was specifically about tone. Third, if the general eval (MMLU-style) drops noticeably below both parents rather than sitting between them, that's a sign the merge coefficient is too aggressive for how far apart the two fine-tunes actually are in weight space, and dropping t toward the stronger parent (or moving to TIES, which is more conservative about which parameters get merged at all) is the first thing to try before abandoning the merge entirely.

When the Answer Is Actually "Train the Third One"

Merging is not always the right call, and the eval table is what tells you that, not intuition. If the SLERP merge underperforms both parents on their respective strong tasks after you've tried adjusting the coefficient and tried TIES as an alternative, that's catastrophic interference that isn't going away with more knob-turning, and the honest engineering call is to retrain. This is also the right call when the two fine-tunes are further apart than "same base, different skill": if one parent was trained on a dataset large enough to meaningfully shift the model's general behavior, not just add a narrow skill, the loss landscapes may no longer be close enough for interpolation to work cleanly.

If retraining is the answer, the fine-tuning cost and GPU guide covers the SFT workflow end to end, and the H100 vs A100 cost guide has the GPU-choice math and VRAM sizing for the training run itself. And if the real goal was never a single checkpoint but keeping both skills independently addressable, serving both fine-tunes as separate LoRA adapters on one base model, the way the multi-adapter guide linked earlier describes, sidesteps the merge-vs-retrain question entirely.

One more data point worth keeping in mind before writing merging off as a hack: merged models represented 20% of the top 50 and 34% of the top 100 entries on the Hugging Face Open LLM Leaderboard, per Arcee AI's own mergekit paper. As the paper's authors put it, "MergeKit can scale from a high-end research cluster all the way down to a personal laptop with no GPU and limited RAM." Try the merge first. It costs you an eval sweep either way.

Whichever path the eval table points to, the GPU-hours behind it come from the same place. Spot-check merge candidates on a Spheron RTX 4090 or scale a retrain up to H100s, all billed per minute with no minimum commitment.

Get started on Spheron →