--- title: "How to Compare Two Simulations" description: "Compare two simulations side by side to understand how their results differ." ---
TL;DR
Run two simulations, open one as the base, click Compare, and pick the second. Every tab (Job Details, Job Result, each Report) renders side by side with synced scrolling. Use it for champion vs. challenger, version vs. version, or sample vs. sample.
## What is it? Run two regular simulations and compare them side by side from one job's results page. Common workflows this supports: * **Champion vs. challenger**: candidate model against the model in production. * **Version vs. version**: a refactored policy against its previous version. * **Sample vs. sample**: the same model on Train vs. Test, or on this quarter vs. last quarter. ## Before you start * Two completed simulations exist. Both must have status **Completed** (or **Partially Completed**, though the two sides may then cover different records). * You have read access to both simulations. * For an apples-to-apples comparison, both simulations should run on the same population. The simplest way is **Prespecified Ids** (see [Run a Simulation](/user-guide/how-tos/run-a-simulation/)). ## How to Compare You compare two completed simulations (see [Before you start](#before-you-start)). The one you open first is the **base**, on the left; the second is the **compared** job, on the right.
#### Open the comparison view 1. Open the simulation you want as the base (the left side) from the **Jobs** tab. 2. Click **Compare** at the top of the page. 3. The job picker opens. By default it lists completed jobs from the **same object and same version**. To pick from a different object, version, or job of the same type, click the switch arrow () next to the picker. 4. Select the simulation to compare. The page reloads in side-by-side mode.
#### Work in the comparison view The base job is on the left; the compared job is on the right. * Every tab (**Job Details**, **Job Result**, and each **Report** tab) renders both jobs side by side. * Scrolling is synced: scrolling one pane scrolls the other, so equivalent sections stay aligned. * From the comparison header you can: * **Change the compared job** on the right by picking a different object, version, or job. The base job on the left is fixed - to compare against a different base, exit and reopen from that job. * **Swap** the two sides with the swap icon () so the compared job becomes the base. * **Exit the comparison** from the kebab menu (⋮) at the top right to return to the single-job view.
## Reading the Comparison The comparison view lines up each job's existing outputs so you can read them together, side by side. * **Job Details** side by side: shows how the two jobs were configured. Useful for spotting that the difference you're seeing is really a sample/filter/version difference rather than a real model difference. * **Job Result** side by side: row-level outputs for each. Download either pane as CSV/Parquet from its own toolbar. * **Report tabs** side by side: the same report renders for each job, so you can compare the AUC, KS, decile table, confusion matrix (for Models) or approval rate / segment distribution (for Policies) directly. A side-by-side shows the two jobs next to each other but doesn't compute anything across them. For true diff metrics, PSI, CSI, a swapset, build reports that read both jobs at once. See **Enable custom comparison** in [Register Reports](/user-guide/how-tos/register-reports/). Once set up, those comparison reports populate automatically when you open the view. ## Champion vs. challenger Testing a candidate against what you run today is the most common reason to compare two simulations. The champion is the model or policy currently in production; the challenger is the one you are considering moving to. To set it up: 1. **Build the challenger** as a real object, either a new version of the existing one or a separate object. 2. **Run the champion** on your chosen population as a normal [Simulation](/user-guide/how-tos/run-a-simulation/). 3. **Run the challenger** on the same population, using **Prespecified Ids** so both jobs score exactly the same records. 4. **Compare them**, opening the champion first so it sits on the left. Which numbers matter depends on what you are testing. For **Models**, AUC, KS, and the decile table show whether the challenger ranks risk better. For **Policies**, approval rate, auto-decision rate, and segment distribution show what it would do to volume. Read them together rather than picking one. A challenger that approves more will usually also approve more bad accounts, so the question is whether the extra volume is worth the extra risk. If the challenger wins, promote it the way you would any other change: send that version through [approval](/user-guide/how-tos/approve-an-object/). Comparing two jobs does not change either object. :::note[Not built the challenger yet?] If the challenger is a Policy rule change you have not built yet, run a [What-If Analysis](/user-guide/how-tos/run-a-what-if-analysis/) instead. You edit the rules on a copy of the Policy, and the result comes back paired against your baseline automatically. If it works, you can write the changes into a real Policy from the job. ::: ## What's next * [Register Reports](/user-guide/how-tos/register-reports/) with **Enable custom comparison** turned on for figures that need to read both jobs at once. * [Run a What-If Analysis](/user-guide/how-tos/run-a-what-if-analysis/) to test a Policy rule change without building the challenger first. * [Approve an Object](/user-guide/how-tos/approve-an-object/) to promote a challenger that wins.