Why We Make GPT and Claude Argue: Multi-Model Debate in Underwriting

A language model will answer any question you ask it with the same even tone whether it is right or wrong. That is fine when the question is how to phrase an email. It is a problem when the question is whether a roofing contractor with a climbing mod and three open claims belongs in a PEO's loss fund.

Underwriting is a discipline of second opinions. A carrier underwriter reviews a submission and a senior underwriter reviews the review. A PEO's pricing committee argues about a loss pick before it goes out. The whole structure exists because one confident person, or one confident model, is not a reliable basis for a decision that will be paid for over the next five years.

So when we built AI into the underwriting layer of HALO IQ, we did not build it around one model. IQ Agent runs two frontier models, currently from OpenAI and Anthropic, on the same deal context at the same time, scores what comes back, and merges the result. This post explains why, how it works, and where it helps and does not.

The problem with one confident answer

Frontier models are good at the kind of reasoning underwriting requires: reading a loss history, noticing that the frequency is concentrated in one location, connecting that to the class mix, and forming a view. They are also prone to a specific failure that experienced underwriters will recognize from junior colleagues. They anchor on the first plausible story and build a confident case for it, and they do not reliably signal when they have done so.

Two models trained by different organizations on different data with different methods do not share the same blind spots. When they look at the same submission and reach the same conclusion by different routes, that agreement is informative. When they disagree, the disagreement is more informative still, because it points to exactly the part of the analysis where a human should look.

That is the core insight. We are not using two models because two is better than one in some vague sense. We are using two because disagreement between independent reasoners is the cheapest signal available for "this needs a person."

How IQ Agent runs a debate

The process has four stages.

First, context is gathered. For an underwriting question about a deal, that means the parsed loss runs aggregated by policy year, the mod projection and the worksheet if it exists, the class codes and payroll, the carrier rate information, the pricing inputs, any compliance or safety data attached to the company, and the activity history on the deal. All of this is structured data produced by the other modules in HALO IQ, which is the reason the debate has something substantive to argue about.

Second, a prompt is built from that context. The prompt is the same for both models and is structured so that the models are asked the same specific questions in the same order: what are the primary risk drivers, what is the quality of the loss data, what loads should be considered on the pick, what would change the recommendation, and what is the recommendation.

Third, both models run in parallel and return their answers.

Fourth, the answers are scored and merged. Scoring looks at whether each model grounded its claims in the context it was given, whether it addressed each question, whether it stated uncertainty where the data warranted it, and whether its reasoning is internally consistent. Where the models agree, the merged answer carries the point with the stronger supporting evidence. Where they disagree, the merged answer presents the disagreement explicitly rather than picking a winner silently.

The output is a structured response an underwriter reads: the risk drivers, the data quality assessment, the suggested loads with rationale, the points of agreement, the points of disagreement, and the recommendation with its confidence.

A labeled hypothetical

Take a hypothetical submission for a 90-employee commercial landscaping company with payroll split across two class codes, three years of loss runs from two carriers, and a projected mod rising from the low 1.1s to the mid 1.2s.

One model reads the loss history and focuses on frequency: eleven claims under the primary threshold over three years, concentrated in the most recent two, suggesting a deteriorating safety culture or a new crew. It recommends a frequency-driven load on the pick and flags the account as marginal.

The other model focuses on the carrier change between year one and year two. It notices that the two carriers reserve differently, that the open reserves from the second carrier are large relative to paid, and that the apparent increase in frequency coincides with the switch. It recommends treating the reserve-driven incurred with caution and suggests that the frequency trend may be partly a reporting artifact.

Both readings are reasonable. Neither is obviously right. A single model would have given one of them confidently. The merged response gives both, notes that they disagree on whether the frequency trend is real, and tells the underwriter that the question to resolve is the reserving philosophy of the second carrier. That is a question a person can answer in a phone call, and it is the right question.

This is a constructed example. It is representative of the kind of disagreement we see, not a record of a specific account.

Where it helps

Multi-model debate earns its cost in three situations.

The first is the marginal account. Clean accounts and obviously bad accounts do not need a debate; the data speaks. The accounts in the middle, where the recommendation depends on how you weigh conflicting signals, are where two independent reads surface the weighing itself.

The second is the data quality question. Models are reasonably good at noticing that something in a submission does not add up, and two models are better than one at noticing different things. A payroll figure that does not reconcile between the ACORD and the census, a claim that appears on one run and not another, a class code that does not fit the described operations: one model may catch it and the other may not, and the merge catches both.

The third is the explanation. Underwriters have to write up their reasoning for a pricing committee, a carrier, or a broker. A merged response that lays out the drivers, the disagreements, and the recommendation is a draft of that write-up, and it is a better draft for having been argued over.

Where it does not help

A debate between two models does not make the loss pick. The pick is a calculation: development factors, Bornhuetter-Ferguson, credibility blending, loads. Those run deterministically in ModIQ, and we would not let a language model compute them. IQ Agent can suggest that a concentration load is warranted and explain why; it does not pick the number.

A debate does not fix bad data. If the loss runs are incomplete, two models will reason confidently about an incomplete picture. The parsing and aggregation have to be right first, and no amount of model sophistication substitutes for a missing policy year.

A debate does not replace the underwriter. The merged response is an input to a decision, not the decision. We have been deliberate about the output format for this reason: it is written to be read and challenged, not to be approved with a click. An underwriter who disagrees with both models should say so, and the system should make that easy.

And a debate is not free. Running two models in parallel costs twice as much as running one. We think that is the right trade for an underwriting decision and the wrong trade for drafting a follow-up email, which is why IQ Agent is used for the former and not the latter.

Why this matters beyond the feature

The broader point is about how AI should enter a risk discipline. The temptation is to put a single model in front of an underwriter as an assistant and let it answer questions. That produces an assistant that is confidently wrong at an unknown rate, and an underwriter who either over-trusts it or ignores it.

The alternative is to build the AI the way the discipline already works: multiple independent opinions, scored against evidence, with disagreement surfaced rather than hidden, and a person making the call. That is more work to build and more expensive to run. It is also the only version we would trust with our own PEO's loss fund.

IQ Agent is that version. It is one of the places where HALO IQ being built for AI from the start, rather than having AI added later, shows most clearly: the structured deal context that makes the debate meaningful exists because every other module produces it, and the debate's output flows back into the proposal narrative and the deal record without anyone copying it anywhere.