Arpad Elo and Georg Rasch published within a few years of each other and never worked together, but they wrote down one formula. Elo used base 10 and a 400-point spread because that suited chess bulletins. Rasch used base e. Nothing else about them differs.
| Chess | LoB Latent Maturity Model |
|---|---|
| Player | Line of Business |
| Time control (blitz, rapid, classical) | Dimension (Runtime, Openness, Data) |
| One rating per player per time control | One θ per LoB per dimension |
| Opponent | An assessment check, Q1 through Q8 |
| Opponent's rating | Item difficulty b |
| A game | One product assessed against one check |
| Win or loss | Pass or fail |
| Tournament performance rating | θ estimated from this cycle |
| Rating deviation (Glicko) | Standard error on θ |
| Anchoring against rating inflation | Anchor items across assessment cycles |
A blitz game does not move your classical rating. In the same way, a Security check tells you nothing about Reliability maturity, so each dimension is estimated from its own checks and the dimensions cannot be merged without throwing information away. FIDE publishes three ratings and no combined one. Merging formats means deciding how much each format counts, which is a judgment call rather than a measurement, and that is also why the assessment proposal declines to compute a single overall LoB star.
The data here is fictional, following the assessment proposal's own convention. Both LoBs were assessed on the Openness dimension, and neither faced the same check. Beta has no customer-facing agents, so the extensibility check was N/A. Gamma publishes no MCP server, so that check was N/A for them.
| LoB | Opponent (the check) | its rating | played | won | score |
|---|---|---|---|---|---|
| Beta | Q8 MCP server published | 1861 | 11 | 2 | 18% |
| Gamma | Q5 Customer extensibility | 1474 | 12 | 5 | 42% |
Raw coverage compares 42% to 18% and stops. Gamma wins. But those are scores from two different tests, being read as though they came from the same test.
That is all θ is. A performance rating: the strength your results imply, given who you played. Chess players work one out after every tournament.
A rating can feel like a formula defending itself, so here is the question raw coverage has no way to answer. What would each of them score against the same check?
| LoB | vs MCP publication (1861) | vs extensibility (1474) |
|---|---|---|
| Beta (1600) | 18% | 67% |
| Gamma (1415) | 7% | 42% |
Beta is ahead on both. Pick either check, put them both on it, and Beta scores higher. There is no opponent anywhere on the board where Gamma comes out in front. So raw coverage had the ranking backwards. Its arithmetic was fine. It just had no way to notice that the two LoBs took different tests.
Two stages, both fed by a single table of observed pass and fail. No self-assessment goes in, and nobody scores the difficulty by hand.
Rows are the units assessed, columns are the eight questions, cells are observed pass or fail. Out comes one difficulty per question, shared across every LoB. That is a difficulty vector, not a matrix, and the sharing is the whole point. A common scale is what makes maturity comparable between LoBs.
Difficulty and maturity are estimated together rather than in sequence. The fit alternates: given the current maturities, update every difficulty; given the current difficulties, update every maturity; repeat until it settles. The origin of the scale is arbitrary, so convention pins it at mean difficulty of zero.
These ratings belong to the checks. The middle column only says which dimension a check sits in; it is not a rating of the dimension. LoB ratings are in the next table.
| Check | sits in dimension | passes | b (logits) | Elo |
|---|---|---|---|---|
| Q8 MCP server published | Openness | 3/25 | +2.08 | 1861 |
| Q7 Contributes to Knowledge Graph | Data and Semantics | 7/37 | +1.49 | 1759 |
| Q2 Processes handled by agents | Experience and Process | 14/37 | +0.40 | 1569 |
| Q4 Agent Memory Service used | Runtime | 18/37 | −0.13 | 1477 |
| Q5 Customer extensibility | Openness | 13/26 | −0.15 | 1474 |
| Q3 Agents on App Foundation | Runtime | 21/37 | −0.52 | 1410 |
| Q6 Data products / eFOS | Data and Semantics | 21/37 | −0.52 | 1410 |
| Q1 Joule integrated | Experience and Process | 33/37 | −2.66 | 1038 |
Hardest at the top. Publishing an MCP server is a grandmaster. Joule integration is a beginner almost everyone beats.
Same scale, other side of the board. A LoB plays one game per product per applicable check, and its rating is whatever best explains the wins and losses. One rating per LoB per dimension, the way a chess player has separate blitz and classical ratings.
| Dimension | Alpha | Beta | Gamma |
|---|---|---|---|
| Experience and Process | 1429 | 1411 | 1547 |
| Runtime | 1495 | 1476 | 1415 |
| Openness | 1497 | 1599 | 1415 |
| Data and Semantics | 1490 | 1420 | 1510 |
1500 is average. Same scale as the checks above, so you can read the two tables against each other: Beta at 1599 in Openness sits below the MCP check at 1861 and above the extensibility check at 1474.
Try a candidate rating. The model says what pass rate that rating predicts for each check in the dimension. Compare those predictions against what the LoB actually scored. The rating that predicts the observed results most closely is the answer.
Concretely, each check contributes the probability of exactly what was seen, so a check where 8 of 14 products passed contributes the chance of getting 8 passes and 6 failures at that candidate rating. Multiply across the checks in the dimension and you have one number saying how well that candidate explains the whole pattern. Then pick the candidate with the highest number.
Alpha, Runtime dimension. This produces the 1495 in table 2 above.
| candidate θ | −1.0 | −0.5 | 0.0 | +0.5 | +1.0 |
|---|---|---|---|---|---|
| log-likelihood | −22.47 | −20.01 | −19.25 | −20.17 | −22.56 |
The best of the five candidates is θ = 0. Refined on a fine grid it lands at θRuntime(Alpha) = −0.03, which is 1495 Elo, the Runtime figure for Alpha in table 2. The easy check pulls the estimate down when products miss it, and the harder check pulls it up when they pass it. Maturity settles where the two pulls balance.
This single rule is where the correction lives. Maturity gets estimated using only the checks that genuinely applied to that LoB. It then gets reported as the coverage that maturity predicts across the full item set, including the checks that were N/A. Think of a student who skipped a question. You work out their level from the questions they did answer, then predict the one they skipped.
When every check applies, this changes nothing at all. Fitting and reporting on the same items returns the observed percentage exactly, by construction. The method stays quiet wherever raw coverage was already fair and speaks up only where the exams differed. If it moved every number, that would be a reason to distrust it.
The model contains a single quantity, θ − b, so maturity and difficulty sit on one scale. A LoB therefore has a position among the checks, not just a score. Being level with a check means an even chance of passing it, so the check immediately above you is your next realistic target.
| LoB and dimension | θ | Comfortably passing | Next realistic target |
|---|---|---|---|
| Beta, Openness | +0.57 | Q2 Q5 Q4 Q3 Q6 Q1 | Q7 Knowledge Graph, at 28% odds today |
| Alpha, Runtime | −0.03 | Q5 Q4 Q3 Q6 Q1 | Q2 Agent-handled processes, at 39% |
| Gamma, Runtime | −0.49 | Q3 Q6 Q1 | Q5 Customer extensibility, at 42% |
A star rating cannot produce that last column. It says “★★★” and stops.
Three fictional LoBs, 37 products, the proposal's eight questions grouped into four dimensions. Raw is observed coverage on whatever checks applied, banded by the star table the proposal already uses. Rated bands maturity against one common yardstick: the coverage that maturity predicts across all eight questions, using the same ruler for every dimension.
| LoB | Dimension | raw cov | raw star | θ | common % | rated star | shift |
|---|---|---|---|---|---|---|---|
| Alpha | Experience and Process | 61% | ★★★★ | −0.41 | 42% | ★★★ | ↓ |
| Runtime | 57% | ★★★ | −0.03 | 49% | ★★★ | same | |
| Openness | 32% | ★★ | −0.02 | 49% | ★★★ | ↑ | |
| Data and Semantics | 39% | ★★ | −0.06 | 48% | ★★★ | ↑ | |
| Beta | Experience and Process | 59% | ★★★ | −0.51 | 40% | ★★ | ↓ |
| Runtime | 55% | ★★★ | −0.14 | 47% | ★★★ | same | |
| Openness | 18% | ★ | +0.57 | 60% | ★★★★ | ↑↑↑ | |
| Data and Semantics | 32% | ★★ | −0.46 | 41% | ★★★ | ↑ | |
| Gamma | Experience and Process | 71% | ★★★★ | +0.27 | 55% | ★★★ | ↓ |
| Runtime | 46% | ★★★ | −0.49 | 40% | ★★★ | same | |
| Openness | 42% | ★★★ | −0.49 | 40% | ★★★ | same | |
| Data and Semantics | 42% | ★★★ | +0.06 | 51% | ★★★ | same |
Star band edges on the common yardstick: 20% at θ −1.81, 40% at θ −0.51, 60% at θ +0.56, 80% at θ +1.83. One star is roughly one logit, or 174 Elo points.
Raw reads 61 / 57 / 32 / 39, a spread of more than two to one, with Openness looking like the problem. Rated reads flat: three stars across all four dimensions. The spread came from how hard the questions were, not from how mature the LoB is. Told to invest, Alpha's team would have poured effort into Openness, where they are comparatively strong, and left Experience and Process alone, where they are missing checks that nearly everyone else passes.
Openness moves from one star to four. Under raw coverage Beta earns the lowest rating on the entire scorecard for shipping MCP servers, which is the hardest thing anyone was asked to do. This is the cell that starts an argument in the review meeting, and it is an argument Beta wins.
The standing objection to a latent model is that it needs calibrated difficulties and a history of responses. Half of that is wrong. Calibration is cross-sectional: difficulties come from respondents times items in a single sitting, and nobody waits a year for an exam to become gradeable. The other half is right, and it comes down to how many rows the matrix has.
Monte Carlo, 8 items, 60 replications, joint maximum likelihood.
| rows | RMSE of b | verdict |
|---|---|---|
| 6 | 9.64 | items indistinguishable |
| 10 | 5.30 | items indistinguishable |
| 20 | 1.05 | items indistinguishable |
| 40 | 0.47 | ordering usable, gaps rough |
| 80 | 0.36 | ordering usable, gaps rough |
| 200 | 0.24 | solid |
| 500 | 0.20 | solid |
The unit of analysis decides everything here. Calibrate on eight or so LoBs and the difficulties are noise, so "Q8 is harder than Q1" becomes a coin flip. Calibrate on products, of which SAP has hundreds and which the proposal already collects, and the first cycle is enough.
Both methods scored against the same truth, varying how often a check is N/A. 400 simulated assessments per row.
| N/A rate | wrong star raw | wrong star rated | gain |
|---|---|---|---|
| 0% | 33.8% | 33.8% | ±0.0 |
| 5% | 37.8% | 36.6% | +1.2 |
| 10% | 38.4% | 35.7% | +2.6 |
| 20% | 43.5% | 37.5% | +6.0 |
| 30% | 47.8% | 37.9% | +9.9 |
Raw climbs, rated stays flat. At a 0% N/A rate the two are identical, as they have to be, since everyone answered the same questions. This buys you nothing when scope is uniform and progressively more as applicability diverges.
That 34% floor is a sample size problem, and no model fixes it. Twelve products per LoB against hard 20-point band cutoffs means a LoB whose true position is 39% lands on either side of the two-star line on the luck of two products. More products per LoB, or wider bands, is the only remedy. Worth settling before anyone treats a one-star difference as a finding.
Under Rasch, the raw score is a sufficient statistic. Which checks you passed adds nothing beyond how many you passed. That is the defining property of the whole Rasch family, and it buys specific objectivity: the guarantee that your rating does not depend on which other LoBs happened to be assessed alongside you.
Elo behaves the same way, which is the quickest way to see this is no quirk of the model:
Opponent strength is already priced into the expected score. Beating a strong opponent gains you more only when it lifts your total above expectation, and never because the win itself is worth extra. To make the pattern count on its own you have to leave the Rasch family for a 2PL model with a discrimination parameter per item, which needs considerably more data and gives up the property that anyone can check the arithmetic, which is what makes stars defensible in the first place.
MCP publication rates 1861 because 3 of 37 products have one today. If everyone ships MCP servers next cycle, that check slides toward easy and the same behaviour earns less maturity. That is the correct outcome, since a capability everyone has stops being evidence of leadership. It does mean cycle-two maturity is not comparable to cycle-one maturity unless some items are anchored at their original difficulty. Choose the anchor set before the first collection. Retrofitting it later is painful, and "did we improve?" is the first question leadership will ask.
Beta's Openness is +0.57 with a standard error of 0.78 logits, or 1600 plus or minus 136 Elo. It moves three stars and is still the least certain cell on the board, because it rests on a single check. A chess federation would call that provisional and decline to publish it. Print the standard error next to every rating, or people will over-read it the same way they over-read a coverage star.