Grader Comparison Dashboard
Analyze grader statistics and score divergence
Grader Overview
| Grader | Claims Reviewed | Total Scores | Avg Score | Std Dev |
|---|---|---|---|---|
|
L
lizzy
|
9 | 90 | 4.34 | 1.16 |
|
B
Bao
|
8 | 80 | 4.5 | 0.89 |
|
T
Test
|
2 | 2 | 3.5 | 0.71 |
|
J
Jawand
|
0 | 0 | 0.0 | 0.0 |
|
R
Richard
|
0 | 0 | 0.0 | 0.0 |
|
R
Rob
|
0 | 0 | 0.0 | 0.0 |
Score Divergence (2+ Point Disagreements)
AB
3499234442
MODERATE
5 divergent dimensions
Rationale:
- Minor severity instead of moderate since no surgery involved and just an ankle sprain. - Claimant's occupation is clearly a student but industry risk category is healthcare outpatient - Adjuster notes mentioned clear liability, but litigation risk rationale mentioned disputed liability leading to misclassification - Since litigation risk was misclassified, it leads to settlement likelihood and management complexity to be misclassified as well.
Rationale:
LLM classified claimant industry risk category as healthcare outpatient instead of education.
Rationale:
it was able to appropriately use the RICE definition
Rationale:
the claim was talking about a high ankle sprain which was relevant to the unstructured data
AJ
3628150755
MINOR
1 divergent dimension
Rationale:
it kind of is making up the price i think
AG
4726770034
MAJOR
2 divergent dimensions
Rationale:
i think it's messing up some of the prices or maybe I am not understanding them
Rationale:
it mentioned there was none for safety which i don't know is true since there was a safety harness, it just was not properly secured, so I don't know if there is a better term for this.
AK
5908806823
MODERATE
3 divergent dimensions
Rationale:
it understands everything, but like for this example, he fell when playing soccer, so I don't think there is any safety that should be involved and the LLM said unknown, and I wonder if there is a better category for this since there really isn't any safety stuff that can be included in this, it was just an accident, like he fell
Rationale:
the LLM may be hallucinating on the cost,
AB
6165669449
MODERATE
2 divergent dimensions
Rationale:
it did not include the safety harness in safety evaluation, which I think may change the category from none to something else. It's not that she wasn't using safety precautions, she just accidently fell.
Rationale:
it did not mention anything about how she feels like every day is reminder of what could have been with her wrist mobility. I don't know if we see this as important, but could it be a reason that she chooses to sue?? also, they left out the safety harness in the safety evaluation, so I don't know if that may be considered for something.
AN
6744942349
MODERATE
3 divergent dimensions
Rationale:
- Expected development pattern should be fast resolution because claimant recovered <6 months, clear liability, and returned to full duty. LLM mentioned "expected to return to full duty soon" - For industry risk category, claimant occupation is marketing specialist not healthcare outpatient
Rationale:
it left out some important points that i think would change some categories to score less harsh, like it should be a simple accident because it was a sprain with PT only, and I don't know if the model picked up on that.
Rationale:
it left out that the surface was dangerous for the safety variable, and it had the type of accident set to moderate when this should really be simple.
AK
7114747163
MODERATE
1 divergent dimension
Rationale:
- Per settlement adjuster notes, total settlement recommendation is approximately $32,600 instead of $118k so ultimate cost category should be 25-50k.
Rationale:
this one has a couple hallucinations, cost and safety procedures
AG
7909573436
MAJOR
1 divergent dimension
Rationale:
LLM mentioned ultimate cost prediction of $118,000 but no where to be found in settlement adjuster notes. Per settlement adjuster notes, the recommended settlement authority is up to $25,000.
Rationale:
the 118k is made up unless I am missing it.
To-Do List
| Claim | Lizzy | Bao |
|---|---|---|
|
AB
3499234442
|
|
|
|
AB
6165669449
|
|
|
|
AG
4726770034
|
|
|
|
AG
7909573436
|
|
|
|
AJ
3628150755
|
|
|
|
AK
5908806823
|
|
|
|
AK
7114747163
|
|
|
|
AN
6744942349
|
|
|
|
AS
1653216458
|
|
|
|
AS
6924817947
|
|
|
|
AT
6478583886
|
|
|
|
BC
6277819370
|
|
|
|
BC
6775379108
|
|
|
|
BF
9399212680
|
|
|
|
BH
1564979706
|
|
|
|
BH
2088482200
|
|
|
|
BH
4462291667
|
|
|
|
BP
8794084622
|
|
|
|
CA
3950946063
|
|
|
|
CH
9343912483
|
|
|