The RAW method uses LLM stochasticity to measure how much user re-engagement drops when a model refuses a request. Testing across 15 models in LMSYS-Chat-1M and WildChat shows that refusals cost between 3 and 17 percentage points in subsequent user re-engagement.
HOW THIS AFFECTS YOU
●
builderYou can now estimate the specific cost of model refusals on user retention.
●
founderHigh refusal rates may directly drive users away from your product.