audio patch: bill speech too — the provider-cost early return pre-empted the tail patch

response_cost_calculator returns provider_response_cost as soon as the response
carries one in hidden params and no custom pricing is attached. That return sits
ABOVE the per-request splice, so tts calls exited at 0.0 and never reached it —
which is why whisper billed $0.01/request and speech billed nothing, with no
[coop-audio] debug line to show for it.

The early return now adds the declared per-request charge. Guarded by the
DECLARATION (input_cost_per_request), not the call type, so token-priced models
like voxtral-small-24b stay inert.
This commit is contained in:
inference-admin committed 2026-11-16 19:55:43 +00:00
1 parent a1613401c3
commit 7a167a8354
1 file changed
+22
+22
View File
@@ -135,12 +135,29 @@ HELPER = '''def _coop_per_request_cost(
ANCHOR_DEF = "def response_cost_calculator(" ANCHOR_DEF = "def response_cost_calculator("
ANCHOR_EARLY = """ if provider_response_cost is not None and not custom_pricing:
return provider_response_cost"""
ANCHOR_RETURN = """ region_name=region_name, ANCHOR_RETURN = """ region_name=region_name,
) )
return response_cost return response_cost
except Exception as e: except Exception as e:
raise e""" raise e"""
PATCHED_EARLY = """ if provider_response_cost is not None and not custom_pricing:
# CO-OP PATCH (2026-11-16): a provider-reported cost is not the
# whole bill on the audio paths. Speech returns here and never
# reaches the tail patch, so add the declared per-request charge
# before returning. Guarded by the DECLARATION, not the call
# type: models without input_cost_per_request (voxtral-small-24b,
# token-priced) stay inert.
return provider_response_cost + _coop_per_request_cost(
model=model,
litellm_model_name=litellm_model_name,
custom_pricing=custom_pricing,
litellm_logging_obj=litellm_logging_obj,
)"""
PATCHED_RETURN = """ region_name=region_name, PATCHED_RETURN = """ region_name=region_name,
) )
@@ -232,6 +249,11 @@ def main() -> int:
patched = source.replace(ANCHOR_DEF, HELPER + ANCHOR_DEF, 1) patched = source.replace(ANCHOR_DEF, HELPER + ANCHOR_DEF, 1)
patched = patched.replace(ANCHOR_RETURN, PATCHED_RETURN, 1) patched = patched.replace(ANCHOR_RETURN, PATCHED_RETURN, 1)
if ANCHOR_EARLY in patched:
patched = patched.replace(ANCHOR_EARLY, PATCHED_EARLY, 1)
print("[coop-patch] early provider-cost return also patched (audio speech)")
else:
print("[coop-patch] WARNING early-return anchor not found; speech may stay unbilled")
compile(patched, path, "exec") # refuse to write something that will not import compile(patched, path, "exec") # refuse to write something that will not import
if not os.access(path, os.W_OK): # practice run against a read-only copy if not os.access(path, os.W_OK): # practice run against a read-only copy