← All provider guides

OpenAI drop-in replacement APIs: what actually breaks when you switch

openai openai-compatible 26 tracked change records

"OpenAI-compatible" has become the de facto wire protocol for LLM serving. Ollama, vLLM, LM Studio, and most hosted inference providers expose a /v1/chat/completions endpoint that accepts the same JSON the OpenAI SDK sends, so in principle you point baseURL at a new host and everything keeps working. In practice the promise is narrower than it sounds, and the gap is where integrations break.

What the compatibility promise actually covers

A drop-in replacement guarantees the request and response shape of a small set of endpoints, almost always /v1/chat/completions and sometimes /v1/embeddings and /v1/models. It does not guarantee:

  • Parameter parity. Fields like logit_bias, seed, response_format, tool calling, or vision inputs are accepted by some servers, silently ignored by others, and rejected with a 400 by the rest. The same request body can succeed, degrade, or fail depending on the host behind the URL.
  • Error shape parity. OpenAI returns a structured error.code / error.type object. Compatible servers frequently return their own error bodies, so retry logic keyed on OpenAI error codes stops classifying failures correctly the day you switch.
  • SDK behavior. The official SDKs evolve independently of the wire protocol. When the openai Node SDK shipped v7.0.0, the breaking change was in the client runtime contract, not the HTTP API. A replacement server cannot shield you from that class of change at all.

The three failure modes we see in real codebases

1. The base URL is compatible, the SDK version is not

Teams pin the wire protocol by switching hosts, then upgrade the SDK and inherit its breaking changes anyway. Of the 26 OpenAI change records in the mendapi database, the breaking entries are dominated by SDK contract changes: runtime requirements, removed types, renamed parameters. Your replacement server never sees these. Your build does.

2. Silent parameter degradation

A request that carries an unsupported field usually does not fail. It returns 200 with the field ignored. Sampling parameters and structured-output settings are the classic cases: the code path looks healthy while output quality quietly changes. Nothing in your logs points at the switch.

3. Auth and header assumptions

Local servers commonly accept any bearer token; hosted compatible providers require their own keys and sometimes extra headers. Code that validates key format, or reads rate-limit headers OpenAI sends and the replacement does not, breaks in ways that only show up under load.

How mendapi treats openai-compatible endpoints

This is not theoretical advice: mendapi's own BYO-LLM layer ships an openai-compatible provider mode. The contract is three environment variables, and the same three lines cover Ollama, vLLM, or any hosted compatible endpoint:

MENDAPI_LLM_PROVIDER=openai-compatible
MENDAPI_LLM_BASE_URL=http://localhost:11434/v1
MENDAPI_LLM_MODEL=<your model>   # API key optional for local servers

Building that client forced the same decisions this page describes. We validate the base URL up front and fail loud on a missing model rather than letting a default drift between servers, because a silently wrong assumption is the expensive kind.

Check your own codebase before you switch

The useful question is not "is this server compatible" but "which OpenAI API surfaces does my code actually touch". That determines whether the switch is safe and which future upstream changes can hit you. mendapi answers it locally, in about 30 seconds, with no code leaving your machine:

npx mendapi scan
# -> findings: file, line, provider, confidence
mendapi deps
# -> provider x API surface x location, per repo

The scan lists every OpenAI SDK call site and endpoint reference with file and line, so you can audit exactly which parameters and endpoints your integration depends on before pointing it somewhere new. After the switch, mendapi deps --match tells you which tracked upstream changes touch the surfaces you use.

Related

Every entry on this page is generated from the mendapi change database at build time and links its upstream source.