Back to index
OtherAn afternoon to get productive, ongoing as a reference·Free to read, pay per token for usage

OpenRouter Documentation and Model Guides

4.2

Genuinely useful documentation for a service that solves a real annoyance, let down by the fact that it is selling you something and the model comparison material reflects that.

What We Liked

  • Explains provider routing and fallbacks clearly, which is the actual product
  • One request format across dozens of providers removes a large amount of glue code
  • The pricing and rate limit pages are specific rather than vague
  • Model listings carry live latency and throughput numbers you can act on

What Could Be Better

  • It is vendor documentation and the comparisons are shaped accordingly
  • Almost nothing on evaluating whether a cheaper model is actually good enough for you
  • The provider-specific quirks section is thinner than the problem deserves
  • Adding a hop between you and the model has failure modes the docs underplay

Detailed review

The problem this solves is boring and real. You build something against one model provider, then you want to try a different model, and you discover that the request format is different, the streaming format is different, the error codes are different, the rate limit headers are different, and the tool calling schema has its own dialect. You end up writing an adapter layer, and then you end up maintaining an adapter layer, which is nobody's idea of a good use of an engineering week. OpenRouter puts a routing service in front of the providers and gives you one format for all of them.

The documentation is worth reading as a learning resource rather than only as a reference, and the reason is the routing material. The pages on provider preferences, fallback ordering, and what happens when a provider is degraded or rate limiting you are the most educational thing here, because they force you to think about questions you probably have not thought about. What should happen when your primary provider returns a five hundred halfway through a stream? Do you want to fail, retry on the same provider, or silently fall back to a different one that will produce a slightly different answer? Is a slightly different answer acceptable in your product? Most people building on a single provider never confront this until it happens in production at an inconvenient hour. The docs make you confront it while you are still reading.

The pricing pages deserve credit for being specific. Cost per million input tokens, cost per million output tokens, and the routing fee, stated plainly with the arithmetic visible. This is a low bar and a surprising number of AI infrastructure companies fail to clear it. Same for the rate limit documentation, which tells you the actual numbers rather than telling you to contact sales.

The live model listings, with throughput and latency measured across providers, are the sort of operational data that is genuinely hard to get anywhere else. If you want to know whether a given open model is being served fast enough somewhere to be usable in an interactive product, the answer is on the page. Now the honest problems. This is a commercial service documenting its own product, and the framing follows from that.

The docs are structured around the assumption that routing through a third party is the correct architecture, and the case against it gets very little space. There are real reasons not to do this. You are adding a network hop and a dependency to every model call. You are sending your prompts, which may contain things you would rather not send, through an additional company.

You are taking on a billing relationship that sits between you and the provider whose model you are actually using. None of these are disqualifying and all of them deserve more than the passing treatment they get. The model comparison material is the weakest part. You get prices, context windows, and throughput, which are the easy facts.

You get almost nothing on the harder question, which is whether the cheap model will do your job as well as the expensive one. That question is answerable, and the answer is that you build an evaluation set for your actual task and run it, and the docs barely gesture at this. It is a strange gap for a service whose main pitch is that switching models is now easy, because the thing that makes switching hard is not the API format, it is knowing whether the switch made your product worse. The provider quirks are underdocumented relative to how much trouble they cause.

Different providers serving nominally the same open model produce meaningfully different results, because of quantisation, because of different default sampling parameters, because of different context handling at the edges. Anyone routing across providers will hit this eventually and the documentation prepares you for it only lightly. My four point two is for documentation that is clear, specific about the things vendors usually fudge, and educational on routing and fallback behaviour in a way that will make you a better engineer even if you never use the service. Marked down because it is a sales document at heart, because the model selection guidance stops exactly where the difficulty starts, and because the operational reality of running through an aggregator is presented more smoothly than it deserves.

Worth an afternoon, and read the routing pages twice.

[ final ]

The verdict.

Read the routing and provider preference pages properly, because that is where the real learning is. Treat the model comparison pages as a catalogue rather than as advice.