Everyone who ships something built on a model API eventually writes the same code. Retry when the provider returns a rate limit error. Fall back to a different model when the first one is down. Cache responses so you stop paying for the same question.
Log requests so you can work out what went wrong. Add a budget check so a runaway loop does not produce a memorable invoice. Almost everyone writes this badly the first time, scattered through the application, and a gateway is the argument that it belongs in one place. Portkey's documentation is a decent education in what that layer should do.
The fallback and retry material is the strongest part. It covers configuring chains of providers, handling different error types differently, and the distinction between a retry that might succeed and one that will not. That last point is where most home rolled implementations fail, retrying a malformed request repeatedly while treating a transient overload the same way. The documentation explains the reasoning rather than just showing configuration syntax, which means the knowledge is useful even if you implement it yourself.
Semantic caching is well explained and it is a concept people frequently get wrong. Caching on exact string matches catches almost nothing in practice, because users phrase the same question differently every time. Caching on embedding similarity catches far more and introduces a threshold problem, where too loose a match returns an answer to a different question. The documentation covers the tradeoff honestly, including where semantic caching is inappropriate, and that honesty is welcome given how much money the feature can save when it is used well.
Budget and rate limiting coverage addresses a failure mode that is more common than people admit. Applications that call models can consume money at a rate limited only by how fast they can loop, and a bug that retries aggressively will find that limit. Documentation that treats spending controls as a first class concern rather than an afterthought is doing its readers a service. The observability material connects logging to debugging usefully.
Knowing which requests failed, what was sent, how long it took and what it cost is the difference between diagnosing a problem in minutes and guessing at it for a day. The documentation frames this around the questions you will actually ask, which suggests it was written by people who have had to answer them. The overlap with LiteLLM is the elephant in the room. Both provide a unified interface across providers, both handle fallbacks and caching, both offer observability, and both have open source and hosted versions.
The documentation for each is naturally silent about why you might prefer the other. My honest read is that the differences are real but not enormous, that LiteLLM has broader provider coverage and a larger community, and that Portkey's configuration model and hosted observability are cleaner. Anyone choosing should spend an hour with both rather than picking whichever they read about first. The open source and hosted boundary is not always clear.
Some capabilities described in the documentation are features of the managed platform, and the pages do not always make that obvious. This is the standard open core presentation problem and it costs readers time when they configure something locally that turns out to be a hosted feature. The architectural cost of a gateway is underdiscussed. You are placing a component between your application and the model provider, which means it can fail, add latency, and become a dependency you now operate.
For a self hosted deployment that is a real operational consideration, and for a hosted gateway you have added a third party to your critical path. Neither is wrong and both deserve more discussion than documentation written by a gateway vendor is likely to provide. Provider integration depth varies. The major providers are documented thoroughly, and smaller or newer ones get briefer treatment where you may need to work out behaviour by experiment.
That is unavoidable given how many providers exist and it is worth checking that your specific provider is well covered before you commit. Three point eight for documentation that explains an important and frequently botched layer with genuine clarity, particularly around reliability patterns and caching. Held back by an unaddressed competitive overlap, open core boundaries that could be clearer, and insufficient discussion of what adding this component costs you.