Show HN: I built an open source AI gateway that proxy calls to multiple LLMs
github.comClients can talk via HTTP or gRPC. Comes with rate limiting, failover, caching, guardrails, logging and an admin API for configuration
Love to get feedback if this is interesting to you
Interesting language choice! What influenced your decision to use Ballerina?
Three main reasons: 1. I've messed around with the language enough that it's familiar to me 2. This gateway is mostly about converting data between formats, and I find Ballerina easier for that 3. I was not looking for super low-latency as a typical gateway because LLMs are slower than stateless services