Before working for 2 years on the Apache APISIX API gateway, I was mainly oblivious to API gateways. It’s only by working with them that I understood their value. Decoupling the client and the server unlocks a lot of options: moving authentication to the API Gateway, securing APIs, deduplicating API requests, etc. In this post, I want to describe how the same pattern applies to AI. AI gateways AI gateways work in a similar way.

Last year, I started to dig a bit around LangChain4J. It’s a fast growing project, and I wanted to get familiar with the updates. I also wanted to check how I could integrate a Model Context Protocol server in LangChain4J.

I’m coming relatively late to the LLM party, but I rarely come very early in the hype cycle. For example, I never bought into blockchain, the solution still searching for problems to solve, nor in microservices, the latest in the cargo cult IT trends. Despite my late arrival at the LLM party, I have been a regular user of LLMs. I use OpenAI for non-controversial questions outside my cone of knowledge, e.g., linguistics or legal; I use GitHub Copilot in my IDE to improve my code.