Repository navigation
Usage questions #16
Description
Activity
Hii Thanks for showing interest in GPTRouter
-> Making it a service instead of a library gives us a health-checking heartbeat that maintains a list of healthy models below a target latency, this helps the model router to switch between requested models in priority array based on the health.
-> Yes it can be used with a NoSQL database but we're using it in production using a Postgres with good reliability.
-> For logging we use an internally deployed version of sentry paired with some logging.
Hi,
I am planning to use GPTRouter but had a few questions before making my decision. Could you please help answer these questions?
Q1: Library vs Router
What is the need for a router as a service instead of a library? I was wondering why you did not go the path of creating a wrapper library from which to invoke the various AI model API.
Here are some reasons I can think of -
a) Manage all the access keys in one place.
b) The library size can get bigger as more and more AI models are integrated, while as a consumer I might be interested in only a one or
2 models.
Q2: Database
Could you also let me know if there is any documentation on what is managed in the Postgres database? Is there ability to switch into a NoSQL database?
Q3: Logging
If I self-host GPTRouter what do I integrate to get all the logging from the GPTRouter for debugging and observability?
Thank you!