Skip to content

Commit

Permalink
[Docs] Update Readme on project intro (#191)
Browse files Browse the repository at this point in the history
* Update Readme on project intro

* Update index.rst accordingly
  • Loading branch information
Liguang Xie authored Sep 19, 2024
1 parent e5e8601 commit 76b0b06
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 12 deletions.
16 changes: 11 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,19 @@
# AIBrix

Welcome to AIBrix, the foundational building blocks for constructing your own GenAI inference infrastructure. AIBrix offers a cloud-native solution tailored to meet the demands of enterprises aiming to deploy, manage, and scale LLMs efficiently.
Welcome to AIBrix, an open-source initiative designed to provide essential building blocks to construct scalable GenAI inference infrastructure. AIBrix delivers a cloud-native solution optimized for deploying, managing, and scaling large language model (LLM) inference, tailored specifically to enterprise needs.

## Key Features

- High density Lora management
- Intelligent and LLM specific routing strategies
- LLM tailored pod autoscaler
- AI runtime sidecar (metrics merge, fast model downloading, admin operations)
The initial release includes the following key features:

- **LLM Gateway and Routing**: Efficiently manage and direct traffic across multiple models and replicas.
- **High-Density LoRA Management**: Streamlined support for lightweight, low-rank adaptations of models.
- **Distributed Inference**: Scalable architecture to handle large workloads across multiple nodes.
- **LLM App-Tailored Autoscaler**: Dynamically scale inference resources based on real-time demand.
- **Unified AI Runtime**: A versatile sidecar enabling metric standardization, model downloading, and management.
- **GPU Hardware Failure Detection (TBD)**: Proactive detection of GPU hardware issues.
- **Benchmark Tool (TBD)**: A tool for measuring inference performance and resource efficiency.


## Quick Start

Expand Down
16 changes: 9 additions & 7 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,18 @@ Welcome to AIBrix
:align: center
:alt: AIBrix

AIBrix is the foundational building blocks for constructing your own GenAI inference infrastructure.
AIBrix offers a cloud-native solution tailored to meet the demands of enterprises aiming to deploy, manage, and scale LLMs efficiently.
AIBrix is an open-source initiative designed to provide essential building blocks to construct scalable GenAI inference infrastructure.
AIBrix delivers a cloud-native solution optimized for deploying, managing, and scaling large language model (LLM) inference, tailored specifically to enterprise needs.

Key features:

* High density Lora management
* Intelligent and LLM specific routing strategies
* LLM tailored pod autoscaler
* AI runtime sidecar (metrics merge, fast model downloading, admin operations)

- **LLM Gateway and Routing**: Efficiently manage and direct traffic across multiple models and replicas.
- **High-Density LoRA Management**: Streamlined support for lightweight, low-rank adaptations of models.
- **Distributed Inference**: Scalable architecture to handle large workloads across multiple nodes.
- **LLM App-Tailored Autoscaler**: Dynamically scale inference resources based on real-time demand.
- **Unified AI Runtime**: A versatile sidecar enabling metric standardization, model downloading, and management.
- **GPU Hardware Failure Detection (TBD)**: Proactive detection of GPU hardware issues.
- **Benchmark Tool (TBD)**: A tool for measuring inference performance and resource efficiency.

Documentation
=============
Expand Down

0 comments on commit 76b0b06

Please sign in to comment.