Tensor parallelism is all you need. Run LLMs on an AI cluster at home using any device. Distribute the workload, divide RAM usage, and increase inference speed.
-
Updated
Dec 18, 2024 - C++
#
llama2
Here are 11 public repositories matching this topic...
校招、秋招、春招、实习好项目，带你从零动手实现支持LLama2/3和Qwen2.5的大模型推理框架。
-
Updated
Jan 7, 2025 - C++
Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
bloom falcon moe gemma mistral mixture-of-experts model-quantization multi-gpu-inference m2m100 llamacpp llm-inference internlm llama2 qwen baichuan2 mixtral phi-2 deepseek minicpm
-
Updated
Mar 15, 2024 - C++
LLaVA server (llama.cpp).
-
Updated
Oct 20, 2023 - C++
Runs LLaMA with Extremely HIGH speed
-
Updated
Nov 21, 2023 - C++
Llama causal LM fully recreated in LibTorch. Designed to be used in Unreal Engine 5
-
Updated
Sep 19, 2024 - C++
Super easy to use library for doing LLaMA/GPT-J stuff! - Mirror of: https://gitlab.com/niansa/libjustlm
-
Updated
Mar 25, 2024 - C++
Multi-Model and multi-tasking llama Discord Bot - Mirror of: https://gitlab.com/niansa/discord_llama
-
Updated
Mar 27, 2024 - C++
Improve this page
Add a description, image, and links to the llama2 topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llama2 topic, visit your repo's landing page and select "manage topics."