GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

LLM inference in C/C++

FlatBuffers: Memory Efficient Serialization Library

A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning …

A high-performance, zero-overhead, extensible Python compiler using LLVM

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

Tensor library for machine learning

Unsupervised text tokenizer for Neural Network-based text generation.

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

Implementation of popular deep learning networks with TensorRT network definition API

Transformer related optimization, including BERT, GPT

Fast inference engine for Transformer models

LightSeq: A High Performance Library for Sequence Processing and Generation

A machine learning compiler for GPUs, CPUs, and ML accelerators

The AI-native database built for LLM applications, providing incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text

Clover: Quantized 4-bit Linear Algebra Library

The official SALIENT system described in the paper "Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining".

The official, proof-of-concept C++ implementation of PocketNN.