Infrastructure to enable deployment of ML models to low-power resource-constrained embedded targets (including microcontrollers and digital signal processors).

OpenMV Camera Module

A curated list of resources for embedded AI

Beatmup: image and signal processing library

ncnn is a high-performance neural network inference framework optimized for the mobile platform

YoloV8 for a bare Raspberry Pi 4 or 5

On-device AI across mobile, embedded and edge for PyTorch

llm deploy project based mnn.

GPT4All: Chat with Local LLMs on Any Device

MNN is a blazing fast, lightweight deep learning framework, battle-tested by business-critical use cases in Alibaba

BLAS library for VideoCore VI QPU (Raspberry Pi 4)

Python library for GPGPU programming on Raspberry Pi 4

C++ library for programming the VideoCore GPU on all Raspberry Pi's.

ShaderNN is a lightweight deep learning inference framework optimized for Convolutional Neural Networks on mobile platforms.

FyuseNet is an OpenGL(ES) based library that allows to run neural network inference on GPUs that support OpenGL or OpenGL/ES, which is the case for most desktop and mobile GPUs on the market.

Python bindings for llama.cpp

Collective communications library with various primitives for multi-machine training.

Tensor parallelism is all you need. Run LLMs on weak devices or make powerful devices even more powerful by distributing the workload and dividing the RAM usage.