QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

an example to use udp to boradcast

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

Code at the speed of thought – Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.

Tensor parallelism is all you need. Run LLMs on weak devices or make powerful devices even more powerful by distributing the workload and dividing the RAM usage.

The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".

LLM101n: Let's build a Storyteller

Collective communications library with various primitives for multi-machine training.

Survey Paper List - Efficient LLM and Foundation Models

unix socket interface for C++ raw IP/IP6/UDP/TCP, Layer2 etc. framework

CVPR 2024 accepted paper, An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learning

Simple Raspberry Pi monitoring tool

FyuseNet is an OpenGL(ES) based library that allows to run neural network inference on GPUs that support OpenGL or OpenGL/ES, which is the case for most desktop and mobile GPUs on the market.

ShaderNN is a lightweight deep learning inference framework optimized for Convolutional Neural Networks on mobile platforms.

Application layer implementation of TCP and UDP protocols using linux RAW sockets

An example about working with raw sockets under GNU/Linux

Self-hosted AI coding assistant

A very basic first attempt at RAG with ollama and Phi-3 Mini

P4 on Raspberry Pi for Networking Education

Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papers on accelerating LLMs, currently focusing mainly on infer…

Packet Test Framework

Collection of awesome LLM apps with RAG using OpenAI, Anthropic, Gemini and opensource models.

[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4V. 接近GPT-4V表现的可商用开源多模态对话模型

p4

Awesome Docker Compose samples