Lists (5)
Starred repositories
High-level, multiplatform C++ network packet sniffing and crafting library.
Scapy: the Python-based interactive packet manipulation program & library.
QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
an example to use udp to boradcast
[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Code at the speed of thought – Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
Tensor parallelism is all you need. Run LLMs on weak devices or make powerful devices even more powerful by distributing the workload and dividing the RAM usage.
The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".
Collective communications library with various primitives for multi-machine training.
Survey Paper List - Efficient LLM and Foundation Models
unix socket interface for C++ raw IP/IP6/UDP/TCP, Layer2 etc. framework
CVPR 2024 accepted paper, An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learning
mtnwrw / fyusenet
Forked from Fyusion-Open-Source/fyusenetFyuseNet is an OpenGL(ES) based library that allows to run neural network inference on GPUs that support OpenGL or OpenGL/ES, which is the case for most desktop and mobile GPUs on the market.
ShaderNN is a lightweight deep learning inference framework optimized for Convolutional Neural Networks on mobile platforms.
Application layer implementation of TCP and UDP protocols using linux RAW sockets
An example about working with raw sockets under GNU/Linux
A very basic first attempt at RAG with ollama and Phi-3 Mini
Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papers on accelerating LLMs, currently focusing mainly on infer…
Collection of awesome LLM apps with RAG using OpenAI, Anthropic, Gemini and opensource models.