Modeling, training, eval, and inference code for OLMo

Mathematical Visual Instruction Tuning for Multi-modal Large Language Models

📖 This is a repository for organizing papers, codes and other resources related to unified multimodal models.

Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

Repository for Show-o, One Single Transformer to Unify Multimodal Understanding and Generation.

research work on multimodal cognitive ai

MiniCPM-V 2.6: A GPT-4V Level MLLM for Single Image, Multi Image and Video on Your Phone

🔥🔥MLVU: Multi-task Long Video Understanding Benchmark

Official Implementation of "Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining"

Repository hosting code used to reproduce results in "Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations" (https://arxiv.org/abs/2402.17152).

(ෆ`꒳´ෆ) A Survey on Text-to-Image Generation/Synthesis.

PyTorch implementation of MAR+DiffLoss https://arxiv.org/abs/2406.11838

Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process

Framework to achieve context distillation in LLMs

MINT-1T: A one trillion token multimodal interleaved dataset.

Implementation of Autoregressive Diffusion in Pytorch

Mixture of A Million Experts

Scaling Diffusion Transformers with Mixture of Experts

Official implementation of MARS: Mixture of Auto-Regressive Models for Fine-grained Text-to-image Synthesis

Transformer implementation for "Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion"

code for "Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion"

A Survey of Image Editing

Official implementation of ⚡ Flash Diffusion ⚡: Accelerating Any Conditional Diffusion Model for Few Steps Image Generation

Anole: An Open, Autoregressive and Native Multimodal Models for Interleaved Image-Text Generation

A RLHF Infrastructure for Vision-Language Models

Visual Instruction Tuning for Qwen2 Base Model

Autoregressive Model Beats Diffusion: 🦙 Llama for Scalable Image Generation

RLAIF-V: Aligning MLLMs through Open-Source AI Feedback for Super GPT-4V Trustworthiness

[ECCV 2024] Bridging Different Language Models and Generative Vision Models for Text-to-Image Generation