|
Canada-0-Fireproofing Azienda Directories
|
Azienda News:
- LLM塞进一个Kernel里？Megakernels介绍 - 知乎
一段话介绍：对于性能瓶颈不在于计算的推理场景（小模型，小batch），作者为了解决模型推理时的内存流水线气泡问题，将Llama-3 2-1B-Instruct写在一个Kernel里来尝试解决内存瓶颈。 并解决了将一个模型写在一个Kernel中会遇到的问题。
- HazyResearch Megakernels: Kernels, of the mega variety . . . - GitHub
Kernels, of the mega variety :) Contribute to HazyResearch Megakernels development by creating an account on GitHub
- Mirage Persistent Kernel: A Compiler and Runtime for Mega-Kernelizing . . .
We introduce Mirage Persistent Kernel (MPK), the first compiler and runtime system that automatically transforms multi-GPU model inference into a single high-performance megakernel
- Megakernels 开源项目最佳实践教程 - CSDN博客
1 项目介绍 Megakernels 是一个由 HazyResearch 开发的大型内核（mega-kernels）项目，旨在通过优化和重构现有的内核代码，提高计算效率，特别是在低延迟和高吞吐量的应用场景中。 该项目使用 Python 和 Cuda 语言，专注于为深度学习等高性能计算任务提供优化的解决
- Triton-distributed MegaKernel初试 - 知乎
首先，Triton可以实现persistent kernel，能精确控制每个SM每个step做什么，这个能力允许我们开发MegaKernel。 其次，Megakernel只需要下发一次参数，让Triton的host侧开销大大减少，基本上和直接跑pybind的CUDA kernel没有差别。 因此，我们开始着手开发Triton-distributed MegaKernel。
- Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1B
The performance limitation with the normal many-kernel execution model is that no thread blocks in a kernel can start until all thread blocks in previous kernels are finished However, it's precisely this property that makes it easy to manage data dependencies
- Compiling LLMs into a MegaKernel: A Path to Low-Latency Inference
Motivated by this question, our team from CMU, UW, Berkeley, NVIDIA, and Tsinghua developed Mirage Persistent Kernel (MPK) — a compiler and runtime system that automatically transforms multi-GPU
- Megakernel Implementations — Triton-distributed documentation
EP All-to-All Fused Kernel: Fused megakernel combining dispatch+groupgemm and groupgemm+combine operations with token optimization (token saving skipping, token sorting, SM scheduling)
- 什么是Mega Kernels | AIUG
Mega Kernels 是一个专注于 深度学习 模型执行优化的开源项目，旨在通过高度优化的内核实现，为深度学习模型提供实时、低延迟的执行环境。 该项目基于 Python 和 PyTorch 深度学习框架，利用 Py Torch 的动态图特性，实现了对模型的即时编译和优化。
- BodhiHu mirage-llm-megakernel - GitHub
Mirage Persistent Kernel (MPK) is a compiler and runtime system that automatically transforms LLM inference into a single megakernel—a fused GPU kernel that performs all necessary computation and communication within a single kernel launch
|
|