Blackwell-Native 8-bit and 4-bit RL in Miles | SGLang님이 토픽에 대해 올림
Blackwell-Native 8-bit and 4-bit RL in Miles
이 제목은 아래 게시물에서 AI가 요약한 것입니다.
🚀 New blog: Towards Blackwell-Native 8-bit and 4-bit RL: End-to-End MXFP8 and NVFP4 RL in Miles We bring two Blackwell-native, low-precision RL recipes to Miles, fully open-sourced across the stack: ☑️ End-to-end MXFP8 across rollout, forward, and backward GEMMs ☑️ Hardware-native NVFP4 W4A4 RL for MoE models ☑️ Bit-exact quantizers between training & rollout to minimize training-rollout mismatch ☑️ Fine-grained, cross-stack per-layer precision control On Qwen3-30B-A3B, all 5 low-precision configs closely track the BF16 reward curve while reducing rollout time. Built together with humans& and NVIDIA. Read full blog: https://lnkd.in/eZMurnZX
댓글을 보거나 남기려면 로그인
관련성이 더 높은 업데이트
- 🚀 The smaller-footprint INT4 and MXFP4 variants of Ling-3.0-flash are now available on ModelScope: 🤖 FP4: https://lnkd.in/ecnsRdrA 🤖 INT4: https://lnkd.in/eUXChdYM Both can run end-to-end on a single NVIDIA DGX Spark via the Spark-adapted SGLang path. In the Ling team’s tests: ⚡ ~80 tok/s decoding 📥 2.5–3.5K tok/s long-input prefill 👥 Smooth support for 3–4 concurrent users Designed for private local coding, agent workflows, and batch inference. According to the Ling team, FP4 W4A16 is the stable default, while W4A8 is tuned for higher throughput. #Ling30Flash #ModelScope #INT4 #MXFP4 #NVIDIA #SGLang #LocalAI #LLM #Quantization #AICoding 댓글을 보거나 남기려면 로그인
팔로워 10,820명
- Source: Unknown