Ggml v1.0.0

v1.0.0 Latest Latest Compare Choose a tag to compare Sorry, something went wrong. Filter Loading Sorry, something went wrong. Uh oh! There was an error while loading. Please reload this page . No results found View all tags github-actions released this 13 Nov 08:34 v1.0.0 417a024 This commit was created on GitHub.com and signed with GitHub’s verified signature . GPG key ID: B5690EEEBB952194 Verified Learn about vigilant mode . Overview ggml 1.0.0 prunes the deprecated public API and the legacy DIAG_MASK ops, adds configurable YaRN truncation and FP8 support, and brings broad performance work across backends: few-row quantized matmuls and MoE fusion on CUDA/Metal, a Vulkan prefill FlashAttention path, a device-side MoE scheduler on SYCL, decomposed Adreno FlashAttention in OpenCL, new WebGPU kernels, expanded Hexagon op coverage, and scheduler improvements that avoid uploading unused inputs. API changes Pruned deprecated API and removed the legacy DIAG_MASK_INF / DIAG_MASK_ZERO ops, bumping the RPC protocol to reject incompatible peers ( llama/29801 , llama/31946 ) Added ggml_rope_set_truncate / ggml_rope_get_truncate to control YaRN correction-bound rounding ( llama/30206 , llama/30802 ) Implemented the backward pass for TRI ( llama/30712 ) Added MUL_MAT src1 precision tests and gating for ggml_prec_set_acc ( llama/31896 , llama/31851 ) Bumped the RPC protocol to v9 with faster loading and graceful failure on unreachable endpoints or failed handshakes ( llama/29413 , llama/30415 ) Core changes Scheduler skips uploading inputs that are not read and prefetches only inputs produced by earlier splits ( llama/31419 , llama/32100 ) MoE cache can compute a share of the cache misses on the CPU and copy the results back asynchronously ( llama/30966 ) Meta backend raises the compute view limit for buffers with few tensors ( llama/31447 ) Backend changes CUDA Added few-row MMQ for batch sizes 2-24 and MMVF for BF16 few-row matmuls ( llama/31533 , llama/31959 ) Added OCP FP8 support and fused NVFP4 gate/up matmuls with GLU in MMQ ( llama/28413 , llama/31296 ) Extended FlashAttention with an sm70 head-size-256 config and MMA head sizes 40/72 ( llama/27997 , llama/30285 ) Improved graph handling: skip compat checks on reuse, re-instantiate on cluster-dim changes, move graph state ( llama/32004 , llama/31284 , llama/32001 ) MoE: skip empty MMQ stream-k tiles, linearize the MoE layout, and sanitize post-bias NaNs ( llama/32067 , llama/31047 , llama/31005 ) Added a debug buffer pool and halved the VMM pool reservation with reserve retries ( llama/31919 , llama/30542 ) HIP: use a fixed BLAS workspace and add an MFMA lightning indexer for CDNA2 ( llama/31838 , llama/29050 ) Added a d_state=64 SSM_SCAN fast path and made radix Top-K deterministic for ties ( llama/31818 , llama/31084 ) Metal Fused MoE routing and kept fusable node groups inside a single command buffer ( llama/31615 , llama/31688 ) Made radix Top-K deterministic for ties, added mul_mm for 32<=K<64, and switched TRI to select ( llama/31084 , llama/30982 , llama/30711 ) Fixed GROUP_NORM for ne03>1 and out-of-bounds accesses in the FA output and SOLVE_TRI kernels ( llama/31852 , llama/31662 , llama/32103 ) Vulkan Added an Intel FlashAttention prefill kernel ( llama/29357 ) Added batched tensor copies, a test-fusion interface, and ReBAR detection ( llama/31418 , llama/30769 , llama/31658 ) Enabled dequantized KV with CM2, YaRN truncation, RDNA3/4 large-tile split-K, and large coopmat matmul on newer AMD drivers ( llama/31526 , llama/30802 , llama/30280 , llama/31587 ) Added a tiled-transpose concat shader and FWHT kernels for block widths above 512 ( llama/30591 , llama/29772 ) Fixed MoltenVK out-of-bounds accesses, GROUP_NORM workgroup limits, and FA scratch sizing ( llama/31890 , llama/31930 , llama/30487 ) Capped scalar FlashAttention at 8 rows for large heads on Qualcomm ( llama/30829 ) SYCL Added a device-side MoE scheduler ( llama/30156 ) Made the oneDNN FA cache thread-safe and un-gated fused SDPA for Alchemist at head_dim=64 ( llama/31138 , llama/31574 ) Cached the device id and reject buffers that would exceed VRAM ( llama/31126 , llama/32013 , llama/31285 ) Added exact rounding for CPY to Q4_1/Q5_1/Q8_0, strided CLAMP, and f16 SWIGLU_OAI ( llama/31465 , llama/31784 , llama/30857 ) Fixed AOT for allocations above 4GB ( llama/31873 ) OpenCL Added decomposed Adreno FlashAttention and improved decode performance ( llama/32046 , llama/31633 ) Added q6_k tiled, q8_0, and f32/q5_k multi-column GEMV kernels and f32 MUL_MAT_ID ( llama/31758 , llama/30947 , llama/31783 ) Added arange, non-contiguous sigmoid, repeat for f16/i32/i16/bf16, and DSV4 HC pre/post kernels ( llama/31829 , llama/31687 , llama/31782 , llama/31623 ) Enabled multi-column GEMV by default for Adreno X2 ( llama/31694 ) WebGPU Added batched tensor API support and returned maxBufferSize from get_max_size ( llama/31473 , llama/31348 ) Added GROUP_NORM, MEAN, COUNT_EQUAL, and PAD_REFLECT_1D kernels ( llama/31490 , llama/31591 , llama/31578 , llama/31877 ) Supported the YaRN truncation config ( llama/30802 ) Hexagon Added Q5_0/Q5_1 support and optimized conv_2d while adding conv_3d ( llama/32094 , llama/31814 ) Fused post-norm residual add into RMS_NORM_MUL and added set_tensor_n ( llama/31822 , llama/31744 ) Added automatic libhtp loading and fixed 64-bit FastRPC handle truncation ( llama/31781 , llama/30952 ) Fixed the host build with clang-cl on Windows ARM64 ( llama/32011 ) CPU Switched MUL_MAT to F32 src1 and accumulate FlashAttention V in F32 ( llama/31854 , llama/29810 ) Added SpacemiT GATED_DELTA_NET RVV and Q4_1 32x256 HP kernels and enabled IME2 MXFP4 repack ( llama/31897 , llama/31929 , llama/31776 ) Enabled the 4x4 simd_gemm tile on ARM NEON and added bf16/q8_0 VXE to llamafile ( llama/32128 , llama/31866 ) CANN Fail early when a buffer type is unsupported ( llama/31270 ) ZDNN Accept non-zdnn tensors, invalidate compute tensors after each op, and fix release backend lib detection ( llama/31824 , llama/31905 ) OpenVINO Supported the YaRN truncation config ( llama/30802 ) More info Releases and versioning of ggml-org projects Changelog since v0.29.0 417a024 ggml : bump version to 1.0.0 ( #1690 ) 771b3d9 requirements : fix numpy for python 3.14 ( #1691 ) f154959 sync : llama.cpp ec04faf fix(hexagon): make host build work with clang-cl on Windows ARM64 (llama/32011) b45738e examples : adapt to removal of ggml_diag_mask_inf 1514439 sync : llama.cpp 5f29555 ggml : prune (llama/29801) 076861c hexagon: add Q5_0 and Q5_1 support (llama/32094) 63ec52d Title: CUDA: few-row MMQ for BS 2-24 (llama/31533) c6fe796 ggml : route GATED_DELTA_NET to the SpacemiT RVV kernel (llama/31897) 4ae6e72 vulkan : Intel FA kernel for prefill. (llama/29357) b22071c ggml-cpu: use the 4x4 simd_gemm tile on ARM NEON (llama/32128) ba8b013 CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints (llama/26048) 05f605a ggml-backend : prefetch only inputs computed by earlier splits (llama/32100) b8a5b87 ggml : dispatch SpacemiT Q4_1 32x256 through the HP kernel (llama/31929) 7a8cef5 sycl: make oneDNN FA cache thread-safe and memoize runtime failures (llama/31138) 6dfea1a sycl : fix AOT for more than 4GB (llama/31873) 012ff44 sycl : use cached device-id in set_main_device (llama/32013) 2f9d47b CUDA: add cuda debug buffer pool (llama/31919) 522d257 opencl: add bin kernel kernel_gemm_noshuffle_q4_0_f32_32b_trans_ila_a7_bin (llama/30947) 0f9dcb7 CUDA: skip empty MoE tiles in MMQ stream-k fixup (llama/32067) 6bed9a8 opencl: add decomposed FA for Adreno (llama/32046) a357e7b metal : prevent out-of-