Apple and NVIDIA Collaborate to Speed Up Large Language Model Performance

Apple engineers shared new details on a collaboration with NVIDIA to implement faster text generation performance with large language models using their Recurrent Drafter (ReDrafter) technique. The technique was integrated into NVIDIA's TensorRT-LLM, which helps run LLMs faster on NVIDIA GPUs. In benchmarking, the integration led to a 2.7x speed-up in generated tokens per second for greedy decoding.