As part of its Advancing AI launch event two weeks ago, AMD unveiled ROCm.ai, a new “developer experience” that introduces AI coding agents into AMD’s ROCm development tools for GPUs. According to AMD, ROCm.ai will dramatically lower the time and effort required to develop highly tuned and performant code. But will have an impact on the market, which is dominated by CUDA, Nvidia’s popular development environment?
ROCm, which originally stood for Radeon Open Compute, is AMD‘s open-source stack for GPU-accelerated computing. The software, which AMD launched about 10 years ago, includes core runtimes, drivers, and libraries that developers can use to write AI, HPC, and other types of applications using popular frameworks, such as PyTorch or TensorFlow.
Getting the most performance out of a GPU with an application has traditionally been a difficult task, best suited for experts who are comfortable working with low-level constructs. However, thanks to AI, that is starting to change, said AMD Senior Vice President of AI Vamsi Boppana.
“Just a few years ago, programming AMD GPUs required deep expertise and significant engineering effort,” Boppana said during last week’s event. “We’ve come a long, long way in a remarkably short period of time.”
ROCm CLI is in preview
For the past year, AMD engineers have been using AI-assisted coding tools internally with ROCm to generate GPU kernels, optimize code, debug problems, and improve performance, Bopanna said. That effort has shown surprisingly good results.
“In some cases, these AI-generated kernels are shockingly good, better than what we expected, sometimes better than the most manually tuned versions,” Bopanna said. “The first time you see this, you’re a little bit skeptical. You run more tests, you try to break it, you look for what went wrong, and then you realize this is real.”
Based on these positive results, the company developed and launched ROCm.AI last week as part of the launch of its Instinct MI400 line of GPUs and its Helios rack system.
ROCm.AI works with existing coding agents, such as Claude, Codex, Cursor, and Gemini, to help guide developers through the development process, including guiding setup and validating the ROCm environment, serving models, exposing local telemetry, and tuning workloads to get the most performance out of the GPUs.
ROCm.AI has several components, some of which are generally available and some of which are in preview. It starts with AMD Skills, the primary way that ROCm.AI interfaces with AI agents. AMD Skills, such as “serving-llms-on-instinct” and “local-ai-app-integration,” are already built into AI agents and available as plug-ins for Claude and other agents.
There is also ROCm CLI, a new command-line path install ROCm software and work with the environment. AMD says the CLI, which is in preview, is “deterministic and scriptable,” and AI agents and the ROCm Console can call it to surface state.
The ROCm Console is another new component launched with ROCm.AI. This console provides visibility into telemetry, logs, runtime status, and diagnostic context of the machine, giving developers and engineer useful data to help tune GPU performance or chase down performance problems.
AMD Instinct MI455X GPU
There’s also ROCm Hyperloom, a new AI-powered workload optimization tool that is also being launched with ROCm.AI. Hyperloom was designed to provide a fully autonomous agentic system for end-to-end inference workload optimization on AMD GPUs. AMD says it targets both host code and GPU kernels “by using AI agents and profiling tools to analyze a workload, identify bottlenecks, apply targeted optimizations, and validate both performance and correctness.”
ROCm.AI helps AI coding tools like Cursor, Claude, and Codex better understand ROCm to optimize workloads for AMD’s full line of GPUs, including Radeon and Instinct GPUs, Bopanna said. “In other words, we are making those popular coding agents into ROCm super users,” he added. “You should be able to describe the workload you want to run, the performance target you want to hit, and then let the agents help you get there.”
Bopanna shared some numbers about ROCm.AI that bear on the question of AMD’s competitiveness with Nvidia’s CUDA, which benchmark tests show are anywhere from 10% to 30% more performant on equivalently powered GPUs.
“You can actually watch on screen here. I write a GPU kernel. It sees an opportunity write a more optimized MoE group GEMM. and the result?” Bopanna asked the crowed. “ROCm.AI delivers a 38%…tokens-per-second improvement on this specific example. This is the experience we want for developers. Simple to start…and powerful to optimize.”
Market share for AI accelerators (Source: Silicon Analysts)
Bopanna also shared that ROCm.AI was able to deliver a 3.3x speedup on a DeekSeek AI model using techniques like block scale fused kernels, quick cache quantization, and expert parallelism. In general, ROCm.AI can improve AI model training performance by an average of 2.4 times through optimized kernels such as fuse flash attention, more efficient checkpointing, and advanced parallelism strategies, he said.
“Our engineers have done amazing work to unlock the capabilities of MI 455 through ROCm.AI, so I’m delighted to share some incredible results on hardware,” he said during the launch event. “Developers should not have to manually reason through every layer of model architecture, kernel selection, comms strategy, and rack-level deployment. AI is transforming this experience, and that is a very, very big shift for GPU hardware and software.”
Nvidia currently dominants the data center GPU market, with an estimated 75% to 80% of the market, versus about 7% for AMD, according to a recent analysis by Silicon Analysts. However, these numbers are not written into stone, and the recent performance advantages that AMD is showcasing with its new MI400 line of GPUs–not to mention the full Helios rack environment and ROCM.AI software–show that AMD is serious about taking share from Nvidia.
CUDA has about a 10-year headstart on ROCm, and is much more widely used and proven in millions of GPUs aro
und the world. AMD cannot match Nvidia with the full breadth of functionality that CUDA offers, even with AI.
But with the launch of ROCm.Aik, AMD is showing that it’s not afraid to leverage the recent advances in AI to help it close the gap with CUDA and chip into the performance and the ease-of-use advantages that CUDA has traditionally held over ROCm. We’ll have to wait and see how the market responds to this new information–not to mention how Nvidia itself reacts.
Editor’s note: This article originally appeared in HPCwire.
The post Can AMD Lower CUDA’s Moat with ROCm.AI? appeared first on AIwire.
