Skip to the stories
Electronics & Semiconductors

RISC-V in AI Chips: Best Suited to Control and Custom Data Paths

  • Ann Mutschler
  • 324 reads

RISC-V’s flexibility can help control accelerators and cut data-transfer latency, but custom instructions and system-level verification take substantial work.

RISC-V in AI Chips: Best Suited to Control and Custom Data Paths

RISC-V’s strongest role in AI chips may be coordinating accelerators rather than performing the heaviest inference calculations. Its open instruction set lets designers add custom operations, but splitting a workload between programmable cores and accelerators can bring costly data movement, compiler work and verification challenges.

That tension came through in a discussion among Steve Roddy, chief marketing officer at Quadric; Francisca Tan, director of product management and product engineering for Tessent at Siemens EDA; Ewald Liess, senior vice president of sales and business development at SignatureIP; and Dave Kelf, CEO of Breker. Their views differ on whether RISC-V belongs inside an AI compute path, but they point to a shared engineering constraint: a correct instruction set does not by itself make the whole chip work efficiently.


L-R: Quadric’s Roddy; Siemens EDA’s Tan; SignatureIP’s Liess; Breker’s Kelf.

Where customization fits

Liess sees the surrounding system as a practical boundary for customization. SignatureIP provides network-on-chip (NoC) and PCI Express, CXL, CHI and AMBA interfaces. In his account, customers can alter RISC-V cores and add instructions while keeping standard interfaces to reach memory and off-chip components. The NoC can then be configured to connect those components, using pre-verified IP and tools.

That approach gives architects room to tailor computation without making every connection between blocks unique. But the interconnect still has to carry the resulting traffic, and its behavior depends on how processors, accelerators and memory work together. Custom instructions solve only part of that design problem.

Roddy argues that beginning with a RISC-V processor for inference can create a difficult partition. Some operations run on a programmable core while others run on separate accelerators; data must move between them, and software has to decide where each part of the workload belongs. As the accelerator grows, Roddy says, the traffic and compiler challenge can grow with it.

His criticism rests on a distinction between workloads. A CPU is built to manage changing tasks and control flow, including many threads whose next operation may be unpredictable. Inference, Roddy argues, often has a long sequence of known operations. He says much of the CPU’s general-purpose machinery is then unnecessary for the main calculation. Digital signal processors (DSPs) are closer to the target in some respects, but their vector operations do not naturally match the matrix-by-matrix work that challenges AI hardware.

Quadric’s response, Roddy says, is a matrix-native machine. He compares the approach with Google’s large data-center TPUs, which he describes as a close parallel. His point is that a design built around matrix computation can avoid some of the partitioning burden he associates with starting from a CPU. That is his assessment of the architecture trade-off, rather than a claim that every AI workload has the same needs.

Using RISC-V as the accelerator’s controller

Kelf agrees that a general-purpose processor may be a poor choice for heavy inference, but sees a useful role for RISC-V above the accelerator’s compute blocks. He says designers have used RISC-V in four examples, at both large and small companies, to replace control state machines that proved difficult to manage.

In those systems, the processor directs several accelerators and helps coordinate their work. Designers can add vector or matrix-multiplication instructions, or operations for particular mathematical tasks. Kelf also describes custom load/store instructions intended to reduce latency as data crosses between partitions. The processor does not need to take over the main inference workload; it can handle enough computation and control to keep the accelerator busy.

One example involved a single-instruction, multiple-threads (SIMT) array. Parallel lanes sometimes diverged, leaving part of the work to be handled separately. Kelf says the designers used the RISC-V control processor for some of that divergent computation while the rest of the accelerator continued its parallel work. He describes the implementation as successful after substantial engineering effort, with the RISC-V core doing limited mathematical processing rather than heavyweight inference.

That distinction helps reconcile the competing views. A RISC-V core may add value in control, data handling or exceptional cases while a purpose-built engine performs most matrix operations. The benefit depends on whether the added instructions and data paths reduce the system’s bottlenecks enough to justify the design and software work.

Custom instructions carry tool and verification costs

Custom hardware needs a matching software toolchain. Roddy points to his 17 years at Tensilica, where instruction customization was automated: changing instructions could produce corresponding compilers and simulators, with verification supported by a structured process. He says that approach carries a commercial royalty, while RISC-V offers a way to avoid paying a processor royalty and to define more of the implementation.

That flexibility comes with work. Roddy’s view is that teams choosing RISC-V customization should be prepared to build and maintain the associated tools. For a small adjustment, he argues, an automated commercial system may be easier to use. He sees RISC-V as a strong fit for control functions that benefit from tuning, while remaining skeptical that it should perform the main inference calculations.

Kelf likewise says teams can add their own operations and use a regular compiler, but they need the knowledge to implement and support those changes. In return, he sees potential flexibility to move some proprietary instructions into a different RISC-V implementation. He also notes that commercial processors, including Arm designs, can include custom instructions, but describes their options as less variable.

Verification must reach beyond the instruction set

Tan stresses that custom instructions need observability during debug and silicon bring-up. Engineers need ways to inspect internal signals and determine whether performance targets are being met. Siemens EDA supports tracing selected custom instructions while a RISC-V processor runs, she says, alongside system-level monitoring of interconnects, buses and other signals in the system-on-chip.

Tan also points to Efficient Trace (E-Trace), a compressed instruction-trace specification that Siemens EDA donated to the RISC-V organization a few years earlier. Compression matters in complex systems because instruction and system traces can generate large amounts of data. Her emphasis is on seeing how the core behaves within the larger chip, not treating processor debug as a standalone task.

Liess makes a related point about system verification. Passing instruction-set checks does not establish that a multi-core system can deliver the data an operation needs. Caches create coherency cases, while traffic bursts across the NoC can expose deadlocks. Verification must therefore test how traffic flows through the complete system, including combinations of cores and accelerators.

For engineering teams, the practical question is where a custom RISC-V core reduces friction in the architecture. It may make sense as a control engine with carefully chosen instructions for load/store behavior or limited calculations. The larger the role assigned to it, the more attention the design needs to pay to compiler support, data movement, traceability, cache coherency and NoC behavior. The decision hinges on whether those system costs are smaller than the flexibility the custom core provides.

Source: Ann Mutschler · semiengineering.com

From the same set