Research & AlgorithmsDANIEL SCHMITTER · RESEARCH & ENGINEERING

ML systems · Research & Algorithms

4.2× faster spline-layer evaluation through cardinal structure

A spline edge may store hundreds of coefficients while needing only four for one prediction. Turning that locality into GPU speed requires choosing the right computation—not just counting fewer operations.

Daniel Schmitter · Research story · Revised draft

Read the paper ↗Paper PDF
EXPLORE THE IDEA

Four taps, not a full grid

The measured gain appears when the grid gets large.

LOCAL SUPPORTOne query touches four coefficients32 shown · 512 stored per edgeActive location changes; support size does notARCHIVED MPS FORWARD PASSLarge-grid measurementDense explicit cardinal5.435 msStreamed local taps1.286 ms4.23×at this workloadSmall-grid losses remain in the result chart
50%
The coefficient-strip interaction is schematic. The 512-coefficient forward timings, 5.435 ms dense and 1.286 ms streamed, are archived MPS measurements for batch 1024, 32 inputs and 64 outputs. The smaller-grid route can lose; the full result plot below retains that crossover.

Follow the information

From input to outcome

The same four weights for an input are reused across output channels. The gather reads a coefficient bank; it does not build the full dense basis tensor. The archived backward benchmark updates coefficients only, not inputs.

Scroll the diagram horizontally to follow the route. Keyboard: focus the diagram, then use the arrow keys.

Input batch → Find active cell → Four cubic weights → Gather coefficients → Layer outputs. The same four weights for an input are reused across output channels. The gather reads a coefficient bank; it does not build the full dense basis tensor. The archived backward benchmark updates coefficients only, not inputs.
Information-flow map. One cardinal layer; dense evaluation can win on smaller grids. Original vector schematic based on the method and evidence discussed in this article; signal shapes and icons are illustrative, not additional measurements. Open full-size diagram ↗

Read the main route from left to right; labelled side branches show additional inputs, checks or feedback. The sections below explain the operations and their experimental limits.

The model is large. The question is local.

Imagine a learned response curve with 512 coefficients. A particular input does not consult every part of the curve. For a cubic cardinal spline, only four neighboring coefficients influence that prediction. That is an unusually concrete opportunity for a neural-network implementation: much of the representation is irrelevant to the current query. But skipping that work efficiently is a hardware problem as well as a mathematical one.

Inside one forward pass

An input batch does not activate an entire grid. For every input channel, the layer finds a cell and computes four weights. Every output channel then reads the four corresponding coefficients of its edge and adds their contributions. The same weight calculation is reused across those outputs. This is where cardinal structure enters the network architecture, rather than remaining an isolated polynomial identity.

The competing dense route first builds a batch-by-input-by-grid feature array. That wastes arithmetic on zeros, but exposes a regular contraction to the accelerator. The local route avoids that array and pays for indexed gathers instead. During coefficient backpropagation, many examples may contribute to the same stored coefficient. A good forward kernel is therefore not automatically a good training kernel.

Two paths through a spline layer
Two paths through a spline layer. Original scientific diagram; the stated component and information flow, not an additional experiment. Open full-size figure ↗

Four weights replace a full basis array

Inside one uniform cell, the four active basis functions are fixed cubic polynomials. A small matrix maps powers of the local coordinate into their weights. The network then gathers the relevant coefficients and sums their contributions. There is no need to run a generic knot recursion for every edge. Nor does the formula require materializing a dense array of mostly zero basis values.

yo=∑i∑q=03wq(ti) cio,ji+q−1\begin{gathered}y_o=\sum_i\sum_{q=0}^{3}w_q(t_i)\,c_{io,j_i+q-1}\end{gathered}
Each input channel selects one cell. Only four coefficients on each outgoing spline edge contribute.

Why the obvious optimization can lose

GPUs are excellent at large regular contractions. A dense basis implementation can exploit that strength, while a local implementation pays for indexed gathers and separate launches. Our streamed implementation processes one of the four taps at a time. This changes the intermediate arrays, but it does not turn every operation into one giant matrix multiplication. Small workloads can favor the ostensibly wasteful dense route.

The crossover is the result

In the archived MPS layer, the batch contains 1024 inputs, with 32 input and 64 output channels. At 512 coefficients per edge, streamed evaluation takes 1.286 ms versus 5.435 ms for dense explicit-cardinal evaluation: 4.23× faster. Including coefficient backpropagation gives 2.95×. At 16, 32, and 64 coefficients, the local route loses. That shape dependence is central to the finding, not a footnote.

Archived measurements, not animation timings. Above one favors streaming. The memory panel reports named array sizes, not peak memory. Both the wins and the small-grid losses are shown.
Archived measurements, not animation timings. Above one favors streaming. The memory panel reports named array sizes, not peak memory. Both the wins and the small-grid losses are shown.

Memory needs equally careful accounting

For the largest case, the dense basis array is 64 MiB and one gathered tap is 8 MiB. Those are sizes of particular arrays—not peak process memory or the full training footprint. Coefficients, outputs, temporary buffers, and the backward graph still exist. The benchmark also does not differentiate inputs, so its backward number is not the cost of training a complete deep KAN.

More detail without more work per query

Imagine deploying a fine-resolution learned sensor calibration or control response. The relevant gain is that more grid detail need not increase the number of coefficients touched by one input. Our large-grid measurements demonstrate that opportunity for one layer. Choosing the implementation by workload—and testing input gradients before stacking layers—is the route from that result to an efficient ML component.

A practical route to smaller learned components

The opportunity is a spline component whose implementation follows the active support of its representation. It could matter when high-resolution learned response curves appear repeatedly inside a larger model. A full training-efficiency claim still requires input gradients, peak-memory instrumentation, and end-to-end accuracy at matched work. What is established is useful: local structure has a measured large-grid payoff, and the same measurement tells us when not to use it.

Evidence & further reading

The links below distinguish the project record from foundational literature. This revised story does not add a new application-validation experiment.

  1. Consolidated research results, including constitutive edges and continual memory. Daniel Schmitter (2026). Local archive snapshot.
  2. Cardinal Exponential Splines: Part I—Theory and Filtering Algorithms. Michael Unser and Thierry Blu (2005). Primary literature.
  3. KAN: Kolmogorov–Arnold Networks. Ziming Liu et al. (2024). Primary literature.

Other posts of interest