Research & AlgorithmsDANIEL SCHMITTER · RESEARCH & ENGINEERING

ML systems · Research & Algorithms

Exact smoothness losses for spline networks—without sampling the integral

A neural response curve is continuous, even when training sees only samples. Its smoothness can sometimes be measured exactly over the whole domain—with one small structured matrix.

Daniel Schmitter · Research story · Revised draft

Read the paper ↗Paper PDF
EXPLORE THE IDEA

An integral becomes geometry

The same continuous curvature, written in coordinates.

THE CONTINUOUS OBJECTA neural response between samplesCurvature energy 4.303THE COEFFICIENT OBJECTOnly overlapping supports interactQ is computed; its corner bands are periodic
50%
Computed periodic cubic curvature Gram for eight basis functions, integrated with four-point Gauss quadrature per cell (exact for these products up to rounding). The curve and quadratic energy use the displayed Gram matrix. Unit cell spacing; not a neural training-speed benchmark.

Follow the information

From input to outcome

This is the regularization branch of a learner. It measures the represented continuous function, not a new set of sampled residuals. R must be rebuilt if the basis or integration geometry changes.

Scroll the diagram horizontally to follow the route. Keyboard: focus the diagram, then use the arrow keys.

Spline coefficients c → Apply fixed Gram R → Quadratic energy → Coefficient gradient → Parameter update. This is the regularization branch of a learner. It measures the represented continuous function, not a new set of sampled residuals. R must be rebuilt if the basis or integration geometry changes.
Information-flow map. Local support gives sparsity; circulant structure needs compatible periodic geometry. Original vector schematic based on the method and evidence discussed in this article; signal shapes and icons are illustrative, not additional measurements. Open full-size diagram ↗

Read the main route from left to right; labelled side branches show additional inputs, checks or feedback. The sections below explain the operations and their experimental limits.

What happens between the training points?

A model can fit all observed values and still oscillate between them. A smoothness loss discourages that behavior, but evaluating it at another collection of points creates another sampling problem. For a spline edge with fixed coordinates, the integral itself has an exact finite representation. The model already carries the information needed to compute it.

A loss branch that sees the whole function

Consider a spline edge inside a learned dynamics model. Its training samples may leave gaps, yet a curvature penalty is meant to control behavior between those samples too. The architecture has a second branch from the coefficient array to a precomputed derivative Gram and then to a scalar energy. It shares parameters with the prediction branch but needs no new sampled inputs.

Backpropagation through this branch is a structured matrix action. For local cubic functions, distant supports do not interact. On a compatible periodic uniform grid, translation invariance provides additional structure. Neither property implies that the empirical data Gram is circulant: its rows depend on where the observations occurred.

Two paths through a spline layer
Two paths through a spline layer. Original scientific diagram; the stated component and information flow, not an additional experiment. Open full-size figure ↗

Move the integral into the coordinates

Write the curve as a weighted sum of known basis functions. Differentiate those functions, expand the squared derivative, and integrate each pair once. The resulting matrix records how the basis derivatives overlap. Every later loss evaluation is a quadratic form in the current coefficients. Its gradient is a matrix-vector product, without drawing new points to estimate the integral.

∫Ω ⁣[fc′′(x)]2dx=cTQc,Qij=∫Ωϕi′′(x)ϕj′′(x)dx\begin{gathered}\begin{gathered}\int_\Omega\!\big[f_c''(x)\big]^2dx=c^TQc,\\Q_{ij}=\int_\Omega\phi_i''(x)\phi_j''(x)dx\end{gathered}\end{gathered}
For a fixed basis, continuous curvature is a quadratic form. Its coefficient gradient is 2Qc.

Why the matrix is small in practice

Local support means distant basis functions do not overlap, so their inner products are zero. A cubic mass matrix needs only seven neighboring taps on a sufficiently large periodic grid. Uniform spacing and periodic boundaries also make the matrix circulant, allowing Fourier-domain inverse actions. Applying a short stencil can still be cheaper than using an FFT; the right algorithm depends on whether we need a product or a solve.

Exact means exact for the represented function

This calculation measures a specified spline on a specified domain. It does not reveal whether the spline matches an unknown true function. It also does not make an arbitrary empirical neural Gram matrix circulant. Nonuniform weights, changing coordinates, and nonperiodic boundaries can remove that symmetry. If the basis changes during learning, the assembled matrix may no longer describe the intended energy.

What we have actually checked

Archived periodic cubic calculations agree between compact and Fourier implementations near double-precision roundoff. A bounded four-point-per-cell quadrature check agrees with the compact inner product to relative error 2.86 × 10⁻¹⁶. Separately, the grid-growth study finds a modest paired benefit from a continuous curvature penalty on a noisy high-frequency target. We have not measured a universal “10× faster neural loss” result.

Make the loss match the physical function

The practical idea is to stop estimating a known continuous loss anew at every training step. This can make regularization deterministic and tied to physical units. The archive verifies the arithmetic, but does not measure a tenfold end-to-end neural training improvement. A compelling next deployment would measure this loss branch in a real optimization workload, including assembly when the basis changes.

An optimizer should not have to relearn geometry

The broader idea distinguishes a trainable function from its known geometry. Coefficients change; the meaning of a derivative energy need not. That separation can make regularization deterministic, preserve the same penalty when a model grows, and expose efficient linear algebra. It gives ML engineers a precise computational component—not a promise that smoothness always improves generalization.

Evidence & further reading

The links below distinguish the project record from foundational literature. This revised story does not add a new application-validation experiment.

  1. An Inner-Product Calculus for Periodic Functions and Curves. Anaïs Badoual, Daniel Schmitter and Michael Unser (2016). Primary literature.
  2. Consolidated research results, including constitutive edges and continual memory. Daniel Schmitter (2026). Local archive snapshot.
  3. Structure-preservation audit. Spline research archive (2026). Local archive snapshot.
  4. Operator-spline theory: consolidated research manuscript. Daniel Schmitter (2026). Local archive snapshot.

Other posts of interest