Revisable physical memory with bounded inference error
Historical source. Some claims in older records were subsequently corrected. The associated article states the adopted interpretation. This record preserves the original source alongside its rendered reading view.
Rendered archival Markdown
This reading view preserves headings, tables, lists, code fragments and mathematical notation from the local research record.
Revisable physical memory with bounded inference error
Executive assessment
A useful compression system need not recover its input waveform. It can instead preserve a declared family of future computations. The practical vision is an instrument that discards most of a long measurement record while retaining the ability to reconsider physical parameters and sensor calibration later. The important restriction is that later questions must remain inside an explicit contract. Neither a small file nor accurate answers to a few selected queries establishes general information recovery.
The completed physical-inference panel demonstrates a narrow version of this capability. A roughly 12.6 kB record of lag products, boundary samples and a sum supports later inference about a stationary noisy stochastic oscillator. Across 48 synthetic records and two calibration choices, it agrees closely with a compiled exact Kalman reference on the fixed 1,271-point physical-parameter grid. Its retained size is at least 40 times smaller than the tested lossless raw representation for the longer records. It is slower end to end on the shorter records. A sufficiently dense ordinary parameter bank also passes the accuracy gate and is faster for warm queries.
The main conceptual development is an answer-error contract. A finite-memory approximation is related to the exact Gaussian likelihood by a model-dependent forgetting bound. The capsule's energy then turns that relation into a bound for the particular observed record, rather than only an average over possible records. A separate numerical audit tested whether capture and query roundoff can be enclosed as well. That audit now passes all 96 cases, with largest posterior-TV bound 0.000297, and independent exact-statistic and full-record reference checks pass. Its approximately 66-minute query cost is substantial and must remain separate from ordinary inference timings.
A subsequent streaming implementation removes the full-record capture workspace. It retains about 12.7 kB using 144,016 bytes of native live state. All eight million-sample record/calibration cases meet the prescribed bound, but six of eight four-million-sample cases do not. The fixed configuration's full scale-up gate therefore fails. Its observed reference agreement remains close, but the error budget and coarse posterior grid prevent an unlimited- duration or resolved continuous-inference claim.
These findings do not establish a general compression breakthrough, a novel theory of sufficient statistics, or a spline-specific advantage. Gaussian message composition, finite-conditioning approximations, time-series statistics and Bayesian data compression all have substantial precedents. The defensible question is whether their particular combination yields a useful, auditable physical-memory capability at an acceptable total cost.
The capability and its limits
Conventional lossless compression promises to reconstruct every input bit. Lossy waveform compression promises a particular distortion measure. The object studied here makes a different promise: answer later likelihood and posterior queries for a stated observation model, with a stated error budget. It is closer to a compressed scientific record than to an audio codec.
Three aspects make this more demanding than retaining a fitted parameter. The physical parameters need not be selected during capture. Detector gain and offset can change between the original and revised query. The answer includes the relative likelihood across competing physical hypotheses, not merely the location of one optimum. Different priors on the same hypothesis set can therefore be applied after capture.
This freedom is limited. The present representation assumes scalar, regularly sampled observations of a stationary linear Gaussian system and known post-gain independent observation-noise variance. It does not retain arbitrary time-local events, permit unrestricted changes to the observation process, or support a new nonlinear model merely because that model also describes physics. A posterior close to the full-data posterior can still be scientifically wrong when the likelihood is misspecified.
The selected application regime is constrained retention or transmission followed by later analysis. It is not automatically the fastest way to analyze a short recording already available on a workstation. Persistent bytes, live capture memory, capture time, query setup, warm query time and numerical assurance cost are different quantities and must be reported separately.
Prior-art and equivalence gate
The first candidate stored Gaussian factors over segment endpoints and composed them by eliminating interior states. This is useful algebra, but associative Gaussian filtering and smoothing already provide the essential construction. Sequential and tree-based factor composition cannot be claimed as a new spline-specific algorithm merely because an operator formulation motivates it.[^1]
The corresponding physical-parameter bank also has an elementary equivalence: transforming ordinary nodal likelihood values into Chebyshev coefficients does not create additional information. It changes a representation and its evaluation procedure. The verification panel checked Gaussian factors, composition order, dense references, later boundary priors and this bank equivalence. All 1,440 checks passed; the largest normalized discrepancy was 7.26e-12. The generic factor-bank novelty claim was closed.
More sophisticated precedents also matter. Structured Gaussian-process methods such as celerite make exact inference practical for relevant oscillatory covariances; a dense cubic-cost Gaussian-process baseline would therefore exaggerate the candidate's benefit.[^2] Family-wide Fourier preprocessing accelerates later queries within a specified kernel family.[^3] Relative binning provides another route, using a local reference for waveform likelihood calculations.[^4] Score compression preserves local information near a fiducial model; it should not be confused with a uniform posterior guarantee over arbitrary future models.[^5]
A particularly close recent precedent is fixed-size time-series compression using lag statistics and boundary information for ARMA-related inference. Paulin and Elvira's analysis includes logarithmically growing truncation order under stated assumptions. Their predictor-recovery result and the present per-record likelihood enclosure are different contracts, but that distinction does not establish priority for every ingredient of this construction.[^6]
Bayesian data compression already frames compression in terms of information retained about a posterior. Its response, noise and prior metadata are part of the representation and cannot be ignored in a storage comparison.[^7] Consequently, the broad slogan "compression that preserves inference" is not a novelty claim available to this project. A publication would need to identify a new guarantee, a genuinely different operating regime, or a validated practical advantage over these established ideas.
Deterministic Gaussian likelihood bounds are also established. Conjugate- gradient lower bounds combine determinant inequalities with a computable residual bound for the data-dependent quadratic term, and use that bound to control the stopping error.[^16] The present distinction is not the existence of a likelihood certificate. It is whether a useful certificate can be computed from a small retained record after the interior observations are unavailable. No matched performance comparison with that method is claimed.
Physical model and retained representation
The latent state contains oscillator position and velocity. Its continuous drift matrix is A=(0−ω21−2γ). The stationary covariance is S=diag(1,ω2), the sampled transition is F=exp(AΔt), and process covariance is Q=S−FSFT. An observation is yt=gxt,1+b+ϵt, with independent Gaussian noise of variance R=0.04 added after detector gain. This order is important: moving the noise through the gain changes the likelihood. An explicit negative control produced discrepancies as large as 42.758 log-likelihood units when the observation semantics were changed incorrectly.
The stationary normalization is a substantive physical assumption. For a white-noise forcing amplitude σ, stationary Lyapunov balance gives Var(x1)=σ2/(4γω2). Fixing that variance to one therefore sets σ2=4γω2 throughout this family. It is not automatically a thermal instrument at fixed temperature and mass. If detector gain and forcing amplitude were both unknown, their product would be observable but the two factors would not be separately identifiable from this channel alone. The two tested calibration choices are specified alternative observation hypotheses, not a solution to that identifiability problem. Frequency is angular frequency in the declared model units.
The retained numerical statistics are Ck=∑t=kN−1ytyt−k for 0≤k≤L, the first and last L observations, the total sum, sample count and observation metadata. The panel fixes L=512. Interior samples are discarded from the candidate representation. The original implementation uses blocked FFT calculations and merge identities; the numerical-enclosure implementation separately recaptures the statistics using controlled pairwise arithmetic.
For a queried physical model, covariance lags determine the best linear predictor from the previous L observations. The first L observations are handled with their exact joint Gaussian likelihood. Subsequent conditional terms use the same finite predictor. Expanding the squared residuals gives lag-product terms plus corrections from the retained beginning and end. Offset changes are handled by the retained linear sum and corresponding boundary algebra. The capsule therefore contains enough information to evaluate this finite-conditioning likelihood without rereading its interior.
This is a chronological Vecchia approximation, not an assertion that the noisy observed oscillator is exactly an order-L Markov process. The broader Vecchia literature provides the appropriate conditional-factorization context.[^8] Finite support of a driving filter, sparse continuous operators and finite conditional memory are not interchangeable properties.
From approximation to an answer-error contract
Let Pθ denote the exact length-N Gaussian observation distribution and Qθ,L the finite-conditioning distribution. Write vj for the prediction variance after conditioning on j previous observations. Stationarity and exact finite conditional factors give DKL(Pθ∥Qθ,L)=21∑t=L+1Nlog(vL/vt−1). If v∞ is the steady prediction variance, an upper bound is Bθ=21(N−L)log(vL/v∞).
A uniform upper bound over a declared parameter family can control prior-predictive expected posterior error. That is useful for a memory-budget screen, but it is not a guarantee for each realized record. In particular, an average divergence statement cannot simply be relabeled an observed-data posterior certificate.
The second step uses covariance structure to obtain a pathwise bound. With exact conditional factors, the relative precision matrix has trace N, and the log-determinant difference is 2DKL. A conservative relative spectral deviation bound is δ(B)=2B+2B(B+1). Because the exact covariance is at least RI, the centered record energy Eb=∑t(yt−b)2 bounds the relevant quadratic form. Thus −B−δ(B)Eb/(2R)≤logQθ,L(y)−logPθ(y)≤δ(B)Eb/(2R). The capsule retains the quantities needed to compute Eb.
Likelihood error matters up to an additive constant shared across hypotheses. If its range has width Ω, the total-variation distance between the normalized posteriors is at most tanh(Ω/4) for any shared prior on that hypothesis set. General posterior sensitivity to likelihood perturbation is established mathematics; the project derives the required specialization and records its total-variation convention explicitly.[^9]
Obtaining a reliable tiny B by subtracting nearly equal floating prediction variances is unsafe. Instead, let P@@LOCALHTML0@@=F(I−P@@LOCALHTML1@@LS(A∗T)LHT. This bound avoids cancellation in vL−v∞. It also exposes the mechanism: stable prediction forgets increasingly distant observations.
Under uniform geometric forgetting and ordinary energy growth, a sufficient lag grows logarithmically with recording length and inverse error tolerance. That is an asymptotic upper-bound statement under assumptions, not a lower bound, proof of optimality, or promise that a fixed 512-lag capsule works for arbitrarily long recordings. Near weakly damped or poorly observed systems, the constants and conditioning can be unfavorable.
The number of real-valued statistics is not an information-theoretic bit rate. Stored magnitudes and numerical precision also matter. All reported objects include their actual binary64 payload, uncertainty radii and metadata; the supported input domain and record length are finite. The asymptotic lag argument does not promise a fixed number of 64-bit words for an unlimited stream, and it is not an optimal rate-distortion theorem.
Frozen experiment and complete comparator results
The confirmation panel uses 12 named independent seed identities, each at two sampling intervals, 0.01 and 0.05, and two lengths, 4,096 and 65,536. The shorter record is a prefix of the longer one within a sampling interval. Those paired lengths are not independent replications. Sampling intervals use separate child random streams. The resulting 48 records are synthetic mechanism tests, not 48 independent physical experiments.
The true generating parameters are frequency 3.1, damping 0.43, gain 1.05 and offset 0.1. Queries use a fixed grid of 41 frequencies and 31 dampings. The original calibration is gain 0.9 and offset zero; the revised calibration is gain 1.05 and offset 0.1. Uniform and biased priors are applied to the same grid. Agreement is with that discrete posterior, not with a continuous parameter posterior; the generating damping is not a grid point.
The low-storage candidate is compared with compiled exact Kalman evaluation, raw evaluation of the same finite-conditioning approximation, lossless zlib storage, and ordinary Chebyshev parameter banks with 8 or 16 nodes per axis over frequency, damping and gain. Offset dependence is handled analytically for the banks as well. No model, lag or bank-degree search was performed on the confirmation results.
Representation
Combined accuracy gate
Retained bytes
Maximum posterior TV
Lag capsule, L=512
96 of 96 cases pass
12,584-12,587
1.705e-8
Ordinary bank, 8 nodes per axis
0 of 96 cases pass
16,695-16,696
0.1720
Ordinary bank, 16 nodes per axis
96 of 96 cases pass
131,384-131,385
4.863e-5
The combined gate requires likelihood-error oscillation no greater than 0.04 and posterior TV no greater than 0.01 under both declared priors. The lag capsule's largest observed oscillation is 1.408e-7; the 16-node bank's is 0.0015471. Agreement with independently replayed finite-conditioning statistics has maximum normalized discrepancy 2.949e-13. All 200 saved physical objects across development and confirmation passed hash, byte and round-trip verification.
End-to-end timing includes capture, serialization, decoding, preparation and both full query grids. At length 4,096, the lag approach takes approximately 1.27-1.29 seconds, compared with 0.126-0.127 seconds for the raw reference and 0.362-0.371 seconds for the accurate bank. At length 65,536, it takes 1.26-1.32 seconds, compared with 2.00-2.08 seconds for the raw reference and 3.37-3.43 seconds for the accurate bank. These are workload-specific prototype timings, not universal speed ratios.
The lag capsule is at least 40.03 times smaller than lossless raw storage on the longer records, but only about 2.51 times smaller on the shorter ones. The accurate bank has faster warm queries. Some model-only bank quantities could be shared or regenerated, and the exact reference could use additional steady-state optimizations. Neither omitted optimization is credited as a measured result. Peak process RSS in this panel was about 141 MiB.
There is an even smaller control when every future query is known in advance. One binary64 likelihood value at each of the 1,271 hypotheses for both fixed calibrations would use 20,336 payload bytes before metadata and would support later priors on that grid exactly. This is a storage calculation, not a newly measured implementation. It would move the inference work into capture and would not support unlisted physical or calibration queries. The lag capsule's case therefore rests on deferred query flexibility and capture cost, not on the false premise that all alternatives must retain raw observations.
Mathematical and numerical assurance
The data-free information screen covered 750 combinations of physical query, sampling interval and record length through 1,048,576 observations. Every case met its declared information-budget condition at the maximum lag, and the largest selected lag in the screen was 195. This was a feasibility screen, not permission to select a favorable lag on confirmation data.
The capsule-only pathwise assessment subsequently covered all 96 frozen record/calibration cases. Its largest numerical posterior-TV envelope was 1.67726e-5, below 0.01. No raw recording or saved raw likelihood was consulted to compute those envelopes. Nevertheless, ordinary floating Riccati solvers, FFT statistics and likelihood arithmetic meant that this stage established a mathematical bound evaluated numerically, not an end-to-end machine certificate.
The separate numerical-enclosure study uses controlled binary64 pairwise products and sums, with rounding bounds based on tree depth. Pairwise summation and its error analysis are classical; their use here is numerical accounting rather than a new compression principle.[^10] Stored objects include uncertainty radii and an integrity hash. Exact integer arithmetic on binary64 inputs is reserved for independent verification of the captured statistics.
Model calculations use 80-decimal-digit interval arithmetic. Proposed Riccati solutions are accepted only when matrix subsolution and supersolution inequalities verify. The finite predictor's error is bounded using its Toeplitz-system residual and the known observation-noise lower bound. Boundary terms, finite-prefix likelihood, normalizers and centered energy are enclosed before combining arithmetic and truncation error.
An initial interval-prefix implementation failed because repeated interval propagation lost a useful positive covariance enclosure. That failure is preserved. The development correction uses verified matrix-order brackets at each prefix step, not an arbitrary posterior error inflation. The corrected eight-record development panel passes, with maximum TV envelope 0.000273, and its 45 unit tests pass. The subsequent frozen full-cohort audit passes all 96 cases, with maximum posterior-TV bound 0.0002971093. The retained objects use 12,695-12,700 bytes including radii, metadata and integrity digest.
Independent verification checks all 24,624 captured lag statistics against exact integer arithmetic on the original binary64 observations. All pass, as do energy, sums, exact boundaries, metadata, source hashes and object round trips. All 122,016 floating full-record likelihood references and their finite-conditioning counterparts lie within their respective intervals. The largest measured posterior TV is 1.7124e-8, and the largest exact- likelihood interval width is 0.00118844. Reference containment is a strong cross-check, not an independently formalized solver proof.
The full query phase takes 3,954.83 seconds: approximately 2,657 seconds of model preparation, 1,170 seconds of prefix calculations and 125.5 seconds of capsule-tail work, plus overhead. It reuses prefix work for paired lengths and reads no raw observations. Independent verification takes 56.05 seconds. Final peak process RSS is 124.25 MiB. The assurance layer is therefore much more expensive than the ordinary calculations in the earlier comparison.
The numerical trust boundary must remain explicit even if the audit passes. The implementation relies on IEEE arithmetic assumptions, compiler settings and the basic interval operations of mpmath, whose documentation describes the interval implementation as experimental.[^11] It is not a formally verified proof assistant. The contract concerns exact normalization of the published likelihood values on the declared grid; it does not independently certify a downstream floating softmax implementation or a continuous-prior integral.
Study 05's controlled capture materializes the input and pairwise scratch space proportional to its length. Its roughly 12.7 kB retained object is not a 12.7 kB live-memory implementation. The separately frozen streaming study addresses that acquisition boundary, while retaining the same query arithmetic and error tolerance. Its results follow.
Bounded live capture and the long-record boundary
The streaming implementation maintains a ring buffer, exact initial boundary and a binary counter of balanced partial sums. It consumes at most 4,096 observations per input block. Finalizing occupied counter levels from smallest to largest preserves the required reduction-depth bound. This is classical balanced summation, not a new compression primitive. A fixed 32-level counter supports the declared domain N<231, rather than an unlimited stream.
Development checks include exact-integer lag statistics, boundary preservation, chunk-partition invariance, snapshot/continue behavior and rejection before state mutation. A last-bit source-reproduction discrepancy was traced to compile-time rather than runtime physical parameters and corrected without weakening the exact-equality test. All development checks passed before the long-stream cohort was admitted.
The frozen panel uses two new seed identities, each with sampling intervals 0.01 and 0.05 and paired prefixes of 1,048,576 and 4,194,304 observations. It keeps the lag, model, 1,271-point query grid, calibration choices and priors unchanged. Four streams yield eight prefixes, not eight independent physical replications. Exact small-state floating Kalman references are accumulated while observations are available; no full raw array or recording is saved. Capsule-only querying and reference verification run as separate phases.
Observations
Sampling interval
Bound pass cases
Largest TV bound
1,048,576
0.01
4/4
0.00444167
1,048,576
0.05
4/4
0.00304670
4,194,304
0.01
0/4
0.01843085
4,194,304
0.05
2/4
0.01220511
All million-sample cases pass. At the longer prefix, both original-calibration cases at interval 0.05 pass, while the other six cases are rejected. The first tested rejected length is 4,194,304; intermediate lengths were not searched. The complete joint gate fails, and no lag, precision, hypothesis or tolerance is changed to rescue it.
All 20,336 full-record floating likelihood references lie inside their intervals. All 32 posterior comparisons remain within their corresponding bounds, with largest measured TV 5.37358×10−6. This close agreement does not override the prescribed rejection. On every rejected case, the finite-conditioning likelihood's numerical enclosure alone is too wide for the chosen posterior budget; physical truncation also contributes in some cases. This is a limit of the current conservative implementation, not proof that the actual error is that large or that no other method can succeed.
Native accumulator allocation stays at 144,016 bytes, and saved capsules occupy 12,700-12,703 bytes. Capture peak RSS is 117.20 MiB and query peak RSS 115.42 MiB. All resource gates pass. Counting each stream only once through its longest prefix, candidate append work takes 6.127 seconds. Source generation takes 0.399 seconds and online reference work 485.206 seconds; the total capture phase takes 491.895 seconds. The reference maintains extra statistics and is not an optimized minimal answer bank.
The numerical query phase takes 2,909.876 seconds, about 48.5 minutes: 2,688.442 seconds of model and prefix-schedule preparation, 199.384 seconds of record-prefix work and 21.701 seconds of capsule-tail work, plus overhead. These measurements include shared model work and paired-prefix reuse. They do not establish embedded battery savings or a universal speed advantage.
An uncompressed binary64 million-sample payload would be roughly 660 times the capsule size. The four-million-sample payload would be roughly 2,642 times its capsule size, but the latter scale fails the full answer gate. These are payload arithmetic comparisons, not newly measured lossless-codec results. The earlier panel remains the measured lossless and ordinary-bank comparison. A query snapshot also does not serialize the complete live counter needed for power-loss recovery, nor permit arbitrary interior edits.
The predeclared resolution diagnostics expose an important scientific limit. Across all 32 combinations, effective support is only 1.0-1.854 grid points. At the longer interval-0.05 prefix, maximum posterior mass is between 0.999994657 and 1.0. The generating damping 0.43 is absent from the grid. Consequently, very small discrete-posterior TV is not evidence of correctly resolved continuous uncertainty; zero boundary-grid mass does not fix this.
The specified calibration revision changes posterior mean frequency from about 3.15 to 3.10 and damping from approximately 0.560-0.580 to 0.420-0.440 across the reported cases. The retained representation reproduces that model-dependent reinterpretation closely after interior observations are discarded. These are descriptive paired outputs, not a newly selected threshold success or an identifiable thermal-calibration experiment.
All 61 campaign tests and 12 subtests pass, including the four multichannel algebra checks. A reporting-key collision was corrected before the grid diagnostics ran, without changing their numerical calculations. The saved- archive integrity audit also passes while preserving the failed numerical gate. No additional physical-data fit is admitted by these results.
A caution from the operator-spline literature
Operator-based spline theory supplies useful connections between continuous stochastic systems, localized innovations and discrete representations. It does not remove the need to distinguish marginal dependence from conditional dependence. A detailed rereading of the generalized-innovation paper identified a problem with the finite-conditional-memory identity displayed in its Property 4.[^12]
For the second derivative driven by Gaussian white noise, the corresponding triangular compactly supported increment kernel yields autocovariances c0=2/3, c1=1/6 and c2=0. Yet the conditional covariance between increments two steps apart, given the intervening increment, is −1/24. Thus compact support and zero distant marginal covariance do not imply the displayed finite-history conditional identity. This elementary counterexample is recorded with exact and numerical tests.
This does not invalidate the whole paper or its operator framework, and it is not a newly discovered general phenomenon in time-series theory. It does prevent an unjustified exact-memory claim in this project. The current finite-conditioning method includes approximation error precisely because the observed process need not have finite conditional memory.
Application qualification
Optical-trap calibration is a coherent application lead because stiffness, damping and detector response have concrete scientific meanings. It also illustrates why a strong simple comparator is indispensable. Directly observed Ornstein-Uhlenbeck calibration already admits a likelihood described by four quadratic statistics. The relevant literature discusses this compact description together with experimental drift and noise limitations.[^13] A 12.7 kB capsule is not progress when four statistics answer the actual question exactly.
Finite exposure changes observation covariance and can make the measured sequence non-Markovian. Published work derives this covariance and uses a one-step approximate likelihood for a compact estimator.[^14] That suggests a possible comparison involving later detector-response or noise revision, but the physical model must be the same for all methods. Comparing a correctly modeled candidate with an intentionally misspecified compact baseline would confuse improved modeling with improved compression.
The scientific endpoint would need to distinguish physical estimation error, uncertainty, model inadequacy and compression-induced error. The cited 2023 dataset is available on request rather than publicly provided; no author contact or data acquisition has taken place. No real-data panel is admitted on the strength of an attractive application story.
Mechanical condition monitoring is another plausible destination, but a stationary Gaussian oscillator is not a general model of machinery. Impacts, changing speed, external forcing, drift and non-Gaussian disturbances can dominate a real maintenance decision. A claim about detecting meaningful damping or resonance changes requires physical evidence and an appropriate decision threshold, not only agreement with a synthetic likelihood.
A possible unifying framework
The broader opportunity is a scientific memory format with explicit query rights. An acquisition contract specifies the observation semantics and admissible model family. A structured representation retains the calculations needed by that family. A later query returns both a numerical answer and an error budget, and the system abstains when that budget is insufficient. This is a possible organizing framework, not a general-purpose compiler already implemented by the present code.
The valuable freedom is to revise a scientific interpretation after interior measurements have been discarded, without pretending that every future interpretation remains available. Raw waveform reconstruction, arbitrary interior corrections, unknown observation mechanisms and changing a model outside the declared family are different capabilities. A practical format must expose those exclusions as clearly as the answers it can return.
The real-arithmetic derivation extends to vector observations by retaining oriented cross-channel lag matrices and boundary vectors. For an observation dimension d, its storage is approximately O(Ld2), not independent of sensor count. The determinant and conditional-innovation argument generalizes, but a validated multichannel query engine, numerical matrix inequalities and resource measurements have not been established. Existing multivariate time-series compression remains a close comparator; merely using several channels does not create novelty.
There is a concrete link to the neuromorphic sensing vision. Published work uses event-based sensing and a PC/FPGA/filter loop to cool three uncoupled levitated particles, while identifying filtering and feedback-noise effects as limitations.[^15] A future scientific-memory question could be whether later calibration or feedback-model revision remains trustworthy after data discard. That would require synchronized control records and a noise model appropriate to feedback, not the independent-noise model imported unchanged. No such application has been run here.
The associated public archive can be inspected selectively. Its directory contains 1,289 CSV files and acquisition notes, while the whole ZIP exceeds 2 GB.[^17] A metadata-only review identifies tracking-column definitions and record-to-feedback-condition mappings. It does not establish complete control provenance or calibrated physical observations. File counts are not independent trials, and no measurement CSV has been opened or fitted in this campaign.
The decisive demonstration would pair a useful deferred scientific decision with a constrained acquisition budget, qualified physical observations and strong compact alternatives. It must show that reanalysis preserves the decision and its uncertainty at lower total resource cost. Better waveform error, a tiny object, or more model parameters alone would not meet that standard. The present work establishes components of that route and tests their boundaries; it does not establish the final instrument-level benefit.
Research decision
The strongest established result is a compact, parameter-revisable physical record with very accurate tested inference and an explicit mathematical path to per-record answer bounds. The major negative results are equally clear: the generic Gaussian-factor bank is not a new method, the small ordinary bank fails the accuracy gate, the larger bank remains a strong competitor, and short-record speed does not favor the candidate.
The bounded scale-up is closed. It establishes fixed live acquisition memory over the tested lengths and passing answer bounds at one million observations, but the same configuration fails its full four-million-observation budget. Further tuning of the revealed cohort is not the next research decision. A flagship application requires qualified measurements and a scientific decision that simpler sufficient-statistic alternatives cannot already answer. The particle archive supplies a concrete source lead, not that qualification or a physical result. No spline component should be added merely to make the work resemble the project's origin.
A practical decision contract can be stated without inventing a new score. If posterior TV is at most ϵ, every declared event probability changes by at most ϵ. A threshold decision is protected when its probability margin exceeds that bound; otherwise the compressed answer can abstain. Expectations of a quantity ranging over [a,b] change by at most (b−a)ϵ. These are standard consequences of total variation, not evidence that the underlying physical model or chosen decision is correct.
Sources
[^1]: Särkkä and García-Fernández, Temporal Parallelization of Bayesian Smoothers, revised manuscript (2020). Primary manuscript. [^2]: Foreman-Mackey et al., Fast and Scalable Gaussian Process Modeling with Applications to Astronomical Time Series, Astronomical Journal (2017). Primary manuscript. [^3]: Greengard, Efficient Fourier Representations of Families of Gaussian Processes, revised manuscript (2024). Primary manuscript. [^4]: Zackay, Dai and Venumadhav, Relative Binning and Fast Likelihood Evaluation for Gravitational Wave Parameter Estimation (2018). Primary manuscript. [^5]: Alsing and Wandelt, Generalized Massive Optimal Data Compression, MNRAS (2018). Primary manuscript. [^6]: Paulin and Elvira, Scalable Estimation of VARMA Models (2026 preprint). Primary manuscript. [^7]: Harth-Kitzerow et al., Toward Bayesian Data Compression, Annalen der Physik 533, 2000508 (2021). Publisher article. [^8]: Katzfuss and Guinness, A General Framework for Vecchia Approximations of Gaussian Processes, Statistical Science (2021). Primary manuscript. [^9]: Doucet, A Note on the Sensitivity of the Posterior Distribution to the Likelihood. Author's note. [^10]: Castaldo, Whaley and Chronopoulos, Reducing Floating-Point Error in Dot Product Using the Superblock Family of Algorithms, SIAM Journal on Scientific Computing (2008). Author-hosted paper. [^11]: mpmath documentation, Arbitrary-Precision Interval Arithmetic. Official documentation. [^12]: Unser, Tafti, Amini and Kirshner, A Unified Formulation of Gaussian Versus Sparse Stochastic Processes - Part II: Discrete-Domain Theory, IEEE Transactions on Information Theory 60(5), 3036-3051 (2014). Author-hosted paper. [^13]: Bera et al., Fast Bayesian Inference of Optical Trap Stiffness and Particle Diffusion, Scientific Reports 7, 41638 (2017). Primary manuscript. [^14]: Perez-Garcia et al., Optimal Calibration of Optical Tweezers with Arbitrary Integration Time and Sampling Frequencies, Biomedical Optics Express 14(12), 6442-6469 (2023). Publisher article. [^16]: Artemev, Burt and van der Wilk, Tighter Bounds on the Log Marginal Likelihood of Gaussian Process Regression Using Conjugate Gradients, ICML (2021). Primary paper. [^15]: Ren et al., Neuromorphic Detection and Cooling of Microparticles in Arrays, Nature Communications 16, 10658 (2025). Primary article. [^17]: Ren, Neuromorphic Detection and Control of Particle Arrays, King's College London dataset, version 1 (2024). Public repository.
Original: research/compression_physical_memory_20260915/REPORT.md · Raw source file
View raw MD source
# Revisable physical memory with bounded inference error
## Executive assessment
A useful compression system need not recover its input waveform. It can instead
preserve a declared family of future computations. The practical vision is an
instrument that discards most of a long measurement record while retaining the
ability to reconsider physical parameters and sensor calibration later. The
important restriction is that later questions must remain inside an explicit
contract. Neither a small file nor accurate answers to a few selected queries
establishes general information recovery.
The completed physical-inference panel demonstrates a narrow version of this
capability. A roughly 12.6 kB record of lag products, boundary samples and a sum
supports later inference about a stationary noisy stochastic oscillator. Across
48 synthetic records and two calibration choices, it agrees closely with a
compiled exact Kalman reference on the fixed 1,271-point physical-parameter
grid. Its retained size is at least 40 times smaller than the tested lossless
raw representation for the longer records. It is slower end to end on the
shorter records. A sufficiently dense ordinary parameter bank also passes the
accuracy gate and is faster for warm queries.
The main conceptual development is an answer-error contract. A finite-memory
approximation is related to the exact Gaussian likelihood by a model-dependent
forgetting bound. The capsule's energy then turns that relation into a bound
for the particular observed record, rather than only an average over possible
records. A separate numerical audit tested whether capture and query
roundoff can be enclosed as well. That audit now passes all 96 cases, with
largest posterior-TV bound 0.000297, and independent exact-statistic and
full-record reference checks pass. Its approximately 66-minute query cost
is substantial and must remain separate from ordinary inference timings.
A subsequent streaming implementation removes the full-record capture
workspace. It retains about 12.7 kB using 144,016 bytes of native live state.
All eight million-sample record/calibration cases meet the prescribed bound,
but six of eight four-million-sample cases do not. The fixed configuration's
full scale-up gate therefore fails. Its observed reference agreement remains
close, but the error budget and coarse posterior grid prevent an unlimited-
duration or resolved continuous-inference claim.
These findings do not establish a general compression breakthrough, a novel
theory of sufficient statistics, or a spline-specific advantage. Gaussian
message composition, finite-conditioning approximations, time-series
statistics and Bayesian data compression all have substantial precedents.
The defensible question is whether their particular combination yields a
useful, auditable physical-memory capability at an acceptable total cost.
## The capability and its limits
Conventional lossless compression promises to reconstruct every input bit.
Lossy waveform compression promises a particular distortion measure. The
object studied here makes a different promise: answer later likelihood and
posterior queries for a stated observation model, with a stated error budget.
It is closer to a compressed scientific record than to an audio codec.
Three aspects make this more demanding than retaining a fitted parameter.
The physical parameters need not be selected during capture. Detector gain and
offset can change between the original and revised query. The answer includes
the relative likelihood across competing physical hypotheses, not merely the
location of one optimum. Different priors on the same hypothesis set can
therefore be applied after capture.
This freedom is limited. The present representation assumes scalar, regularly
sampled observations of a stationary linear Gaussian system and known
post-gain independent observation-noise variance. It does not retain arbitrary
time-local events, permit unrestricted changes to the observation process, or
support a new nonlinear model merely because that model also describes
physics. A posterior close to the full-data posterior can still be scientifically
wrong when the likelihood is misspecified.
The selected application regime is constrained retention or transmission
followed by later analysis. It is not automatically the fastest way to analyze
a short recording already available on a workstation. Persistent bytes,
live capture memory, capture time, query setup, warm query time and numerical
assurance cost are different quantities and must be reported separately.
## Prior-art and equivalence gate
The first candidate stored Gaussian factors over segment endpoints and
composed them by eliminating interior states. This is useful algebra, but
associative Gaussian filtering and smoothing already provide the essential
construction. Sequential and tree-based factor composition cannot be claimed
as a new spline-specific algorithm merely because an operator formulation
motivates it.[^1]
The corresponding physical-parameter bank also has an elementary equivalence:
transforming ordinary nodal likelihood values into Chebyshev coefficients does
not create additional information. It changes a representation and its
evaluation procedure. The verification panel checked Gaussian factors,
composition order, dense references, later boundary priors and this bank
equivalence. All 1,440 checks passed; the largest normalized discrepancy was
7.26e-12. The generic factor-bank novelty claim was closed.
More sophisticated precedents also matter. Structured Gaussian-process
methods such as celerite make exact inference practical for relevant
oscillatory covariances; a dense cubic-cost Gaussian-process baseline would
therefore exaggerate the candidate's benefit.[^2] Family-wide Fourier
preprocessing accelerates later queries within a specified kernel family.[^3]
Relative binning provides another route, using a local reference for waveform
likelihood calculations.[^4] Score compression preserves local information near a
fiducial model; it should not be confused with a uniform posterior guarantee
over arbitrary future models.[^5]
A particularly close recent precedent is fixed-size time-series compression
using lag statistics and boundary information for ARMA-related inference.
Paulin and Elvira's analysis includes logarithmically growing truncation order
under stated assumptions. Their predictor-recovery result and the present
per-record likelihood enclosure are different contracts, but that distinction
does not establish priority for every ingredient of this construction.[^6]
Bayesian data compression already frames compression in terms of information
retained about a posterior. Its response, noise and prior metadata are part
of the representation and cannot be ignored in a storage comparison.[^7]
Consequently, the broad slogan "compression that preserves inference" is
not a novelty claim available to this project. A publication would need to
identify a new guarantee, a genuinely different operating regime, or a
validated practical advantage over these established ideas.
Deterministic Gaussian likelihood bounds are also established. Conjugate-
gradient lower bounds combine determinant inequalities with a computable
residual bound for the data-dependent quadratic term, and use that bound to
control the stopping error.[^16] The present distinction is not the existence
of a likelihood certificate. It is whether a useful certificate can be
computed from a small retained record after the interior observations are
unavailable. No matched performance comparison with that method is claimed.
## Physical model and retained representation
The latent state contains oscillator position and velocity. Its continuous
drift matrix is $A=\left(\begin{smallmatrix}0&1\\-\omega^2&-2\gamma\end{smallmatrix}\right)$.
The stationary covariance is $S=\operatorname{diag}(1,\omega^2)$, the sampled
transition is $F=\exp(A\Delta t)$, and process covariance is $Q=S-FSF^T$.
An observation is $y_t=g x_{t,1}+b+\epsilon_t$, with independent Gaussian
noise of variance $R=0.04$ added after detector gain. This order is important:
moving the noise through the gain changes the likelihood. An explicit negative
control produced discrepancies as large as 42.758 log-likelihood units when
the observation semantics were changed incorrectly.
The stationary normalization is a substantive physical assumption. For a
white-noise forcing amplitude $\sigma$, stationary Lyapunov balance gives
$\operatorname{Var}(x_1)=\sigma^2/(4\gamma\omega^2)$. Fixing that variance
to one therefore sets $\sigma^2=4\gamma\omega^2$ throughout this family.
It is not automatically a thermal instrument at fixed temperature and mass.
If detector gain and forcing amplitude were both unknown, their product would
be observable but the two factors would not be separately identifiable from
this channel alone. The two tested calibration choices are specified
alternative observation hypotheses, not a solution to that identifiability
problem. Frequency is angular frequency in the declared model units.
The retained numerical statistics are $C_k=\sum_{t=k}^{N-1}y_t y_{t-k}$ for
$0\leq k\leq L$, the first and last $L$ observations, the total sum, sample
count and observation metadata. The panel fixes $L=512$. Interior samples are
discarded from the candidate representation. The original implementation
uses blocked FFT calculations and merge identities; the numerical-enclosure
implementation separately recaptures the statistics using controlled
pairwise arithmetic.
For a queried physical model, covariance lags determine the best linear
predictor from the previous $L$ observations. The first $L$ observations are
handled with their exact joint Gaussian likelihood. Subsequent conditional
terms use the same finite predictor. Expanding the squared residuals gives
lag-product terms plus corrections from the retained beginning and end.
Offset changes are handled by the retained linear sum and corresponding
boundary algebra. The capsule therefore contains enough information to
evaluate this finite-conditioning likelihood without rereading its interior.
This is a chronological Vecchia approximation, not an assertion that the
noisy observed oscillator is exactly an order-$L$ Markov process. The broader
Vecchia literature provides the appropriate conditional-factorization
context.[^8] Finite support of a driving filter, sparse continuous operators
and finite conditional memory are not interchangeable properties.
## From approximation to an answer-error contract
Let $P_\theta$ denote the exact length-$N$ Gaussian observation distribution
and $Q_{\theta,L}$ the finite-conditioning distribution. Write $v_j$ for
the prediction variance after conditioning on $j$ previous observations.
Stationarity and exact finite conditional factors give
$D_{\mathrm{KL}}(P_\theta\Vert Q_{\theta,L})=
\frac12\sum_{t=L+1}^{N}\log(v_L/v_{t-1})$.
If $v_\infty$ is the steady prediction variance, an upper bound is
$B_\theta=\frac12(N-L)\log(v_L/v_\infty)$.
A uniform upper bound over a declared parameter family can control
prior-predictive expected posterior error. That is useful for a memory-budget
screen, but it is not a guarantee for each realized record. In particular,
an average divergence statement cannot simply be relabeled an observed-data
posterior certificate.
The second step uses covariance structure to obtain a pathwise bound. With
exact conditional factors, the relative precision matrix has trace $N$, and
the log-determinant difference is $2D_{\mathrm{KL}}$. A conservative relative
spectral deviation bound is
$\delta(B)=2B+2\sqrt{B(B+1)}$.
Because the exact covariance is at least $RI$, the centered record energy
$E_b=\sum_t(y_t-b)^2$ bounds the relevant quadratic form. Thus
$-B-\delta(B)E_b/(2R)\leq\log Q_{\theta,L}(y)-\log P_\theta(y)
\leq\delta(B)E_b/(2R)$.
The capsule retains the quantities needed to compute $E_b$.
Likelihood error matters up to an additive constant shared across hypotheses.
If its range has width $\Omega$, the total-variation distance between the
normalized posteriors is at most $\tanh(\Omega/4)$ for any shared prior on
that hypothesis set. General posterior sensitivity to likelihood perturbation
is established mathematics; the project derives the required specialization
and records its total-variation convention explicitly.[^9]
Obtaining a reliable tiny $B$ by subtracting nearly equal floating prediction
variances is unsafe. Instead, let $P_*$ be the stationary Riccati solution and
$A_*=F(I-P_*H^TH/v_\infty)$ the closed-loop prediction matrix. Monotonicity
and a Riccati increment inequality give
$v_L-v_\infty\leq H A_*^L S(A_*^T)^L H^T$.
This bound avoids cancellation in $v_L-v_\infty$. It also exposes the
mechanism: stable prediction forgets increasingly distant observations.
Under uniform geometric forgetting and ordinary energy growth, a sufficient
lag grows logarithmically with recording length and inverse error tolerance.
That is an asymptotic upper-bound statement under assumptions, not a lower
bound, proof of optimality, or promise that a fixed 512-lag capsule works for
arbitrarily long recordings. Near weakly damped or poorly observed systems,
the constants and conditioning can be unfavorable.
The number of real-valued statistics is not an information-theoretic bit
rate. Stored magnitudes and numerical precision also matter. All reported
objects include their actual binary64 payload, uncertainty radii and metadata;
the supported input domain and record length are finite. The asymptotic lag
argument does not promise a fixed number of 64-bit words for an unlimited
stream, and it is not an optimal rate-distortion theorem.
## Frozen experiment and complete comparator results
The confirmation panel uses 12 named independent seed identities, each at two
sampling intervals, 0.01 and 0.05, and two lengths, 4,096 and 65,536. The shorter
record is a prefix of the longer one within a sampling interval. Those paired
lengths are not independent replications. Sampling intervals use separate
child random streams. The resulting 48 records are synthetic mechanism tests,
not 48 independent physical experiments.
The true generating parameters are frequency 3.1, damping 0.43, gain 1.05
and offset 0.1. Queries use a fixed grid of 41 frequencies and 31 dampings.
The original calibration is gain 0.9 and offset zero; the revised calibration
is gain 1.05 and offset 0.1. Uniform and biased priors are applied to the same
grid. Agreement is with that discrete posterior, not with a continuous
parameter posterior; the generating damping is not a grid point.
The low-storage candidate is compared with compiled exact Kalman evaluation,
raw evaluation of the same finite-conditioning approximation, lossless zlib
storage, and ordinary Chebyshev parameter banks with 8 or 16 nodes per axis
over frequency, damping and gain. Offset dependence is handled analytically
for the banks as well. No model, lag or bank-degree search was performed on
the confirmation results.
| Representation | Combined accuracy gate | Retained bytes | Maximum posterior TV |
| --- | --- | --- | --- |
| Lag capsule, $L=512$ | 96 of 96 cases pass | 12,584-12,587 | 1.705e-8 |
| Ordinary bank, 8 nodes per axis | 0 of 96 cases pass | 16,695-16,696 | 0.1720 |
| Ordinary bank, 16 nodes per axis | 96 of 96 cases pass | 131,384-131,385 | 4.863e-5 |
The combined gate requires likelihood-error oscillation no greater than 0.04
and posterior TV no greater than 0.01 under both declared priors. The lag
capsule's largest observed oscillation is 1.408e-7; the 16-node bank's is
0.0015471. Agreement with independently replayed finite-conditioning
statistics has maximum normalized discrepancy 2.949e-13. All 200 saved
physical objects across development and confirmation passed hash, byte and
round-trip verification.
End-to-end timing includes capture, serialization, decoding, preparation and
both full query grids. At length 4,096, the lag approach takes approximately
1.27-1.29 seconds, compared with 0.126-0.127 seconds for the raw reference and
0.362-0.371 seconds for the accurate bank. At length 65,536, it takes
1.26-1.32 seconds, compared with 2.00-2.08 seconds for the raw reference and
3.37-3.43 seconds for the accurate bank. These are workload-specific prototype
timings, not universal speed ratios.
The lag capsule is at least 40.03 times smaller than lossless raw storage on
the longer records, but only about 2.51 times smaller on the shorter ones.
The accurate bank has faster warm queries. Some model-only bank quantities
could be shared or regenerated, and the exact reference could use additional
steady-state optimizations. Neither omitted optimization is credited as a
measured result. Peak process RSS in this panel was about 141 MiB.
There is an even smaller control when every future query is known in advance.
One binary64 likelihood value at each of the 1,271 hypotheses for both fixed
calibrations would use 20,336 payload bytes before metadata and would support
later priors on that grid exactly. This is a storage calculation, not a newly
measured implementation. It would move the inference work into capture and
would not support unlisted physical or calibration queries. The lag capsule's
case therefore rests on deferred query flexibility and capture cost, not on
the false premise that all alternatives must retain raw observations.
## Mathematical and numerical assurance
The data-free information screen covered 750 combinations of physical query,
sampling interval and record length through 1,048,576 observations. Every
case met its declared information-budget condition at the maximum lag, and
the largest selected lag in the screen was 195. This was a feasibility screen,
not permission to select a favorable lag on confirmation data.
The capsule-only pathwise assessment subsequently covered all 96 frozen
record/calibration cases. Its largest numerical posterior-TV envelope was
1.67726e-5, below 0.01. No raw recording or saved raw likelihood was consulted
to compute those envelopes. Nevertheless, ordinary floating Riccati solvers,
FFT statistics and likelihood arithmetic meant that this stage established
a mathematical bound evaluated numerically, not an end-to-end machine
certificate.
The separate numerical-enclosure study uses controlled binary64 pairwise
products and sums, with rounding bounds based on tree depth. Pairwise
summation and its error analysis are classical; their use here is numerical
accounting rather than a new compression principle.[^10] Stored objects
include uncertainty radii and an integrity hash. Exact integer arithmetic
on binary64 inputs is reserved for independent verification of the captured
statistics.
Model calculations use 80-decimal-digit interval arithmetic. Proposed Riccati
solutions are accepted only when matrix subsolution and supersolution
inequalities verify. The finite predictor's error is bounded using its
Toeplitz-system residual and the known observation-noise lower bound.
Boundary terms, finite-prefix likelihood, normalizers and centered energy
are enclosed before combining arithmetic and truncation error.
An initial interval-prefix implementation failed because repeated interval
propagation lost a useful positive covariance enclosure. That failure is
preserved. The development correction uses verified matrix-order brackets
at each prefix step, not an arbitrary posterior error inflation. The corrected
eight-record development panel passes, with maximum TV envelope 0.000273,
and its 45 unit tests pass. The subsequent frozen full-cohort audit passes
all 96 cases, with maximum posterior-TV bound 0.0002971093. The retained
objects use 12,695-12,700 bytes including radii, metadata and integrity digest.
Independent verification checks all 24,624 captured lag statistics against
exact integer arithmetic on the original binary64 observations. All pass,
as do energy, sums, exact boundaries, metadata, source hashes and object
round trips. All 122,016 floating full-record likelihood references and their
finite-conditioning counterparts lie within their respective intervals.
The largest measured posterior TV is 1.7124e-8, and the largest exact-
likelihood interval width is 0.00118844. Reference containment is a strong
cross-check, not an independently formalized solver proof.
The full query phase takes 3,954.83 seconds: approximately 2,657 seconds of
model preparation, 1,170 seconds of prefix calculations and 125.5 seconds
of capsule-tail work, plus overhead. It reuses prefix work for paired lengths
and reads no raw observations. Independent verification takes 56.05 seconds.
Final peak process RSS is 124.25 MiB. The assurance layer is therefore much
more expensive than the ordinary calculations in the earlier comparison.
The numerical trust boundary must remain explicit even if the audit passes.
The implementation relies on IEEE arithmetic assumptions, compiler settings
and the basic interval operations of mpmath, whose documentation describes
the interval implementation as experimental.[^11] It is not a formally
verified proof assistant. The contract concerns exact normalization of the
published likelihood values on the declared grid; it does not independently
certify a downstream floating softmax implementation or a continuous-prior
integral.
Study 05's controlled capture materializes the input and pairwise scratch
space proportional to its length. Its roughly 12.7 kB retained object is not
a 12.7 kB live-memory implementation. The separately frozen streaming study
addresses that acquisition boundary, while retaining the same query arithmetic
and error tolerance. Its results follow.
## Bounded live capture and the long-record boundary
The streaming implementation maintains a ring buffer, exact initial boundary
and a binary counter of balanced partial sums. It consumes at most 4,096
observations per input block. Finalizing occupied counter levels from smallest
to largest preserves the required reduction-depth bound. This is classical
balanced summation, not a new compression primitive. A fixed 32-level counter
supports the declared domain $N<2^{31}$, rather than an unlimited stream.
Development checks include exact-integer lag statistics, boundary preservation,
chunk-partition invariance, snapshot/continue behavior and rejection before
state mutation. A last-bit source-reproduction discrepancy was traced to
compile-time rather than runtime physical parameters and corrected without
weakening the exact-equality test. All development checks passed before the
long-stream cohort was admitted.
The frozen panel uses two new seed identities, each with sampling intervals
0.01 and 0.05 and paired prefixes of 1,048,576 and 4,194,304 observations.
It keeps the lag, model, 1,271-point query grid, calibration choices and priors
unchanged. Four streams yield eight prefixes, not eight independent physical
replications. Exact small-state floating Kalman references are accumulated
while observations are available; no full raw array or recording is saved.
Capsule-only querying and reference verification run as separate phases.
| Observations | Sampling interval | Bound pass cases | Largest TV bound |
| --- | --- | --- | --- |
| 1,048,576 | 0.01 | 4/4 | 0.00444167 |
| 1,048,576 | 0.05 | 4/4 | 0.00304670 |
| 4,194,304 | 0.01 | 0/4 | 0.01843085 |
| 4,194,304 | 0.05 | 2/4 | 0.01220511 |
All million-sample cases pass. At the longer prefix, both original-calibration
cases at interval 0.05 pass, while the other six cases are rejected. The
first tested rejected length is 4,194,304; intermediate lengths were not
searched. The complete joint gate fails, and no lag, precision, hypothesis
or tolerance is changed to rescue it.
All 20,336 full-record floating likelihood references lie inside their
intervals. All 32 posterior comparisons remain within their corresponding
bounds, with largest measured TV $5.37358\times10^{-6}$. This close agreement
does not override the prescribed rejection. On every rejected case, the
finite-conditioning likelihood's numerical enclosure alone is too wide for
the chosen posterior budget; physical truncation also contributes in some
cases. This is a limit of the current conservative implementation, not proof
that the actual error is that large or that no other method can succeed.
Native accumulator allocation stays at 144,016 bytes, and saved capsules
occupy 12,700-12,703 bytes. Capture peak RSS is 117.20 MiB and query peak RSS
115.42 MiB. All resource gates pass. Counting each stream only once through
its longest prefix, candidate append work takes 6.127 seconds. Source
generation takes 0.399 seconds and online reference work 485.206 seconds;
the total capture phase takes 491.895 seconds. The reference maintains extra
statistics and is not an optimized minimal answer bank.
The numerical query phase takes 2,909.876 seconds, about 48.5 minutes:
2,688.442 seconds of model and prefix-schedule preparation, 199.384 seconds
of record-prefix work and 21.701 seconds of capsule-tail work, plus overhead.
These measurements include shared model work and paired-prefix reuse. They
do not establish embedded battery savings or a universal speed advantage.
An uncompressed binary64 million-sample payload would be roughly 660 times
the capsule size. The four-million-sample payload would be roughly 2,642
times its capsule size, but the latter scale fails the full answer gate.
These are payload arithmetic comparisons, not newly measured lossless-codec
results. The earlier panel remains the measured lossless and ordinary-bank
comparison. A query snapshot also does not serialize the complete live
counter needed for power-loss recovery, nor permit arbitrary interior edits.
The predeclared resolution diagnostics expose an important scientific limit.
Across all 32 combinations, effective support is only 1.0-1.854 grid points.
At the longer interval-0.05 prefix, maximum posterior mass is between
0.999994657 and 1.0. The generating damping 0.43 is absent from the grid.
Consequently, very small discrete-posterior TV is not evidence of correctly
resolved continuous uncertainty; zero boundary-grid mass does not fix this.
The specified calibration revision changes posterior mean frequency from
about 3.15 to 3.10 and damping from approximately 0.560-0.580 to 0.420-0.440
across the reported cases. The retained representation reproduces that
model-dependent reinterpretation closely after interior observations are
discarded. These are descriptive paired outputs, not a newly selected
threshold success or an identifiable thermal-calibration experiment.
All 61 campaign tests and 12 subtests pass, including the four multichannel
algebra checks. A reporting-key collision was corrected before the grid
diagnostics ran, without changing their numerical calculations. The saved-
archive integrity audit also passes while preserving the failed numerical
gate. No additional physical-data fit is admitted by these results.
## A caution from the operator-spline literature
Operator-based spline theory supplies useful connections between continuous
stochastic systems, localized innovations and discrete representations.
It does not remove the need to distinguish marginal dependence from
conditional dependence. A detailed rereading of the generalized-innovation
paper identified a problem with the finite-conditional-memory identity
displayed in its Property 4.[^12]
For the second derivative driven by Gaussian white noise, the corresponding
triangular compactly supported increment kernel yields autocovariances
$c_0=2/3$, $c_1=1/6$ and $c_2=0$. Yet the conditional covariance between
increments two steps apart, given the intervening increment, is $-1/24$.
Thus compact support and zero distant marginal covariance do not imply the
displayed finite-history conditional identity. This elementary counterexample
is recorded with exact and numerical tests.
This does not invalidate the whole paper or its operator framework, and it
is not a newly discovered general phenomenon in time-series theory. It does
prevent an unjustified exact-memory claim in this project. The current
finite-conditioning method includes approximation error precisely because
the observed process need not have finite conditional memory.
## Application qualification
Optical-trap calibration is a coherent application lead because stiffness,
damping and detector response have concrete scientific meanings. It also
illustrates why a strong simple comparator is indispensable. Directly observed
Ornstein-Uhlenbeck calibration already admits a likelihood described by four
quadratic statistics. The relevant literature discusses this compact
description together with experimental drift and noise limitations.[^13]
A 12.7 kB capsule is not progress when four statistics answer the actual
question exactly.
Finite exposure changes observation covariance and can make the measured
sequence non-Markovian. Published work derives this covariance and uses a
one-step approximate likelihood for a compact estimator.[^14] That suggests
a possible comparison involving later detector-response or noise revision,
but the physical model must be the same for all methods. Comparing a
correctly modeled candidate with an intentionally misspecified compact
baseline would confuse improved modeling with improved compression.
The scientific endpoint would need to distinguish physical estimation error,
uncertainty, model inadequacy and compression-induced error. The cited
2023 dataset is available on request rather than publicly provided; no
author contact or data acquisition has taken place. No real-data panel is
admitted on the strength of an attractive application story.
Mechanical condition monitoring is another plausible destination, but a
stationary Gaussian oscillator is not a general model of machinery. Impacts,
changing speed, external forcing, drift and non-Gaussian disturbances can
dominate a real maintenance decision. A claim about detecting meaningful
damping or resonance changes requires physical evidence and an appropriate
decision threshold, not only agreement with a synthetic likelihood.
## A possible unifying framework
The broader opportunity is a scientific memory format with explicit query
rights. An acquisition contract specifies the observation semantics and
admissible model family. A structured representation retains the calculations
needed by that family. A later query returns both a numerical answer and an
error budget, and the system abstains when that budget is insufficient.
This is a possible organizing framework, not a general-purpose compiler
already implemented by the present code.
The valuable freedom is to revise a scientific interpretation after interior
measurements have been discarded, without pretending that every future
interpretation remains available. Raw waveform reconstruction, arbitrary
interior corrections, unknown observation mechanisms and changing a model
outside the declared family are different capabilities. A practical format
must expose those exclusions as clearly as the answers it can return.
The real-arithmetic derivation extends to vector observations by retaining
oriented cross-channel lag matrices and boundary vectors. For an observation
dimension $d$, its storage is approximately $O(Ld^2)$, not independent of
sensor count. The determinant and conditional-innovation argument generalizes,
but a validated multichannel query engine, numerical matrix inequalities and
resource measurements have not been established. Existing multivariate
time-series compression remains a close comparator; merely using several
channels does not create novelty.
There is a concrete link to the neuromorphic sensing vision. Published work
uses event-based sensing and a PC/FPGA/filter loop to cool three uncoupled
levitated particles, while identifying filtering and feedback-noise effects
as limitations.[^15] A future scientific-memory question could be whether later
calibration or feedback-model revision remains trustworthy after data discard.
That would require synchronized control records and a noise model appropriate
to feedback, not the independent-noise model imported unchanged. No such
application has been run here.
The associated public archive can be inspected selectively. Its directory
contains 1,289 CSV files and acquisition notes, while the whole ZIP exceeds
2 GB.[^17] A metadata-only review identifies tracking-column definitions and
record-to-feedback-condition mappings. It does not establish complete control
provenance or calibrated physical observations. File counts are not independent
trials, and no measurement CSV has been opened or fitted in this campaign.
The decisive demonstration would pair a useful deferred scientific decision
with a constrained acquisition budget, qualified physical observations and
strong compact alternatives. It must show that reanalysis preserves the
decision and its uncertainty at lower total resource cost. Better waveform
error, a tiny object, or more model parameters alone would not meet that
standard. The present work establishes components of that route and tests
their boundaries; it does not establish the final instrument-level benefit.
## Research decision
The strongest established result is a compact, parameter-revisable physical
record with very accurate tested inference and an explicit mathematical path
to per-record answer bounds. The major negative results are equally clear:
the generic Gaussian-factor bank is not a new method, the small ordinary bank
fails the accuracy gate, the larger bank remains a strong competitor, and
short-record speed does not favor the candidate.
The bounded scale-up is closed. It establishes fixed live acquisition memory
over the tested lengths and passing answer bounds at one million observations,
but the same configuration fails its full four-million-observation budget.
Further tuning of the revealed cohort is not the next research decision.
A flagship application requires qualified measurements and a scientific
decision that simpler sufficient-statistic alternatives cannot already answer.
The particle archive supplies a concrete source lead, not that qualification
or a physical result. No spline component should be added merely to make the
work resemble the project's origin.
A practical decision contract can be stated without inventing a new score.
If posterior TV is at most $\epsilon$, every declared event probability changes
by at most $\epsilon$. A threshold decision is protected when its probability
margin exceeds that bound; otherwise the compressed answer can abstain.
Expectations of a quantity ranging over $[a,b]$ change by at most
$(b-a)\epsilon$. These are standard consequences of total variation, not
evidence that the underlying physical model or chosen decision is correct.
## Sources
[^1]: Särkkä and García-Fernández, Temporal Parallelization of Bayesian Smoothers, revised manuscript (2020). [Primary manuscript](https://arxiv.org/abs/1905.13002).
[^2]: Foreman-Mackey et al., Fast and Scalable Gaussian Process Modeling with Applications to Astronomical Time Series, Astronomical Journal (2017). [Primary manuscript](https://arxiv.org/abs/1703.09710).
[^3]: Greengard, Efficient Fourier Representations of Families of Gaussian Processes, revised manuscript (2024). [Primary manuscript](https://arxiv.org/abs/2109.14081).
[^4]: Zackay, Dai and Venumadhav, Relative Binning and Fast Likelihood Evaluation for Gravitational Wave Parameter Estimation (2018). [Primary manuscript](https://arxiv.org/abs/1806.08792).
[^5]: Alsing and Wandelt, Generalized Massive Optimal Data Compression, MNRAS (2018). [Primary manuscript](https://arxiv.org/abs/1712.00012).
[^6]: Paulin and Elvira, Scalable Estimation of VARMA Models (2026 preprint). [Primary manuscript](https://arxiv.org/abs/2608.06340).
[^7]: Harth-Kitzerow et al., Toward Bayesian Data Compression, Annalen der Physik 533, 2000508 (2021). [Publisher article](https://doi.org/10.1002/andp.202000508).
[^8]: Katzfuss and Guinness, A General Framework for Vecchia Approximations of Gaussian Processes, Statistical Science (2021). [Primary manuscript](https://arxiv.org/abs/1708.06302).
[^9]: Doucet, A Note on the Sensitivity of the Posterior Distribution to the Likelihood. [Author's note](https://www.stats.ox.ac.uk/~doucet/doucet_notesensitivityposteriortolikelihood.pdf).
[^10]: Castaldo, Whaley and Chronopoulos, Reducing Floating-Point Error in Dot Product Using the Superblock Family of Algorithms, SIAM Journal on Scientific Computing (2008). [Author-hosted paper](https://www.cs.utsa.edu/~atc/pub/J42.pdf).
[^11]: mpmath documentation, Arbitrary-Precision Interval Arithmetic. [Official documentation](https://mpmath.org/doc/current/contexts.html).
[^12]: Unser, Tafti, Amini and Kirshner, A Unified Formulation of Gaussian Versus Sparse Stochastic Processes - Part II: Discrete-Domain Theory, IEEE Transactions on Information Theory 60(5), 3036-3051 (2014). [Author-hosted paper](https://bigwww.epfl.ch/publications/unser1402.pdf).
[^13]: Bera et al., Fast Bayesian Inference of Optical Trap Stiffness and Particle Diffusion, Scientific Reports 7, 41638 (2017). [Primary manuscript](https://arxiv.org/abs/1610.00315).
[^14]: Perez-Garcia et al., Optimal Calibration of Optical Tweezers with Arbitrary Integration Time and Sampling Frequencies, Biomedical Optics Express 14(12), 6442-6469 (2023). [Publisher article](https://doi.org/10.1364/BOE.495468).
[^16]: Artemev, Burt and van der Wilk, Tighter Bounds on the Log Marginal Likelihood of Gaussian Process Regression Using Conjugate Gradients, ICML (2021). [Primary paper](https://proceedings.mlr.press/v139/artemev21a.html).
[^15]: Ren et al., Neuromorphic Detection and Cooling of Microparticles in Arrays, Nature Communications 16, 10658 (2025). [Primary article](https://www.nature.com/articles/s41467-025-65677-0).
[^17]: Ren, Neuromorphic Detection and Control of Particle Arrays, King's College London dataset, version 1 (2024). [Public repository](https://doi.org/10.18742/28069136.v1).