Question

we have a particle detector hard-wired to use 16-bit and 8-bit buffers. Every now and then, there are certain [predicted] peaks of particle fluxes passing through it; that’s okay. What is not okay is that these fluxes usually reach magnitudes above the capacity of the buffers to store them; thus, overflows occur. On a chart, they look like the flux suddenly drops and begins growing again. Can you propose a [mostly] accurate method of detecting points of data suffering from an overflow?

P.S. The detector is physically inaccessible, so fixing it the ‘right way’ by replacing the buffers doesn’t seem to be an option.

Update: Some clarifications as requested. We use python at the data processing facility; the technology used in the detector itself is pretty obscure (treat it as if it was developed by a completely unrelated third party), but it is definitely unsophisticated, i.e. not running a ‘real’ OS, just some low-level stuff to record the detector readings and to respond to remote commands like power cycle. Memory corruption and other problems are not an issue right now. The overflows occur simply because the designer of the detector used 16-bit buffers for counting the particle flux, and sometimes the flux exceeds 65535 particles per second.

Update 2: As several readers have pointed out, the intended solution would have something to do with analyzing the flux profile to detect sharp declines (e.g. by an order of magnitude) in an attempt to separate them from normal fluctuations. Another problem arises: can restorations (points where the original flux drops below the overflowing level) be detected by simply running the correction program against the reverted (by the x axis) flux profile?

Answer 1

int32[] unwrap(int16[] x)
{
   // this is pseudocode
   int32[] y = new int32[x.length];
   y[0] = x[0];
   for (i = 1:x.length-1)
   {
      y[i] = y[i-1] + sign_extend(x[i]-x[i-1]);
      // works fine as long as the "real" value of x[i] and x[i-1]
      // differ by less than 1/2 of the span of allowable values
      // of x's storage type (=32768 in the case of int16)
      // Otherwise there is ambiguity.
   }
   return y;
}

int32 sign_extend(int16 x)
{
   return (int32)x; // works properly in Java and in most C compilers
}

// exercise for the reader to write similar code to unwrap 8-bit arrays
// to a 16-bit or 32-bit array