The analyse time series tool

The analyse tool (yes, folks, that's the British spelling!) is a (fairly) simple C++ program that can read a variety of time-series data formats and perform various processes on it, before outputting it to a format suitable for plotting - in particular, with gnuplot.

It's this tool which powers the interactive graph generator on this site; but feel free to make your own service with it.

Analyse is licenced under GPLv3. If you've added some sexy feature you think I should roll back into my source, send me a patch at 'paul' at this domain, but please check first to avoid duplication of effort.

What it can do

The analyse tool can perform any of the following steps on the data, in any order:

  • Select samples in a given range
  • Select samples at a given interval
  • Apply scale and offset to samples
  • Take the derivative (first difference) of the data
  • Integrate the data (cumulative)
  • Take the log (base 10) of the data
  • Do a running mean over a given number of samples
  • Average and reduce by a given number of samples
  • Remove a running mean from the data, leaving the noise (or signal, according to taste)
  • Fourier transform to the frequency domain
  • Take the absolute magnitude of complex Fourier series
  • Low or high-pass filter in the frequency domain
  • Inverse Fourier back to time domain
  • Apply a Hann window to the data to remove Fourier edge effects

It can read data in the following formats:

  • HADCRUT3/HADSST2/CRUTEM3 monthly mean data
  • HADCRUT4/CRUTEM4 monthly mean data
  • GISTEMP monthly mean data
  • UAH NSSTC monthly mean MSU data
  • RSS monthly mean MSU data
  • BEST monthly mean data
  • SIDC monthly sunspot number (SSN)
  • PMOD daily TSI (averages to monthly)
  • ESRL Mauna Loa CO2 monthly average
  • JISAO PDO index
  • ESRL AMO index
  • NSIDC Sea Ice Index
  • CSSE Covid-19 CSV

The 'merge' tool can read multiple datasets in gnuplot format and create a new dataset with their mean values (used to create WTI)

Download

Source

analyse-0.9.6.tgz

This is C++11 which should build in any sane compiler. A Makefile for GNU systems (e.g. Linux) is provided, along with some test data and example commands.

To plot the graphs direct from the Makefile ('make display') you need gnuplot (Debian: apt-get install gnuplot), and to view them a PNG viewer (Debian: apt-get install imagemagick)

To create and listen to the audio samples, you need twolame (Debian: apt-get install twolame) and any old media player

Documentation? See the Help pages here, run it with no arguments or Use The Source!

Apology of the month

Analyse's DFT is slow

In this first version the DFT is the simplest, most naive implementation I could write, because I wanted to understand the basics. This means it's O(N2). As someone who makes a living writing scalable real-time systems, I hang my head in shame ;-).

What this means for non-mathematicians is that if you do Fourier analysis on large data sets (anything more than a couple of thousand samples) your computer will grind for ages and use up lots of electricity, which could be seen as rather ironic....

It's high on the list to fix, honest!