Home/ Questions/Q 547403
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-13T10:58:10+00:00

I have a dataset with 100 000 datapoints which I have to plot on

  • 0

I have a dataset with 100 000 datapoints which I have to plot on a graph. The resulting graph will be about 500px wide, so for every pixel there will be about 200 datapoints, which seems quite unnecessary.

I need to find a way to get rid of the excess datapoints without losing the shape of the graph to speed up the rendering. Currently the rendering of all 100 000 points can take 10+ seconds as I’m also using anti-aliasing and other “effects”.

I tried to approach this problem by just taking every 200th datapoint and plotting them, but this results in some of the more significant points missing out (think about spikes in the graph that I want to be able to show). I also thought of splitting the dataset in chunks of 200 datapoints, then taking the maximum value from every chunk but that wont work either.

Is anyone aware of a method that would suit my needs here? The language I’m using is PHP, graph is created by GD and data is coming from MySQL, so optimizations to some of those are welcome.

The data is in this format:

Datetime               Value
2005-01-30 00:00:00    35.30
2005-01-30 01:00:00    35.65
2005-01-30 02:00:00    36.15
2005-01-30 03:00:00    35.95
...

And the resulting graph currently looks like this:

alt text http://www.ulmanen.fi/stuff/graph-sample.png

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    It seems to me that 1 in 200 is pretty serious data loss, and if those 200 values that should be represented with one value on the graph aren’t close enough to be meaningfully substituted with an average, you have yourself a problem. If average isn’t good enough, you must find a criterium to tell what data is more significant and should be included, and we can’t help you with it because we don’t know what kind of data it is, its statistical properties, or why any value would be more significant than the other. With those additional info, maybe a more specific answer could be given.

    EDIT: After looking at the graph, it seems that you need both minimum and maximum in a given interval, because the dark blue area are values between those two, correct? Maybe you can take 100 values and make a graph from minimum, maximum, and average, so that every point in graph is made with 6 instead of 200 values, or something like that.

    • 0