Re: storing floating point data with using float or double

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

You can store the values as fixed point values in shorts (Int16). You would be able to get a precision of three decimals if you'd like.

Multiply the values by 1000 and convert to short:

-5249
5988
1255
7584

The numbers you could store this way would range from -32.768 to 32.767.

Eat my shorts. ;)

Steve wrote:
Hi,

I've developed a testing application that stores formatted results in a database. Recently it was requested that I add the ability to generate graphs from the raw, un formatted test results (100,000+ float values)

I don't intend to store all of the 100,000 datapoints, but rather a subset of say 250. Due to the volume of testing that we need and the volume of results stored, I need to be VERY careful with data size and keep things as lean and small as possible.

Given the example data: (-5.2485, 5.9875, -1.2548, 7.5842)
Those samples represent our data value boundaries, they won't get any bigger than that, they are signed and they are floating point.

I don't need the precision of a float, the above values would be fine to be stored as:
-5.29
5.99
-1.25
7.58

A couple of ideas I have come up with is to try compressing the array of 250 floats and storing the compressed binary in the DB BLOB field. I also though about breaking the values into an sbyte and a byte like this:
-5.29

represented as:
sbyte l = -5;
byte r = 29;

This would save me to bytes from using a float. If I then compressed this "byte data" I could save even more space.

Anyway, I have never done any of this before and wanted to invite you guys to offer any suggestions or past experiences you might of had with doing something like this.

As you can see, I'm not real sure what I'm doing ;0)

Thanks for reading,
Steve

.

Relevant Pages

  • Re: Rounnding Issue
    ... In the database, if the datatype is a float, it will not contain EXACTLY 20.45 because that value CANNOT be stored in a float. ... A float is a binary fraction and asking a binary fraction to store 20.45 is like asking a decimal fraction to store 1/3. ...
    (microsoft.public.sqlserver.programming)
  • Re: Float comparison
    ... But a double value that is the result of evaluating an expression is ... the float lost the precision. ... expression and store it in a float, ...
    (comp.lang.c)
  • Re: Extremely wierd problem I just cannot explain...works in debug, does not work in release.
    ... a float or double should be used to store _only_ measured ... kimi no koto omoidasu hi nante nai no wa ... kimi no koto wasureta toki ga nai kara ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Float value changes unexpectedly when performing math on field
    ... Float fields, by definition, have some built-in imprecision. ... This is a side effect being able to store very large numbers in smaller spaces. ... So, ideally, if that is not what you want, you should NUMERIC or DECIMAL, but not float. ... rollover to zero so I have a rollover count and rollover point so I can ...
    (microsoft.public.sqlserver.server)
  • Re: storing floating point data with using float or double
    ... I don't intend to store all of the 100,000 datapoints, but rather a subset ... I don't need the precision of a float, the above values would be fine to ... A couple of ideas I have come up with is to try compressing the array of ... though about breaking the values into an sbyte and a byte like this: ...
    (microsoft.public.dotnet.languages.csharp)