Take the 2-minute tour ×

I have a program written in C# and some parts are writing in native C/C++. I use doubles to calculate some values and sometimes the result is wrong because of too small precision. After some investigation i figured out that someone is setting the floating-point precision to 24-bits. My code works fine, when i reset the precision to at least 53-bits (using _fpreset or _controlfp), but i still need to figure out who is responsible for setting the precision to 24-bits in the first place.

Any ideas who i could achieve this?

share|improve this question
add comment

3 Answers

up vote 13 down vote accepted

This is caused by the default Direct3D device initialisation. You can tell Direct3D not to mess with the FPU precision by passing the D3DCREATE_FPU_PRESERVE flag to CreateDevice. There is also a managed code equivalent to this flag (CreateFlags.FpuPreserve) if you need it.

More information can be found at Direct3D and the FPU.

share|improve this answer
I would suggest changing this to be the "blessed" answer. –  akauppi Jul 30 '09 at 7:21
add comment

What about a binary search by partitions into your program and determining which calls reduce the precision?

share|improve this answer
I think binary search might be used in some parts, but this should only affect when floats or doubles are used in search keys. –  Andreas Roth Jul 29 '09 at 15:33
I think you misunderstand, Andreas. He's suggestting you comment out bits of existing code and calls until you find the one call that is getting your precision hosed on you. One good way to start is to just comment out half your code. If it still fails, it's in the uncommented calls. Keep narrowing it down that way until you are down to one call. –  T.E.D. Jul 29 '09 at 15:55
Follow a value through your program to see where it loses precision. Binary search means to start at a point roughly in the middle of the data flow and narrow it down from there. –  starblue Jul 29 '09 at 15:55
ok, now i understood. –  Andreas Roth Jul 29 '09 at 16:01
Gave you a -1 because this case is obvious, and not needing "binary search". Greg Hewgill has the right answer, imho. –  akauppi Jul 30 '09 at 7:21
show 1 more comment

Is your code using DirectX or XNA at all? I've certainly heard that there are problems due to that - some DirectX initialization code (possibly only in the managed wrapper?) reduces the precision.

share|improve this answer
I'm using DirectX in my program. Can you give me a hint which DX function would reduce the precision? Or any link to documentation? –  Andreas Roth Jul 29 '09 at 15:31
I haven't used it myself, so this is just hearsay - but I believe it's in the initialization code for DirectX. Quite what will go wrong if you change it back, I don't know... –  Jon Skeet Jul 29 '09 at 16:09
Yes you're absolutly right. You have to specify D3DCREATE_FPU_PRESERVE when you initialize the DX device. –  Andreas Roth Jul 30 '09 at 10:36
add comment

Your Answer


