system/freebsd-src
system/freebsd-src
1
0
Fork 0
mirror of https://github.com/freebsd/freebsd-src synced 2024-10-05 16:10:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

Quick fix for stack buffer overrun in rev.1.13. Oops. The prec == 1

Browse source 
arg to __kernel_rem_pio2() gives 53-bit (double) precision, not single
precision and/or the array dimension like I thought.  prec == 2 is
used in e_rem_pio2.c for double precision although it is documented
to be for 64-bit (extended) precision, and I just reduced it by 1
thinking that this would give the value suitable for 24-bit (float)
precision.  Reducing it 1 more to the documented value for float
precision doesn't actually work (it gives errors of ~0.75 ulps in the
reduced arg, but errors of much less than 0.5 ulps are needed; the bug
seems to be in kernel_rem_pio2.c).  Keep using a value 1 larger than
the documented value but supply an array large enough hold the extra
unused result from this.

The bug can also be fixed quickly by increasing init_jk[0] in
k_rem_pio2.c from 2 to 3.  This gives behaviour identical to using
prec == 1 except it doesn't create the extra result.  It isn't clear
how the precision bug affects higher precisions.  113-bit (quad) is
the largest precision, so there is no way to use a large precision
to fix it.
This commit is contained in:
Bruce Evans 2005-11-23 02:06:06 +00:00
parent 2010798dab
commit 33f8f56e09
Notes: svn2git 2020-12-20 02:59:44 +00:00 
svn path=/head/; revision=152706
1 changed files with 4 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
lib/msun/src/e_rem_pio2f.c
View file

@ -62,7 +62,7 @@ pio2_1t = 6.07710050650619224932e-11; /* 0x3DD0B461, 0x1A626331 */
int32_t __ieee754_rem_pio2f(float x, float *y)
{
double z,w,t,r,fn;
double tx[3];
double tx[3],ty[2];
int32_t e0,i,nx,n,ix,hx;
GET_FLOAT_WORD(hx,x);
@ -98,9 +98,9 @@ pio2_1t = 6.07710050650619224932e-11; /* 0x3DD0B461, 0x1A626331 */
tx[2] = z;
nx = 3;
while(tx[nx-1]==zero) nx--; /* skip zero term */
n = __kernel_rem_pio2(tx,&z,e0,nx,1,two_over_pi);
y[0] = z;
y[1] = z - y[0];
n = __kernel_rem_pio2(tx,ty,e0,nx,1,two_over_pi);
y[0] = ty[0];
y[1] = ty[0] - y[0];
if(hx<0) {y[0] = -y[0]; y[1] = -y[1]; return -n;}
return n;
}