kCalculation Ranges, Number of Digits, and Precision

The calculation range, number of digits used for internal calculation, and calculation precision depends on the type of calculation you are performing.

Calculation Range and Precision

Calculation Range ±1 10–99to ±9.999999999 1099 or 0

Number of Digits for

15 digits

Internal Calculation

 

Precision

In general, ±1 at the 10th digit

 

for a single calculation. Precision

 

for exponential display is ±1 at

 

the least significant digit. Errors

 

are cumulative in the case of

 

consecutive calculations.

Function Calculation Input Ranges and Precision

Functions

 

Input Range

sinx

DEG

0 x 9109

 

RAD

0 x 157079632.7

 

GRA

0 x 11010

cosx

DEG

0 x 9109

 

RAD

0 x 157079632.7

 

GRA

0 x 11010

tanx

DEG

Same as sinx, except when x= (2n–1)90.

 

RAD

Same as sinx, except when x= (2n–1)π/2.

 

GRA

Same as sinx, except when x= (2n–1)100.

sin–1x

0 x 1

cos–1x

 

 

tan–1x

0 x 9.9999999991099

sinhx

0 x 230.2585092

coshx

 

 

sinh–1x

0 x 4.9999999991099

cosh–1x

1 x 4.9999999991099

tanhx

0 x 9.9999999991099

 

 

E-137