kCalculation Ranges, Number of Digits, and

Precision

The calculation range, number of digits used for internal calculation, and calculation precision depends on the type of calculation you are performing.

Calculation Range and Precision

Calculation Range

 

±1 10–99to ±9.999999999 1099 or 0

Number of Digits for Internal

15 digits

Calculation

 

 

 

 

 

 

 

 

 

Precision

 

 

In general, ±1 at the 10th digit for a single

 

 

 

calculation. Precision for exponential display

 

 

 

is ±1 at the least significant digit. Errors are

 

 

 

cumulative in the case of consecutive

 

 

 

calculations.

 

 

 

 

Function Calculation Input Ranges and Precision

 

 

 

 

Functions

 

 

Input Range

sinx

DEG

0 x 9109

 

RAD

0 x 157079632.7

 

GRA

0 x 11010

cosx

DEG

0 x 9109

 

RAD

0 x 157079632.7

 

GRA

0 x 11010

tanx

DEG

Same as sinx, except when x= (2n–1)90.

 

RAD

Same as sinx, except when x= (2n–1)π/2.

 

GRA

Same as sinx, except when x= (2n–1)100.

sin1x

0 x 1

cos1x

 

 

 

tan1x

0 x 9.9999999991099

sinhx

0 x 230.2585092

coshx

 

 

 

sinh1x

0 x 4.9999999991099

cosh1x

1 x 4.9999999991099

tanhx

0 x 9.9999999991099

tanh1x

0 x 9.99999999910–1

logx/lnx

0 x 9.9999999991099

10x

–9.9999999991099x  99.99999999

ex

–9.9999999991099x  230.2585092

'x

0 x  1  10100

x2

x 1 1050

1/x

x 1 10100 ; x G 0

3'x

x 1 10100

x!

0 x  69 (x is an integer)

nPr

0 n  11010, 0 r  n (n, r are integers)

1 {n!/(nr)!}  110100

nCr

0 n  11010, 0 r  n (n, r are integers)

1 n!/r!  110100 or 1 n!/(nr)!  110100

E-48