kCalculation Ranges, Number of Digits, and
Precision
The calculation range, number of digits used for internal calculation, and calculation precision depends on the type of calculation you are performing.
Calculation Range and Precision
Calculation Range |
| ±1 ⋅ | ||
Number of Digits for Internal | 15 digits | |||
Calculation |
|
| ||
|
|
| ||
|
|
|
| |
Precision |
|
| In general, ±1 at the 10th digit for a single | |
|
|
| calculation. Precision for exponential display | |
|
|
| is ±1 at the least significant digit. Errors are | |
|
|
| cumulative in the case of consecutive | |
|
|
| calculations. | |
|
|
|
| |
Function Calculation Input Ranges and Precision | ||||
|
|
|
| |
Functions |
|
| Input Range | |
sinx | DEG | 0 x 9109 | ||
| RAD | 0 x 157079632.7 | ||
| GRA | 0 x 11010 | ||
cosx | DEG | 0 x 9109 | ||
| RAD | 0 x 157079632.7 | ||
| GRA | 0 x 11010 | ||
tanx | DEG | Same as sinx, except when x= | ||
| RAD | Same as sinx, except when x= | ||
| GRA | Same as sinx, except when x= | ||
0 x 1 | ||||
|
|
| ||
0 x 9.9999999991099 | ||||
sinhx | 0 x 230.2585092 | |||
coshx | ||||
|
|
| ||
0 x 4.9999999991099 | ||||
1 x 4.9999999991099 | ||||
tanhx | 0 x 9.9999999991099 | |||
0 x | ||||
logx/lnx | 0 x 9.9999999991099 | |||
10x | ||||
ex | ||||
'x | 0 x 1 10100 | |||
x2 | x 1 1050 | |||
1/x | x 1 10100 ; x G 0 | |||
3'x | x 1 10100 | |||
x! | 0 x 69 (x is an integer) | |||
nPr | 0 n 11010, 0 r n (n, r are integers) | |||
1 | ||||
nCr | 0 n 11010, 0 r n (n, r are integers) | |||
1 n!/r! 110100 or 1 |