Chapter 9 SpecificationsDC, Resistance, and Temperature Accuracy SpecificationsDC, Resistance, and Temperature Accuracy Specifications

± ( % of reading + % of range ) [1]

Includes measurement error, switching error, and transducer conversion error

Function

Range [3]

Test Current or

24 Hour [2]

 

 

 

 

Temperature

90 Day

1 Year

Coefficient /° C

 

 

Burden Voltage

23 ° C ± 1 ° C

23 ° C ± 5 ° C

23 ° C ± 5 ° C

0 ° C – 18 ° C

 

 

 

 

 

 

 

 

 

 

28 ° C – 55 ° C

DC Voltage

100.0000 mV

 

0.0030

+

0.0035

0.0040

+ 0.0040

0.0050

+ 0.0040

0.0005 + 0.0005

 

1.000000 V

 

0.0020

+

0.0006

0.0030 + 0.0007

0.0040 + 0.0007

0.0005 + 0.0001

 

10.00000 V

 

0.0015

+

0.0004

0.0020 + 0.0005

0.0035 + 0.0005

0.0005 + 0.0001

 

100.0000 V

 

0.0020

+

0.0006

0.0035 + 0.0006

0.0045 + 0.0006

0.0005 + 0.0001

 

300.000 V

 

0.0020

+

0.0020

0.0035 + 0.0030

0.0045 + 0.0030

0.0005 + 0.0003

 

 

 

 

 

 

 

 

 

 

 

Resistance [4]

100.0000 Ω

1 mA current source

0.0030

+

0.0035

0.008

+ 0.004

0.010

+ 0.004

0.0006 + 0.0005

 

1.000000 kΩ

1 mA

0.0020

+

0.0006

0.008

+ 0.001

0.010

+ 0.001

0.0006 + 0.0001

 

10.00000 kΩ

100 μA

0.0020

+

0.0005

0.008

+ 0.001

0.010

+ 0.001

0.0006 + 0.0001

 

100.0000 kΩ

10 μA

0.0020

+

0.0005

0.008

+ 0.001

0.010

+ 0.001

0.0006 + 0.0001

 

1.000000 MΩ

5 μA

0.002

+

0.001

0.008

+ 0.001

0.010

+ 0.001

0.0010 + 0.0002

 

10.00000 MΩ

500 nA

0.015

+

0.001

0.020

+ 0.001

0.040

+ 0.001

0.0030 + 0.0004

 

100.0000 MΩ

500 nA 10 MΩ

0.300

+

0.010

0.800

+ 0.010

0.800

+ 0.010

0.1500 + 0.0002

DC Current

10.00000 mA

< 0.1 V burden

0.005 + 0.010

0.030

+ 0.020

0.050

+ 0.020

0.002 + 0.0020

HP 34901A Only

100.0000 mA

< 0.6 V

0.010 + 0.004

0.030

+ 0.005

0.050

+ 0.005

0.002 + 0.0005

 

1.000000 A

< 2 V

0.050 + 0.006

0.080

+ 0.010

0.100

+ 0.010

0.005 + 0.0010

Temperature

Type

Best Range Accuracy [5]

Extended Range Accuracy [5]

 

Thermocouple

B

1100°C to 1820°C

1.2°C

400°C to 1100°C

1.8°C

0.03°C

 

E

-150°C to 1000°C

1.0°C

-260°C to -150°C

1.5°C

0.03°C

 

J

-150°C to 1200°C

1.0°C

-210°C to -150°C

1.2°C

0.03°C

 

K

-100°C to 600°C

1.0°C

-230°C to -100°C

1.5°C

0.03°C

 

N

-100°C to 1300°C

1.0°C

-220°C to -100°C

1.5°C

0.03°C

 

R

300°C to 1760°C

1.2°C

-50°C to 300°C

1.8°C

0.03°C

 

S

400°C to 1760°C

1.2°C

-50°C to 400°C

1.8°C

0.03°C

 

T

-100°C to 400°C

1.0°C

-240°C to -100°C

1.5°C

0.03°C

RTD

R0 from 49Ω

-200°C to 600°C

0.06°C

 

 

 

 

0.003°C

 

to 2.1 kΩ

 

 

 

 

 

 

 

 

 

Thermistor

2.2 k, 5 k, 10 k

-80°C to 150°C

0.08°C

 

 

 

 

0.002°C

[1]Specifications are for 1 hour warm up and 61⁄2 digits

[2]Relative to calibration standards

[3]20% over range on all ranges except 300 Vdc and 1 Adc ranges

[4]Specifications are for 4-wire ohms function or 2-wire ohms using Scaling to remove the offset. Without Scaling, add 1Ω additional error in 2-wire ohms function.

[5]1 year accuracy. For total measurement accuracy, add temperature probe error.

404