Methodology and Analysis of the ILCs Provided by INRIM from 2016 to 2018

Abstract

To assess the technical competence of calibration laboratories, national accreditation bodies (NABs) take into account the results of national and/or international proficiencies testing (PTs) including the inter-laboratory comparisons (ILCs). As the Italian NAB stopped acting as PT provider since 2015 due to the result of a peer assessment by the European co-operation for accreditation, the National Institute of Metrological Research (INRIM) reorganized itself and reoriented part of its activity to become a ILCs provider, thus supporting the calibration laboratories. This paper describes how INRIM has been organized as ILCs provider to guarantee the accreditation activity continuation and gives information about the main typologies of the provided ILCs and details on the analysis of the ILCs results in different metrological areas. From 2016 to 2018, INRIM provided 114 ILCs, in the areas of acoustics ultrasonic and vibration, electricity and magnetism, length (L), mass (M) and thermometry (T), involving 138 laboratories, for which 375 ILC reports were issued.

1 Introduction

The technical surveillance of calibration laboratories accredited according to [1] is important for the competitiveness of modern countries. In an effective technical surveillance, the execution of inter-laboratory comparisons (ILCs) is of great importance. In an ILC, the measurements of a surveyed laboratory are compared with the measurements of a reference laboratory to establish whether they are compatible. ILCs are therefore useful to establish the competence of secondary laboratories, operators skill, equipment suitability and the correctness of the dissemination process from national standards, normally maintained at the National Measurement Institutes (NMIs) [2,3,4,5,6,7]. With ILCs, the measurements compatibility can be verified among different laboratories or of different measurement methods. It is also possible to establish the equivalence of national standards among NMIs [8]. NMIs and secondary laboratories are at the top of the measurement system of each modern industrialized country. Each step of this chain must be under control to assure that final products are reliable. This control can be made also by means of ILCs that therefore are a strategic mean to assure the reliability of measurement systems supporting the high-tech industry.

2 The Italian Framework

According to [9], the Italian NAB ACCREDIA, as signatory of the multilateral EA agreement,¹ must demonstrate evaluating in an accreditation process, the laboratories' technical competence, also through their satisfactory participation in national and/or international ILCs. In the past, the Calibration Service in Italy (SIT) first, and after ACCREDIA, to finalize the accreditation applications of laboratories, provided itself ILCs, with the Italian NMIs technical support. EA² assessment in 2015 considered this practice “noncompliant,” thus, since that year, ACCREDIA ceased this activity. In 2015 were active 172 accredited laboratories in 267 measurement sectors [10], although a ILCs provider was not available. To compensate this lack and to guarantee the continuity of the accreditation assessments, INRIM, and in particular, the Technical Service for Accreditation of Laboratories Department (STALT) organized itself as ILCs provider. Since then, INRIM organizes ILCs in accordance with [11] and it is accepted by ACCREDIA as ILC provider being a NMI signatory of the CIPM-MRA arrangement.³ Therefore, INRIM has been included in the list of the EA LC Working Group on ILC for calibrations (EA LC WG ILC Calibration).

2.1 Traceability Assurance for ILCs

Usually, secondary Laboratories send their standards to INRIM (or to other NMIs) for periodical calibration. In this way, traceability chains from national standards till to final users are established. For example, Figs. 1 and 2 show typical traceability chains from INRIM to secondary electrical and thermal calibration laboratories. In the scheme of Fig. 1 a, high-precision digital multimeter (DMM) belonging to an accredited electrical laboratory is periodically calibrated at INRIM. Anyway, the traceability of the reference measurements of the ILCs provided by INRIM is guaranteed by the Italian law 11 August 1991, no. 273 “Establishment of the national calibration system” and it is in accordance with the document ILAC-P10: 2013 par. 2, point 1) and with the CMCs included in the CIPM MRA (www.kcdb.bipm.org). Italy has a total of 404 CMCs divided as follows: 115 for Electricity and Magnetism (EM), 100 for Temperature (T), 63 for Mass (M), 43 for Length (L), 42 for Acoustics Ultrasonic and Vibration (AUV), 20 for Photometry and Radiometry (PR), 12, nine for Time and Frequency (TF) and nine for Chemistry and Biology (QM).

Fig. 1

Traceability schemes from INRIM to secondary electrical laboratories through a high-precision DMM. In a, the DMM acts as reference standard, while in b, it acts as transfer standard

Fig. 2

Typical traceability scheme from INRIM to secondary thermal laboratories for calibration of thermometers

3 INRIM Organization for ILCs

INRIM, as Italian NMI, realizes and compares with other NMIs' primary measurement standards for all SI units except for the ionizing radiations field. It carries out scientific research focused on metrology, materials science and innovative technologies. NMI underpins the SI system, disseminates and transfers scientific results, technology and know-how to scientific, industrial and service users. Furthermore, it produces, and coordinates, even within the European Union, programs and international organizations, scientific and technological research activities, through its own facilities or in collaboration with universities and other public and private national and international entities. According to the customer needs, INRIM organizes ILCs for the physical quantities within its competence. To organize the ILCs-provider activity, INRIM chose a unique communication channel for customers through an electronic mailbox managed by an organizing contact. This was the input/output channel for ILC management activities (requests receipt, quotations submissions, order transmissions, questions and replies, information sharing, and so on). The aim has been to centralize in STALT the ILCs management and looking elsewhere, in other INRIM Departments or in other NMIs, for the reference measurements providers if not available in STALT. The technical part of the activity has been instead made by a technical contact (TC) whose task has been the management of the traveling standard, of the technical communications with the customer, the data processing and the drafting of the ILC report. A dedicated technical working group has been set up involving a member of the electrical, thermal and mechanical areas that are the metrological areas in which the largest number of secondary laboratories is accredited. This working group manages the technical aspects to operate in agreement with [11] drafting the templates of technical–economic quotation, LC protocol and ILC report, and their latest revisions are available for colleagues on the relevant INRIM shared area. In the following, the steps of an ILC are summarized:

• Quotation preparation and dispatch and order receipt;

• Protocol drafting and sending;

• ILC management;

• Data processing and report issue.

3.1 Quotation Editing and Sending and Order Receipt

The information of a new ILC is given sending the technical–economic quotation. The quotation form is filled by the TC, based on the needs of accredited laboratories in the relevant area, and transmitted to customers by the INRIM communication channel. The specific supply conditions indicated in the quotation form were established by the INRIM administration while TC defines the amount. This depends on:

• The costs of INRIM calibrations;

• The shipping costs incurred by INRIM;

• Staff engagement costs to draft the protocol, to monitor the traveling standards, to draft the ILC report;

• Possible compensation costs, when a traveling standard could no longer be utilized after the ILC. Any discounts are applied during invoicing. Figure 3 shows the process of the contract management.

  Fig. 3

  

  Outline of the process of an ILC quotation drafting and sending and order receipt

3.2 ILC Protocol

The ILC protocol is the document with technical and organizational details of the ILC including the calibrations calendar, and details of participants. It has to contain:

• Aim of the document;

• ILC provider Contacts;

• Provider of the reference measurements;

• Participant laboratories and ILC calendar;

• Traveling standards;

• Standards management;

• Execution of the activity;

• Presentation of the results;

• Used method to evaluate the results;

• Data use and confidentiality;

• References.

TC drafts the ILC protocol while a competent different and independent person carries out its review. Figure 4 shows the issue and dispatch process of the ILC protocol.

Fig. 4

Outline of the ILC protocol issue and dispatch to participants

3.3 ILC Management

TC applies the protocol from the issue to the final report. In detail, TC:

• Contacts the customers and organizes the shipment of the traveling standards;

• Is a reference point for ILC participants;

• Manages any unforeseen events (including changes to the calendar;

• Collects the results of participants and of the reference laboratory and the related documentation as required by the protocol;

• Performs the data evaluation;

• Issues the ILC report.

4 ILC Reports

For each ILC participant, an ILC report is issued to guarantee the results reservation. ILC reports have to contain:

• A summary;

• Details of the ILC provider;

• ILC contacts;

• Details of possible subcontracting;

• The ILC scheme;

• Management of the traveling standards including any damages;

• Execution of the activity;

• Reference values;

• Results of the participating laboratories;

• Evaluation of the results;

• Comments on the results (if applicable);

• Confidentiality statement;

• References.

5 Evaluation of the ILC Results

In the following, examples of results evaluation of ILCs between INRIM and secondary laboratories are reported. Results in forms of tables/figures can instead be found in some papers listed in references [2,3,4,5,6,7,8].

5.1 Electrical Case

The example deals with a bilateral ILC (i.e., between INRIM and one laboratory) consisting in the calibration of a multifunction electrical calibrator in the five low-frequency quantities, DC and AC voltage, DC and AC current and DC resistance. To establish the measurement compatibility between the INRIM and laboratory measurements, the evaluation of the results has to be carried out by determining the normalized error E_n, defined according to [11], and taking into account the case that the laboratory standards could have been calibrated by the laboratory that provided the reference measurements (INRIM). Therefore, the data have to be evaluated as follows: The error of the calibrator defined in (1) and (2), respectively, should be considered, as measurand, for INRIM and for the participating laboratory:

$$ E_{\text{I}} = \frac{{(m_{\text{I1}} - s) + (m_{\text{I2}} - s)}}{2s} $$

(1)

$$ E_{\text{L}} = (m_{\text{L}} - s)/s $$

(2)

where m_I1 and m_I2 are the INRIM measurements at the setting s, obtained before and after the measurements at the laboratory, while m_L indicates the value measured by the laboratory at the same setting s. The INRIM and laboratory results have to be defined as:

$$ E_{\text{I}} \pm U_{\text{I}} $$

(3)

$$ E_{\text{L}} \pm U_{\text{L}} $$

(4)

where U_I and U_L are their expanded uncertainties. From them, the standard uncertainties are obtained:

$$ u_{\text{I}} \cong \frac{1}{2}U_{\text{I}} \quad {\text{and}}\quad u_{\text{L}} \cong \frac{1}{2}U_{\text{L}} $$

(5)

The following difference for each measurement point has to be calculated:

$$ y = E_{\text{L}} - E_{\text{I}} $$

(6)

whose standard uncertainty is:

$$ u^{2} (y) = [u^{2} (E_{\text{I}} ) + u^{2} (E_{\text{L}} ))] - 2u(E_{\text{L}} )u(E_{\text{I}} ) \times r(E_{\text{L}} ,E_{\text{I}} ) $$

(7)

where r(E_I, E_L) is the correlation coefficient between the INRIM and laboratory errors. Finally, the normalized error E_n is:

$$ E_{\text{n}} = \frac{y}{{U_{y} }} $$

(8)

where U_y = 2u_y for a 95% confidence level. The ILC result is satisfactory if |E_n| ≤ 1, for each measurement point. Table 1 reports the results of an ILC for DC voltage in the 100 mV range.

Table 1 ILC results in the 100 mV range in DC voltage

5.2 Volume Case

The example deals with a multilateral ILC (i.e., between INRIM and some secondary laboratories) consisting in the calibration of a 100 ml pycnometer at 20 °C and of a 20-l standard tank at 15 °C and 20 °C (three volume measurements). The evaluation of the results of the participating laboratories has to be made by determining the E_n values. In this case, it can be assumed that the INRIM and laboratories’ measurements are independent. For each laboratory and for each standard, E_n has to be evaluated as follows:

$$ E_{\text{n}} = \frac{{\left| {\Delta V} \right|}}{U(\Delta V)} $$

(9)

where ΔV = V_Lab − V_INRIM and:

$$ U(\Delta V) = 2\sqrt {u_{{}}^{2} (V_{\text{INRIM}} ) + u_{l}^{2} (V_{\text{Lab}} )} $$

(10)

Table 2 reports the results for the 100 ml pycnometer for one participating laboratory.

Table 2 Evaluation of the volume comparison for the 100 ml pycnometer at 20 °C for one participating laboratory

5.3 Mass Case

The example deals with a bilateral ILC consisting in the calibration of eight mass standards from 20 mg to 50 kg. To evaluate the stability of the traveling standards during the comparison, the normalized error E_n between the INRIM measurements before m_Xi(I) and after m_Xi(F) the laboratory measurements has to be calculated. It has to be E_n| ≤ 1. The following differences have to be calculated:

$$ \Delta m_{Xi} = m_{Xi} \left( F \right) - m_{Xi} \left( I \right) $$

(11)

with

$$ U(\Delta m_{Xi} ) \, = \, 2\sqrt {\left[ {\frac{{U\left[ {m_{Xi} \left( I \right)} \right]}}{2}} \right]^{2} + \left[ {\frac{{U\left[ {m_{Xi} \left( F \right)} \right]}}{2}} \right]^{2} - 2r_{Xi} \left[ {\frac{{U\left[ {m_{Xi} \left( I \right)} \right]}}{2}} \right]\left[ {\frac{{U\left[ {m_{Xi} \left( F \right)} \right]}}{2}} \right]} $$

(12)

Then,

$$ E_{\text{n}} (I,F) = \frac{{\Delta m_{Xi} }}{{U(\Delta m_{Xi} )}} $$

(13)

To quantify the correlation coefficient r_Xi between INRIM measurements before and after for each mass standard i (where i = 1,…,8), the correlation effects have to be taken into account due to the:

• Uncertainty of the common mass standard used for the calibration;

• nonlinearity of the balance;

• operator and environment.

• The reference value is the arithmetic mean of the values measured before and after the laboratory measurements:

  $$ m_{Xir} = \, 0.5 \times \, \left[ {m_{X} i\left( I \right) \, + m_{X} i\left( F \right)} \right] $$

  (14)

The uncertainty component due to mass instability has also to be added in the uncertainty of the reference value:

$$ u_{{{\text{instab}}_{i} }} = \left| {\Delta m_{Xi} } \right|/\sqrt {12} $$

(15)

The expanded uncertainty of the reference value has to be evaluated taking into account the correlation coefficients r_Xi for all mass standards and the contribution due to the instability:

$$ U(m_{Xir} ) \, = \, 2\sqrt {\left[ {\frac{{U\left[ {m_{Xi} } \right]}}{2}} \right]^{2} + \left[ {\frac{{U\left[ {m_{Xi} } \right]}}{2}} \right]^{2} + 2 \, u_{{{\text{instab}}_{i} }} + 2r_{Xi} \left[ {\frac{{U\left[ {m_{Xi} } \right]}}{2}} \right]\left[ {\frac{{U\left[ {m_{Xi} } \right]}}{2}} \right]} $$

(16)

The laboratory results m_Xil and uncertainties U(m_Xil) for each mass standard have to be compared with reference values:

$$ \Delta m_{Xil} = m_{Xil} {-}m_{Xir} $$

(17)

For the uncertainty evaluation, the correlation coefficient r_Xil between INRIM and the laboratory measurements has to be considered as:

• r_Xil = 0.3 if the laboratory mass standards are traceable directly to INRIM;

• r_Xil = 0.1 if the laboratory mass standards are traceable to an accredited laboratory traced to INRIM;

• r_Xil = 0 if the laboratory mass standards are traceable to another NMI or to an external accredited laboratory. The expanded uncertainty is therefore U(∆m_Xil) = 2u(∆m_Xil) with:

  $$ u\left( {\Delta m_{Xil} } \right) \, = \sqrt {\left[ {\frac{{U\left( {m_{Xil} } \right)}}{2}} \right]^{2} + u^{2}_{{{\text{instab}}_{i} }} + \left[ {\frac{{U\left( {m_{Xir} } \right)}}{2}} \right]^{2} - 2r_{Xil} \left[ {\frac{{U\left( {m_{Xil} } \right)}}{2}\frac{{U'\left( {m_{Xir} } \right)}}{2}} \right]^{{}} } $$

  (18)

  where

  $$ U'\left( {m_{Xir} } \right) = \sqrt {\left[ {\frac{{U\left[ {m_{Xi} } \right]}}{2}} \right]^{2} + \left[ {\frac{{U\left[ {m_{Xi} } \right]}}{2}} \right]^{2} + 2r_{Xi} \left[ {\frac{{U\left[ {m_{Xi} } \right]}}{2}} \right]\left[ {\frac{{U\left[ {m_{Xi} } \right]}}{2}} \right]} $$

  (19)

  is the expanded uncertainty of the reference value without the instability component. Then,

  $$ E_{\text{n}} = \frac{{\Delta m_{Xil} }}{{U\left( {\Delta m_{Xil} } \right)}} $$

  (20)

Table 3 shows the results of one ILC according to the previous analysis.

Table 3 Evaluation of the mass comparison for one participating laboratory

5.4 Length Case

The example deals with a multilateral ILC consisting in the calibration of a 22-mm-diameter steel ball, a RTH hemisphere and a 50-mm-diameter Tesa ring (Fig. 3). The measurand is the deviation from the ideal roundness of the standards, RONt, which is obtained as difference between the maximum and minimum profile radii, with reference circumference evaluated with the least squares method.

The plane where the measured profile has to placed depends on the standard:

• For the sphere, it has to be parallel to the base of the support of the standard and has to be placed in correspondence with the maximum section of the standard (equator);

• For the hemisphere, it has to be parallel to the base of the support of the standard and has to be placed about 3 mm from the fixing ring of the standard;

• for the ring, the mid-height profile has to be considered.

Gaussian filtering has to be applied to the data, with a cut off frequency f_C = 50 UPR. The roundness error measured by INRIM with multiple orientation technique at the end of the circulation has to be considered the reference value. The analysis of the results follows the same treatment of the electrical case only considering the correlation coefficient between INRIM and laboratory measurements as 0.1.

5.5 Pressure Case

The example deals with a multilateral ILC consisting in the calibration of a pressure transducer with absolute method between 10 kPa and 130 kPa. To assess the stability of the traveling standard during the comparison, the pilot laboratory has to perform the calibration at least three times. Therefore, the reference value (p_i,ref) is the mean value of the calibrations. Its standard uncertainty is:

$$ u\left( {p_{{i,{\text{ref}}}} } \right) = \sqrt {u^{2} \left( {e_{{i,{\text{ref}}}} } \right) + u^{2} \left( {\text{drift}} \right)} $$

(21)

where e_i,ref is the INRIM measurement error, defined as relative difference versus the reference applied pressure, value in the ith measurement point and u(e_iref) is its standard uncertainty. u(drift) is the standard uncertainty of the drift evaluated as:

$$ u^{2} \left( {\text{drift}} \right) = \hbox{max} \left( {\frac{{\hbox{max} (e_{{i,j,{\text{INRIM}}}} ) - \hbox{min} (e_{{i,j,{\text{INRIM}}}} )}}{{\sqrt {12} }}} \right) $$

(22)

For each participating laboratory, the value of the measured pressure p_i,lab for a nominal pressure value pt_i is:

$$ p_{{i,{\text{lab}}}} = pt_{i} + e_{{i,{\text{lab}}}} $$

(23)

where

$$ u\left( {p_{{i,{\text{lab}}}} } \right) = u\left( {e_{{i,{\text{lab}}}} } \right) $$

(24)

is the standard uncertainty of the laboratory error. The values obtained by the laboratories and the reference ones have to be defined, respectively, as:

$$ p_{{i{\text{lab}}}} \pm \, U_{{i{\text{lab}}}} $$

(25)

$$ p_{{i{\text{ref}}}} \pm \, U_{{i{\text{ref}}}} $$

(26)

The analysis of the results follows the same treatment of the case in par. 5.1 but considers the correlation coefficient between INRIM and laboratory measurements as in par. 5.3.

6 Management of the Outcomes of the ILCs

Let us consider the example of a multilateral ILC in the field of low-frequency electrical quantities (DC and AC voltage, DC and AC current and DC resistance. In this kind of ILC, the allowed participants for each ILC are usually no more than ten laboratories to avoid too long times for the circulation, drift and damages to the traveling instrument (normally a high precision 8^1/2 digit DMM or a top-level multifunction calibrator). INRIM carries out its measurement before and after the circulation and, if necessary, in the mid-time of the circulation itself. When, during the evaluation of the results, it is observed that some measurements of one laboratory are compatible with those of INRIM, the very first thing is to ask the laboratory to check the correct transcription of their measurement results. After the evaluation of the results, INRIM provides, for each participant laboratory, an ILC report containing only its results for confidentiality reasons along with the E_n for each ILC measurement point. Each applicant or accredited laboratory, in the framework of the first accreditation or in the accreditation renewal, must show to its accreditation body its ILC report in which all the points out of compatibility are outlined. If |E_n| ≫ 1 in several points and in different quantities, the ILC result is clearly unsatisfactory and the laboratory must communicate this negative result to the accreditation body that will interrupt the accreditation or the accreditation renewal processes. The laboratory, after suitable internal verifications and corrective actions, for example to enlarge its or the requested CMC, has to repeat the ILC to resume the accreditation or the accreditation renewal. Instead, if |E_n| > 1 in only few points of the same quantity, the laboratory can make immediate corrective actions to submit to INRIM for evaluation. If INRIM considers adequate these corrective actions can update the laboratory report indicating the suitable corrective actions proposed by the laboratory". With this amended report, the accreditation body can decide to carry out an additional measurement comparison only in the points where |E_n| > 1 when the technical assessor will carry out the planned inspection visit to the laboratory itself. For other INRIM quantities, as thermal and mechanical ones, the number of laboratories admitted to the participation to ILCs is limited to 12–15 laboratories. For thermal and mechanical ILCs, the typical way of circulation is at” flower petal” in which the traveling standard comes back to INRIM for recalibration, after the calibration at three-four laboratories, to check its integrity and stability.

7 Summary of the INRIM Activity as ILCs Provider from 2016 to 2018

As the Italian Accredited Calibration Laboratories (at the end of 2019 about 194 plus five Reference Materials Producers) must submit their ILCs plan to ACCREDIA on a 4-year period, they usually ask INRIM to schedule their ILCs over that period for all the quantities for which they are accredited or ask accreditation. As consequence, from 2016 to 2018, INRIM provided 114 ILCs, in the metrological areas of acoustics ultrasonic and vibration (AUV), electricity and magnetism (EM), length (L), mass (M) and thermometry (T), involving 138 companies, for which 375 ILC reports were issued. Main customers are the national calibration accredited laboratories that have to demonstrate to the NAB their competence through ILC results, but also laboratories that have to validate their calibration methods or to provide evidence of their competence to customers. Occasionally, also foreign customers took part in the ILCs. Table 4 shows the main typologies of the provided ILCs along with indication of the traveling standards/instruments, measurand definitions, measurement methods, physical quantities, measurement ranges and uncertainties. It is possible to observe wide uncertainty ranges as, for each physical quantity or type of traveling instrument/standard, different level of ILCs can be provided according to the requested uncertainties for participation and to the employed traveling instrument/standard. This allows the laboratories to choose suitable ILCs to their accreditation status or improvement needs.

Table 4 Main ILCs typologies provided by INRIM from 2016 to 2018

Table 5 shows the no. of ILC reports issued from 2016 to 2018, including those in Time and Frequency area, while Fig. 5 reports the no. of provided ILCs for each metrological area in the same period.

Fig. 5

Roundness standards for the ILC

Table 5 ILC reports issued from 2016 to 2018, including the reports issued in the time and frequency area

Those reported in Table 4 are the ILCs routinely provided by INRIM. Nevertheless, several specific ILCs on request by laboratories or other customers were also provided. The participant laboratories carried out the measurements following their approved procedures written according to the measurement methods reported in Table 4.

Figure 6 shows the issued ILC reports divided by the metrological areas and by year.

Fig. 6

Provided ILCs from 2016 to 2018

Figure 6 shows that the number of provided ILCs in these 3 years is not decreased. Rather, the number of ILCs is still slightly growing despite several accredited laboratories participated to ILCs organized by foreign NMIs or by accredited ILCs provider.

8 Conclusions

After the ending of the ILCs-provider activity by ACCREDIA, INRIM replaced ACCREDIA as independent and qualified body in the national territory, acting as ILCs provider, mainly in order to guarantee continuity of the accreditation processes and the stability of the national calibration system. The activity of ILCs provider is made in accordance with [11]. Almost all of the ILC requests came from Italian accredited laboratories that have been managed by means of the INRIM communication channel or directly by the TCs. Future aims of this activity will be the publication on the INRIM Web site of the ILC program over a medium–long period and of the ILC directory in addition to that for calibration and testing activities. An analysis of the ILCs results obtained in these 3 years will be performed in order to make future improvements to this activity in both managing and technical aspects. A customers’ satisfaction investigation could also be made to obtain useful suggestions from customers.