Original Article


https://doi.org/10.5005/jp-journals-10054-0180
Indian Journal of Medical Biochemistry
Volume 25 | Issue 2 | Year 2021

Perception of Measurement Uncertainty among Laboratorians and Clinicians in Indian Scenario


Kavyashree P Siddaramegowda 1 , Anitha Devanath 2

1,2Department of Biochemistry, St. John’s Medical College, Bengaluru, Karnataka, India

Corresponding Author: Anitha Devanath, Department of Biochemistry, St. John’s Medical College, Bengaluru, Karnataka, India, Phone: +91 9900144602, e-mail: anitha.d@stjohns.in

How to cite this article Siddaramegowda KP, Devanath A. Perception of Measurement Uncertainty among Laboratorians and Clinicians in Indian Scenario. Indian J Med Biochem 2021;25(2):60–64.

Source of support: Nil

Conflict of interest: None

ABSTRACT

Aim and objective: Measurement uncertainty (MU) calculation for quantitative parameters is a mandatory requirement as per ISO 15189:2012. The concept of MU and its applicability is still ambiguous although the terminology has been around for more than two-and-a-half decades. Since accreditation bodies are aligned to ISO 15189:2012, it is interesting to understand the extent of awareness of MU and perception of its usefulness in our clinical setting.

Materials and methods: A questionnaire-based survey was conducted to understand the awareness and perception of MU along with interviews and focused group discussions. Prior training on how to use MU in a clinical setting was given before the survey. Responses to the questionnaire were analyzed using Microsoft Excel.

Results: The majority of laboratorians were aware of the terminology and confident to explain the use of MU during test result-related queries. However, there were challenges anticipated such as complexity in calculating the ranges, incorporation in laboratory information system, and acceptability by patient population. Both laboratorians and clinicians felt that MU helps in analyzing patient results more accurately and this process of change would require more time for better acceptance.

Conclusion: We conclude from our study that laboratory consultants with enough knowledge of MU can confidently introduce and implement MU in their daily practice. Clinicians were willing to interpret results with MU provided it was documented alongside the test report especially for the critical parameters which is obviously the challenging aspect for the labs.

Clinical significance: The feasibility of the introduction of MU alongside patient’s report is useful in interpreting critical parameters and provides a scientific evidence for consideration in a change of patient management rather than an arbitrary subjective analysis of serial monitoring of results.

Keywords: Measurement uncertainty, MU implementation, Perception of MU, Qualitative research, Serial result monitoring.

INTRODUCTION

Measurement uncertainty (MU) is defined as “a parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurand.” 1 In simpler terms, MU is given as an interval around a reported laboratory result that specifies the location of the measured value with the given probability which is a non-negative parameter. 2 A measurement result accompanied by a quantitative statement of its uncertainty gives a complete picture of the result dispersion that helps to decide adequacy for its intended purpose and ascertains consistency with similar results. There are multiple guidelines by national standardization institutes, professional and accreditation bodies that explain methods for estimation and expression of MU. 38

Measurement uncertainty calculation is mandatory for laboratories that follow the ISO 15189:2012 standard guidelines; clause 5.5.1.4, ISO 15189:2012 states that:

“The laboratory shall determine measurement uncertainty for each measurement procedure in the examination phase to report measured quantity values on patients” samples. 9

Labs opting for accreditation must prepare an MU document as a mandatory requirement. In this study, we intend to evaluate the awareness, feasibility, and application of the concept of MU among lab personnel and clinicians. Measurement uncertainty has been deemed to have the following applications: First, for laboratory professionals, it gives information about the quality of the result, providing evidence of compliance with analytical performance characteristics. It helps to improve the quality of care. Second, for physicians/clinicians (and patients, sometimes), it may help in the interpretation of test results, especially when values are compared with either previous value for the same patient or reference intervals or clinical decision limits, providing objective information. 10 It also helps in the acceptance of result transferability between various laboratories.

MATERIALS AND METHODS

Study Design

This study employed a qualitative research design that involved conducting 30 questionnaire-based surveys, 30 individual interviews, and 3 focus group discussions. The study method was reviewed and approved by Institutional Ethical Review Board. This study was initiated and carried out as a part of a pilot project for the implementation of MU in our tertiary care hospital. Though it was initiated as a hospital-based study, it was extended to the majority of the lab consultants specialized in Biochemistry and clinicians across India.

Subject Selection

Subjects were selected through purposeful sampling to ensure that there is basic knowledge about the interpretation of test results. Subjects for the study comprised lab consultants, clinicians at the level of senior residents, assistant professors, and professors. Subjects were selected to ensure a proper mix of clinical/professional experience that was based on the number of years of training.

Survey Based on Questionnaire

Subjects who agreed to complete the questionnaire survey were included in the study. Initially, we gave training material followed by a questionnaire to evaluate the understanding and perception of the concept of MU. In this group, target subjects were laboratory consultants and senior residents working in accredited labs from academic institutions or corporate-based labs. Incomplete and ambiguous responses to the questionnaire were excluded.

Individual Interview

Thirty individual interviews were conducted with clinicians that comprised of senior residents and assistant professors. The training material was given with details on calculation and application of MU (Tables 1 and 2: Training material: Calculation and application of MU). A ready reckoner of values was provided to the clinician for reference. Clinicians willing to apply MU for at least five patients were included. After a gap of 3 weeks, clinicians were interviewed to understand the challenges faced by them due to MU application.

Focus Group Subjects

Three focus group discussions were held, and the group comprised of clinicians (with varying levels of experience-professor, assistant professor, senior resident) and laboratory personnel (investigator). Investigator was doing the notetaking during the discussion. The group size varied from 3 to 5. Training material and information pertaining to the usefulness of MU were provided before the discussion. All focus group discussions were held in a tertiary care hospital.

CONTENT DEVELOPMENT

Questionnaire for Survey

Questions were framed in simple English to elicit responses that would provide insight into the awareness, depth of understanding, and ability to implement (Table 3: questionnaire). The questionnaire was distributed among ten laboratory personnel that comprised lab consultants to validate the content, phrasing, and sequence of questions. Reliability analysis revealed >0.7% of Cochrane’s alpha coefficient. Then, it was released for the survey. Thirty participants consented to complete the survey.

Individual Interview Guide

The guide for an interview was based on a questionnaire used for the survey with few modifications (Table 4: questionnaire for interview) to elicit responses for challenges encountered during its implementation. The length of the interview time was an average of 20 minutes. Responses were recorded during the interview and completed immediately thereafter.

Table 1: Calculation and expression of MU
MU = 1.96 × SD (SD is the standard deviation or standard uncertainty)
Example: Intermediate imprecision for serum creatinine: SD = 0.05 mg/dL
    MU (absolute value) for serum creatinine: 1.96 × 0.05 = 0.098 mg/dL
    How to express MU?
MU is expressed as test result ± MU
Example: Test result of 1.3 mg/dL for serum creatinine, MU is expressed as
1.3 mg/dL ± 0.098 mg/dL or dispersion of result is 1.2–1.4 mg/dL (rounded off to first decimal)
Table 2: Application of MU for patient values
  • Comparison of a patient value with a previous value of the same type to differentiate whether it is different from the previous value
    Intermediate imprecision for serum creatinine: SD = 0.05 mg/dL; MU is 0.098 or 0.1 mg/dL
    Example: Patient previous value of serum creatinine: 1.6 mg/dL; dispersion of result is 1.5–1.7 mg/dL
          Patient current value of serum creatinine: 1.3 mg/dL; dispersion of result is 1.2–1.4 mg/dL
          The result interval does not overlap. Hence, both values are different.
  • Comparison of a patient value with a clinical decision value
    Intermediate imprecision for serum sodium: SD= 2.14 mEq/L; MU is 4.28 mEq/L
    Example: Patient value is 124 mEq/L; dispersion of result is 119.7 to 128.3 mEq/L
          Medical decision limits: lower critical limit−115 mEq/L
                              Upper critical limit–150 mEq/L
Result interval does not overlap with the medical decision limit. Hence, the current patient value will not be considered under the critical decision limit.

Focus Group Guide or Moderator’s Guide

Content or questions were developed based on experiences drawn from individual Interviews. Time taken for group discussion was an average of 40 minutes. Responses were recorded, discussed, and summarized after each discussion.

DATA ANALYSIS

After each session, all responses were typed and reviewed by investigators to identify the main theme of each discussion. The experiences and challenges in understanding and implementation were documented. A questionnaire-based survey was evaluated based on the responses with respect to the proportion of subjects.

RESULTS

The questionnaire-based survey, individual interviews, and focus groups involved 75 participants–42 clinicians with varying years of experience (2–15 years) in clinical practice and 33 lab consultants at the level of lab heads, senior, and junior consultants (Table 5). Four main consensus themes emerged from the discussions, namely, (1) knowledge and awareness of MU, (2) knowledge and understanding of the calculation of MU, (3) advantages and challenges in Implementation of MU, and (4) clinician suggestions and requirements with regards to MU.

Table 3: Questionnaire for lab consultants
  • Are you aware of measurement uncertainty?
  • Is it easy to calculate MU?
  • Is it difficult to understand the concept of MU?
  • Is it difficult to explain the concept of MU to technicians?
  • Should technicians be trained to interpret MU?
  • Do you think MU should be incorporated as a part of the patient test report?
  • Would you be confident to sign out the report with MU?
  • Do you think MU incorporation in the patient test report showcases the confidence of the laboratory?
  • Would it be confusing to clinicians to have access to such information?
  • Would it be confusing to patients to have access to such information?
  • Do you think the patient would lose confidence in lab reports after knowing about MU?
  • Do you think clinicians would lose confidence in lab reports after knowing about MU?
  • Do you think MU could help in explaining the variant reports over a period of time?
  • Do you think MU would help the lab in explaining to patients the reliability of test reports?
  • If you had a choice, would you still calculate MU?
Table 4: Questionnaire for interview
  • Did you apply the concept of the uncertainty of measurement?
  • Is it easier and user-friendly to apply the calculations?
  • Is it time-consuming to recall and apply the values for an individual patient?
  • Would it be ideal to have a printout of probable MU of analytes alongside the patient result?
  • Do you think we should incorporate this as a part of the printed report?
  • Will this add to your existing knowledge to know whether a patient’s result is different from his/her previous report?
  • Do you think this knowledge will be confusing to apply?
  • After having known this concept, will it cloud your judgment about lab results?
  • Will you apply this knowledge in your daily clinical practice?
  • If yes, does it help you in judging the prognosis of the patient?
  • Would it be redundant to incorporate this information in the patient report?
  • Would it cause confusion to the patient to have this component of MU in the patient report?
  • Do you want to know the MU for all the analytes?
  • Would it be more useful to know MU for critical parameters?
  • Would it be more useful to know MU for parameters that have narrow medical decision limits?
Table 5: Participant characteristics (n = 75)
Lab consultants
    Lab heads (> 10-year experience) 15
    Lab senior consultants (5–10-year experience) 10
    Lab junior consultants (< 5-year experience)   8
Clinicians
    Senior consultants (professors, associate professors) 12
    Junior consultants (assistant professors, lecturers) 30

Theme 1: Knowledge and Awareness of MU

Lab Consultants

The majority of lab consultants (97%) were aware of the concept of MU though a small percentage (3%) were completely unaware. It was perceived as a new statistical concept that did not apply to small and newly established labs. Few consultants from accredited labs were aware of the concept and felt that MU was a part of the accreditation documentation process.

Clinicians

The concept of MU and its utility was not a common knowledge among clinicians. Only two clinicians expressed interest and were aware of MU. The majority of them expressed that MU was not applicable in clinical practice and felt that it was probably essential in lab practice for lab personnel’s perusal.

Theme 2: Knowledge and Understanding the Calculation of MU

Lab Consultants

Lab consultants expressed that it was easy to calculate MU and felt that calculation should be carried out by technicians. Forty-four percent of lab consultants believed that MU training for technicians should become a part of their regular training curriculum.

Clinicians

Clinicians opined that MU value was a common factor for all analytes. When it was explained to them that each analyte has specific MU, they expressed that it was difficult to apply to every analyte through calculated MU was provided for each parameter. Calculation of MU per se was considered as an easy formula.

Theme 3: Advantages and Challenges in Implementation of MU

Lab Consultants

Lab consultants felt that MU implementation was a mandatory requirement of the accreditation process but had little practical input. Forty-four percent of lab personnel perceived that it would be a challenge to train technical staff to calculate and interpret MU. Forty-six percent of participants perceived that incorporation of MU in patient lab reports would enhance the reliability of test results and helps in a better understanding of the test results. Sixty percent of the participants felt that reports with uncertainty will showcase the confidence of the laboratory about the quality of the result. The majority (80%) of them opined that MU helps in distinguishing two consecutive values for the patient and aids in meaningful scientific judgment.

Seventy percent of participants felt confident to sign the reports with MU while they were skeptical about acceptance of MU by clinicians and 43% felt that it might confuse patients. This could eventually result in a loss of confidence in lab reports since it increases the complexity and time taken to interpret the reports with MU.

A small percentage of lab consultants (20%) were concerned with the change in MU and the frequency of such changes. Whether such changes (if significant) would impact the treatment outcome and reliability factor on laboratory results. If MU were to be incorporated in the test report, then, changes in MU would add to the confusion for patients with two consecutive reports with different MU values. Furthermore, incorporation of MU component in Laboratory Information System alongside test results would require complex programming capabilities of the vendor to retain both MU values (previous and newly changed MU values).

Clinicians

The majority of clinicians expressed concern with regards to the calculation of MU for each analyte though it was explained and provided with reading material. Only 10% of the clinicians showed interest in MU values for all the analytes. Sixty-seven percent of the participants felt that knowledge about MU is confusing and needs more efficient training to interpret the same in daily practice. Forty-three percent of them felt that the applicability of MU is user-friendly but time-consuming to recall and apply to the patient’s results, especially in a busy outpatient setup. Most of the clinicians could not apply the concept because it was an additional engaging mathematical calculation with appropriate analyte-specific MU value recall. Seventy percent of them felt that theoretically, it helps in judging whether the patient result is different from his/her own previous reports. But in a few scenarios wherein the patient gets his lab reports from different labs every time, it would be difficult to apply this concept since it is not a standard established practice across all labs. Fifty percent of the participants felt that this concept will cloud the judgment about lab results. Clinicians expressed concern as to whether the theoretical gap in two consecutive values (lab calculated MU) mirrors the clinical condition of the patient.

Senior clinicians were not willing to introduce MU into their clinical practice for the interpretation of test results. And since the majority of the decisions on patient management were directed by senior clinicians, it was a definite challenge for juniors to implement in patient management. Most junior clinicians expressed concern about the versatility in the interpretation of lab reports whenever there are changes in MU values.

Theme 4: Clinician Suggestions and Requirements with Regard to MU

Twenty-seven percent of the clinicians were unwilling to apply/try this concept in their daily clinical practice. They believed, it was redundant to incorporate this information in the patient report, because clinicians often correlate the test result with the patient’s health status and never rely only on the laboratory results. 80–90% of clinicians wanted to know MU for critical parameters and those parameters which have narrow decision limits. Fifty percent of the clinicians opined that it will be ideal to have MU printed alongside patient results, more so, in the form of ranges and obtained value rather than in the form of a percentage.

DISCUSSION

Continual efforts toward reliable and dependable lab reports have led to an emphasis on traceability, method performance, quality assurance, and quality of test results. Measurement uncertainty provides insight into the quality of test results released by the laboratory. These focus group discussions, interviews, and a questionnaire-based survey was done to explore the perceptions and acceptance of MU among laboratory personnel and clinicians. We restricted the end-users to lab personnel and clinicians since this concept is still undergoing metamorphosis and would appear alien to the patient population and nursing team.

The concept of MU is still under a lot of debate with regards to the inclusion of bias and its uncertainty in MU calculation and acceptability limits for every quantitative parameter. Though these concepts are quite clear in a metrological and electrochemical laboratory, there are lacunae in medical testing laboratories. Despite these challenges, we wanted to know the current mindset of the lab personnel and clinicians.

Laboratories calculate MU as a part of ISO 15189:2012 requirement and have very little interest in its utility. Though lab consultants felt it was easy to understand, it was viewed as a task to be performed by lab technicians. Focus on technician training and understanding capability was one of the major aspects during discussions. It was perceived as a difficult task to achieve and implement in day-to-day practice. There were overwhelming positive views on the quality of reports when MU was included, however, major apprehension was expressed to incorporate it, even in footnotes for reports. Though the concept of MU was clear, there was ambiguity in using it since acceptance/tolerance levels for MU are yet to be established. Serious concern was to convince clinicians about MU–it was perceived as a laborious task since it would be a time-consuming, statistical approach that could be of little interest to clinicians. Another unsurmountable task was to educate and convince patients who belong to varying degrees of literacy. The fear was that lab reports might be misunderstood as either wrong or unreliable. It was perceived that the inclusion of MU might result in a loss of confidence in lab reports. Since the practice of MU inclusion is not uniform across all labs in India, it would add to the confusion and patient non-acceptance of lab reports.

It was difficult to recruit senior clinicians to answer the questionnaire since they expressed that it did not add value to their clinical judgment. Senior clinicians were not easily amenable to the suggestion of including MU for report interpretation in their daily practice. They substantiated that “clinicians assess the patient’s condition and interpret lab reports” in their practice. We wanted them to evaluate lab results with the inclusion of MU and further compare, monitor, and validate the patient’s clinical status. Senior clinicians were not convinced to implement since it would be time-consuming and change in clinical practice that has not been time-tested. MU was perceived as a statistical alteration to the result that was unacceptable. On the contrary, junior clinicians were willing to train and understand the concept. They were willing to apply and interpret reports when a ready reckoner was provided. However, it was perceived as an extra effort to refer to ready reckoner during a busy OPD schedule which was a hurdle in implementing on a large scale. Implementation of MU in the wards was more challenging since decision-making on patient management was directed by senior clinicians along with juniors as a team. Since senior clinicians were not completely convinced of the utility of MU, juniors were skeptical to introduce these concepts. During interview and focus group discussions, few senior clinicians were ready to implement it in their schedule, provided, it was incorporated in the patient report. There were major concerns on whether MU reflects a true clinical picture especially while considering and comparing consecutive values. However, there were no doubts with regards to theoretical scientific explanation but the logistics of applying the MU in daily practice was challenging such as the incorporation of MU alongside the patient test result. Clinicians felt that if there was enough published literature to prove that MU mirrors the clinical condition for monitoring consecutive values, it was useful in judging critical parameters and for parameters with narrow medical decision limit (MDL). When MDL was discussed as a guideline for decision-making, there was concern with its age-specific applicability. It was clearly indicated that MDL should not be included in the reports along with MU since it may not go together with clinical practice.

In 2004, Plebani 11 formulated a proposal on the communication of MU to clinicians. They added two aspects (a) TE obtained in their laboratory according to the imprecision [data from the internal quality control (IQC) at a concentration closer to the decision level] and bias (data from the external quality assurance schemes), (b) the Reference Change Value (RCV) for measurands primarily used in patient monitoring (e.g., tumor and bone markers), based on the imprecision and the biological variation (data from the Westgard database). In his report, he explained there was initial concern by some physicians, particularly related to the term “total error” which was interpreted in a negative sense, most users expressed satisfaction for this additional information, particularly regarding the RCV. He reported that a huge interest was expressed by students attending the medical degree and post-graduate courses, named after a series of teaching and educational initiatives on the concept of biological variation, quality specifications, and related performance characteristics.

In a subsequent study by Plebani et al. 10 reported on “when and how to communicate MU to physicians”. They described three different scenarios to add MU in laboratory reports. The different scenarios apply to the type of information usually included in laboratory reports to facilitate the interpretation of results, that is: (a) the measurand Reference Interval, (b) diagnostic cut-offs and decision limits, and, finally, (c) the RCV. 12 Another study done by Ayyildiz 13 in Turkey titled “The importance of measuring the uncertainty of the second generation of total testosterone analysis states that the individual MU results for each test should be given together with the test results to the clinician and the patient after finding out the significant difference in change of clinical practice with respect to testosterone interpretation”.

There are very few published studies to compare our study design. The introduction of MU alongside the test report across all labs might be useful and helpful for clinicians to make an informed decision. Uncertainty in a measurement quantity is a result of our incomplete knowledge of the factors that influence the measured quantity. Since there is no uniformity across labs, it might pose a challenge while comparing results from different labs when the patient is monitored over a longer period of duration. This would be beneficial when the sequential assessment is from the same lab. Testing laboratories should regularly inspect all elements of the test method and the conditions prevailing during its application in order to evaluate the uncertainty associated with a test result.

CONCLUSION

We conclude from our study that laboratory consultants with enough knowledge of MU can confidently introduce and implement MU in their daily practice. Clinicians were willing to interpret results with MU, provided, it was documented alongside the test report especially for the critical parameters which, is obviously the challenging aspect for the labs. However, it would require a great deal of cooperation and acceptability of the end-users for it to be a part of routine practice and earn the status of a report attribute.

CLINICAL SIGNIFICANCE

The feasibility of the introduction of MU alongside patient’s report is useful in interpreting critical parameters and provides a scientific evidence for consideration in the change of patient management rather than an arbitrary subjective analysis of serial monitoring of results.

REFERENCES

1. Working Group1 of the Joint Committee for Guides in Metrology. Evaluation of measurement data - guide to the expression of uncertainty in measurement. 1st ed. JCGM 2008; 100, Available at: http://www.iso.org/sites/JCGM/GUM/JCGM100/C045315e-html?csnumber=50461 .Accessed on 17th July 2020.

2. Rifai N, Horvath AR, Wittwer C, ed. Tietz textbook of clinical chemistry and molecular diagnostics. St. Louis, Missouri: Elsevier; 2018. p. 1867.

3. Clinical and Laboratory Standards Institute (CLSI). Expression of measurement uncertainty in laboratory medicine – Approved Guideline. CLSI document C51-A. Wayne, USA: CLSI; 2012.

4. The Royal College of Pathologists of Australasia. Uncertainty of measurement. Guideline No. 2/2004. Available at: https://hercwules.files.wordpress.com/2013/07/rcpa-uncertainty.pdf .Accessed June 15th 2020.

5. United Kingdom Accreditation Service. The expression of uncertainty and confidence in measurement traceability. M3003. 3rd ed., 2012. Available at: http://www.ukas.com/download/publications/publications-relating-to-laboratory-accreditation/M3003_Ed3_final.pdf .Accessed June 15th 2020.

6. National Pathology Accreditation Advisory Council (NPAAC). Requirements for the estimation of measurement uncertainty. 2007 ed. Available at: http://www.health.gov.au/internet/main/publishing.nsf/content/B1074B732F32282DCA257BF0001FA218/$File/dhaeou.pdf .Accessed on 17th July 2020.

7. American Association for Laboratory Accreditation. Policy on estimating measurement uncertainty for ISO 15189 testing laboratories.P903, 2014. Available at http://www.a2la.org/policies/15189_P903.pdf .Accessed 18th July 2020.

8. Bell S, The beginner’s guide to uncertainty of measurement. Available at: http://publications.npl.co.uk/npl_web/pdf/mgpg11.pdf .Accessed 10th June 2020.

9. International Organization for Standardization (ISO), ISO 15189:2012 Medical Laboratories – Requirements for Quality and Competence. Geneva, Switzerland: International Organization for Standardization; 2012.

10. Plebani M, Sciacovelli L, Bernardi D, et al. What information on measurement uncertainty should be communicated to clinicians, and how? Clin Biochem 2018;57:18–22. ISSN 0009-9120 10.1016/j.clinbiochem.2018.01.017.

11. Plebani M. What information on quality specification should be communicated to clinicians, and how? Clin Chim Acta 2004;346(1):25–35. DOI: 10.1016/j.cccn.2004.03.019.

12. Padoan A, Sciacovelli L, Aita A, et al. Measurement uncertainty in laboratory reports: a tool for improving the interpretation of test results. Clin Biochem 2018;57:41–47. DOI: 10.1016/j.clinbiochem.2018.03.009.

13. Ayyildiz SN. The importance of measuring the uncertainty of second-generation total testosterone analysis. Int J Med Biochem 2018;1:34–39.

________________________
© The Author(s). 2021 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted use, distribution, and non-commercial reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.