Quality Control and Quality Assurance Practices

 

Cervical Cytology Practice Guideline

Approved by the ASC Executive Board November 10, 2000

 

 

VII. Quality Control and Quality Assurance Practices

 

Quality Control is defined as a system for verifying and maintaining a desired level of quality in an individual test or process.  Quality control activities span the testing process from the moment of specimen collection until the time the physician receives the report.  Quality Assurance (QA) is defined by the College of American Pathologists as systematic monitoring of quality control results and quality practice parameters to assure that all systems are functioning in a manner appropriate to excellence in health care delivery.98  Quality assurance is a coordinated system designed to detect, control and prevent the occurrence of errors and, ultimately, to further a clinician’s ability to appropriately care for his or her patient. A number of quality control/quality assurance measures for cytopathology have been specified by the Clinical Laboratory Improvement Amendments of 1988.99  All quality assurance processes must be described and documented in a quality assurance program in the laboratory.

VII.A.  Pre-analytical Quality Control

Each laboratory must perform and maintain records of routine quality control relating to specimen receipt, preparation and staining.  Most of these activities are required by lab accreditation agencies and include such things as review of stain quality and maintenance records, microscope and instrument maintenance, as well as instrument calibration records.100

VII.B.  Screening and Reporting of Gynecologic Specimens

Federal regulations require that the individual examining a gynecologic cytology specimen be a qualified cytotechnologist or pathologist in a certified laboratory.67  These individuals may examine up to 100 slides per 24 hours (average 12.5 slides/ hour) and in not less than eight hours.  This number is not a performance target but a maximum allowed by law. Pathologists are limited by this ceiling when they perform primary screening.  Each laboratory must establish individual workload limits for each cytotechnologist. 69  These limits must be reviewed every six months by the Technical Supervisor of the lab and re-assessed using lab defined performance standards.  The record of slides reviewed by the primary screening cytotechnologist or pathologist must be documented and retrievable for inspectors during the retention period prescribed by CLIA ’88 or applicable state law.  Cytotechnologists and pathologists must also maintain work logs for any primary screening site (in cases of multiple site employment), again, for the applicable retention period.  As discussed in section VI, all specimens must be reported using descriptive nomenclature; use of a numerical reporting system alone is unacceptable.101

VII.C.  Review of Abnormal Gynecologic Cases

 

A cervical cytology specimen initially evaluated by a cytotechnologist as reactive, reparative, atypical, premalignant, or malignant must be referred to a pathologist for final interpretation and final report.  Discordance between pathologist and cytotechnologist interpretation is often used as a basis for identifying areas for continuing education. Peer review is often included in a quality assurance program. Multiple people may review difficult or interesting cases for educational and interpretive purposes.  Seeking the opinion of an outside consultant may be considered for unusually difficult cases with significant clinical implications.  Documentation of all reviews is essential for quality assurance monitoring.

 

VII.D.  Rescreening of Negative Cases

 

CLIA ’88 regulations specify that at least 10% of samples interpreted as negative by each cytotechnologist be re-screened by a pathologist or a qualified supervisory cytotechnologist prior to reporting.  Specimens from women considered to be at increased risk for cervical cancer must be included in the review process. Risk status may be determined by review of patient history provided by the clinician on the current requisition. The laboratory must have a clearly defined policy of its definition of high risk as well as its method for random selection of cases.102  Automated re-screening of negative cases has different requirements. (See VIII.D. )

VII.E.  Cytology-Histology Correlation and Clinical Follow Up

 

The laboratory must compare all pre-malignant and malignant gynecological cytology reports with subsequent histopathology, if available, and determine the causes of any discrepancy.  Cyto-histologic correlation can be a helpful educational tool used to refine methods of evaluation for both cytology and biopsy specimens. The correlation process should be documented in the laboratory quality assurance program.  Cyto-histologic correlation may be performed prospectively at the time of histologic review with integration of the correlation into the biopsy report.  Negative biopsy specimens in the context of recognized SIL or cancer by cytology often indicates a surgical sampling discrepancy.103  Comments regarding such cyto-histologic discordance in the surgical pathology report may be helpful in directing further patient management.  Correlation may also be performed retrospectively.  The laboratory must have a clearly defined policy regarding the methods used for cyto-histologic correlation.

If histologic material is not available, the laboratory may attempt to obtain follow-up material or information on patients. This is frequently achieved by sending a letter to the ordering physician requesting follow up information.

VII.F.  Retrospective Reviews

 

Federal regulations stipulate that all negative cervical cytology obtained within the last five years must be reviewed when a new high-grade squamous intraepithelial lesion or carcinoma is detected by cytology.  This review includes all available negative smears in the laboratory (either on site or in storage.)  If significant discrepancies are detected that would affect current patient care, the clinician must be notified and an amended report issued.104  It is up to the technical supervisor of the laboratory to define significant discrepancy in the laboratory standard operating procedure manual.  Retrospective reviews rarely detect abnormalities that affect current patient care.72  Therefore, amended reports are almost never indicated.  However, documentation of the fact that the review occurred should be made separately in internal quality assurance records. Where the review does not result in the issuance of a corrective report, CLIA does not require that specific interpretive discrepancies be documented.   Retrospective reviews are subject to the biasing effect of knowledge of outcome, and this fact should be kept in mind during any such review.  The main benefit derived from 5- year retrospective review is education of the laboratory staff.

Bias due to knowledge of clinical outcome, context of slide examination and hindsight all plague retrospective reviews.  Every reasonable effort should be made to minimize bias when reviewing cases/slides for laboratory or individual performance evaluation.  There are a number of methods to attempt this including:

  • Review by multiple individuals,
  • Review without knowledge of clinical outcome,
  • Review of the index case embedded in a slide sequence containing a range of normal and abnormal cases.105

VII.G.  Measures of Screening Performance

 

Cervical cytology is a highly successful screening test.  Cervical cytology is limited (as are all screening tests) by both false positive (FP) and false negative (FN) results.  A false positive is defined as a “positive” test result for a woman who does not have a cervical abnormality.  “Positive” results are variably defined in the medical literature; however, squamous or glandular intraepithelial lesions or cancer are the most reproducible benchmarks defining a positive result.  There are multiple reasons for false positive cytology.  For example, LSIL may be present at the time of the screening Pap test and the lesion may have regressed prior to biopsy, or a small lesion may not have been sampled with colposcopically directed biopsies or ECC.103 False positives are likely to occur at some level because of the difficult, subjective, interpretive character of cytologic evaluation, and due to pressures to minimize false negative results.

A false negative is defined in this document as a negative cervical cytology test result in a woman with a cervical squamous or glandular intraepithelial lesion or cancer.  The false negative rate for high grade intraepithelial lesions likely to progress to cancer and for invasive cancer itself is of greatest concern to all parties involved in the screening process.  False negative results may be a consequence of (a.) Patient sampling by the clinician or (b.) Laboratory screening or interpretation.  Sampling false-negatives occur when abnormal cells from the lesion are not collected or are not transferred to the slide.  A laboratory screening or interpretive false negative is one in which abnormal cells are present on the slide, but are not identified by screening or are misinterpreted after being noticed during screening.

The false negative rate is the sum of lesions missed in sampling plus the false negative proportion (FNP.)  The FNP is the measure of the laboratory component of false negative results and is defined as the number of false negative reports divided by the total number of women screened who have a cervical abnormality (False Negative Proportion = False Negative reports/True Positive reports + False Negative reports).106107108

       FN

 FNP =

 _________
   TP + FN

The value of determining the FNP for a laboratory is widely acknowledged; however, precise calculation of the FNP requires both 100% re-screening of negative cases and unachievable 100% accuracy. The accuracy of rescreening is the major variable that affects the calculation.  In everyday practice, the FNP may be estimated based on rescreening a sample of cases selected at random.  The best estimates of true false positive and false negative rates are achieved from large prospective studies in which all slides are independently reviewed and differences of opinion are resolved by an independent panel of cytologists.109  Based upon data collected in the medical literature, it may be extremely difficult to reduce the FNP below 5 to 10%.110  The false negative proportion calculated for a laboratory represents an estimate of the staff’s average screening sensitivity.  If sampling false negatives are added to the laboratory FNP, the overall false negative rate of cervical cytology may approach 20% or higher.111 

The threshold of abnormality used to define FN and TP must be consistent and every effort to reduce bias should be undertaken. For laboratory and individual performance, a false negative threshold of either ASCUS or LSIL may be used. An LSIL threshold is preferred because the degree of reproducibility of an ASCUS/AGUS interpretation is low.84112

CLIA ’88 mandates that a laboratory must evaluate individual performance in comparison to overall laboratory performance.  Regulations do not mandate any specific method of evaluation. Most frequently used measures include: random rescreening, targeted rescreening of specific patient groups, seeding abnormal cases into the screening and rescreening pools, and retrospective rescreening of negative cervical cytology specimens from patients with a current high grade abnormality. Retrospective rescreening evaluates past rather than current performance and is therefore difficult to statistically standardize for comparison of screening performance.  Statistical measures may include comparison of an individual’s FNP to that of the overall laboratory. Regardless of the method used the laboratory should establish performance expectations, document performance in comparison to these expectations, and have a program for corrective action when individuals do not meet the laboratory’s specific requirements.

 

VII.H.  Proficiency Testing and Continuing Medical Education

 

Proficiency testing has been mandated under CLIA ‘88 for individuals examining gynecologic specimens.113  To date, a national system has not been devised.  However, a number of state and private programs provide proficiency evaluation.  Examples include:

  1. State of Maryland Gynecologic Cytopathology Proficiency Program  (HCFA approved)
  2. New York State Cytopathology Proficiency Testing Program
  3. CAP Interlaboratory Comparison Program in Cervicovaginal Cytopathology
  4. CytoQuest ® Glass Slide Program from Midwest Institute for MedicalEducation (MIME)
  5. CheckSample ®, CheckPath ® and STAR® Programs from the American Society of Clinical Pathologists

Liquid-based cervical cytology specimens should be included in proficiency testing programs for laboratories that use this methodology.

Ongoing education is a requirement for proficiency in cytology.  This requirement can be fulfilled by participation in proficiency testing, intradepartmental slide review sessions, attending workshops and symposia, teaching cytotechnology students, pathology residents and fellows, independent study, and community outreach programs.  To maintain professional licensure, some states and professional societies have varied requirements for continuing medical education.

VII.I.  Variability in Practice

The total percentage of negative cases rescreened, and selection method will vary among laboratories. Some labs may randomly select 10% of the negative smears from a combination of both high risk and non-high risk patients. Other labs may select 10% of non-high risk cases in addition to some or all high-risk cases for re-screening. Since accuracy of rescreening has a major impact on a laboratory’s estimate of its screening false-negative rate, efforts to optimize the accuracy of rescreening are as important as efforts to optimize the accuracy of primary screening. This should be taken into account in a laboratory’s assignment of rescreening duties.  Laboratories using automated screening devices at a minimum must follow the manufacturers’ directions that have been approved by the FDA and deemed compliant with CLIA regulations according to HCFA.


(Click Here to Advance to the Next Chapter)

 

Bibliography/References

67  Clinical Laboratory Improvement Amendments of 1988; Final Rule. Federal Register.   Feb. 28 1992; 57: 493.1483.

69  Clinical Laboratory Improvement Amendments of 1988; Final Rule. Federal Register.  Feb. 28, 1992; 57: 493.1257(b).

72  Jones BA, Davey DD.  Quality management in gynecologic cytology using interlaboratory comparison.  Arch Pathol Lab Med 2000; 124: 672-681.

84  Robb JA.  The “ASCUS” swamp Diagn Cytopathol 1994; 11: 319-320.

98  Quality Improvement Manual in Anatomic Pathology. Chicago: College of American Pathologists; 1993.

99  Clinical Laboratory Improvement Amendments of 1988; Final Rule. Federal Register. Feb. 28, 1992; 57: 493.1257.

100  Clinical Laboratory Improvement Amendments of 1988; Final Rule. Federal Register. Feb. 28, 1992; 57: 493.1202 (c) (3).

101  Clinical Laboratory Improvement Amendments of 1988; Final Rule. Federal Register. Feb. 28, 1992; 57: 493.1257(e)(2).

102  Clinical Laboratory Improvement Amendments of 1988; Final Rule. Federal Register.  Feb. 28, 1992; 57: 493.1257(d)(1)(i).

103  Tritz DM, Weeks JA, Spires SE, Sattich M, Banks H, Cibull ML, Davey DD. Etiologies for non-correlating cervical cytologies and biopsies. Am J Clin Pathol 1995; 103:594-7.

104  Clinical Laboratory Improvement Amendments of 1988; Final Rule. Federal Register. Feb. 28, 1992; 57: 493.1257(d)(3).

105  Renshaw AA. Analysis of error in calculating the false-negative rate in the interpretation of cervicovaginal smears: the need to review abnormal cases.  Cancer Cytopathol 1997; 81:264-71.

106  Nagy GK. False negative rate – a misnomer, misunderstood and misused. Acta Cytol 1997; 41:778-780.

107  Krieger PA, Naryshkin S. Random rescreening of cytologic smears: A practical and effective component of quality assurance programs in both large and small cytology laboratories. Acta Cytol 1994; 38:291-8.

108  Krieger PA, Cohen T, Naryshkin S.  A practical guide to Papanicolaou smear rescreens. Cancer Cytopathol 1998; 84:130-137.

109  Wilbur DC, Prey MU, Miller WM, Pawlick GF, Colgan TJ.  The AutoPap System for Primary Screening in cervical cytology.  Acta Cytol 1998; 42: 214-220.

110  Koss LG. Cervical (Pap) smears. New directions.  Cancer Suplement 1993; 71:1406-1412.

111  DeMay RM. The Pap Smear.  In:  The Art and Science of Cytopathology. Chicago: ASCP Press. 1996; 143-147.

112  Raab SS. Can glandular lesions be diagnosed in Pap smear cytology? Diagn Cytopathol 2000; 23:127-133.

113  Clinical Laboratory Improvement Amendments of 1988; Final Rule. Federal Register. Feb. 28, 1992; 57: 493.945.