Psychometric properties of chronic low back pain diagnostic classification systems: a systematic review

Abstract

Objectives

To identify and critically appraise studies evaluating psychometric properties of functionally oriented diagnostic classification systems for Non-Specific Chronic Low Back Pain (NS-CLBP).

Methods

This review employed methodology consistent with PRISMA guidelines. Electronic databases and journals: (PubMed, EMBASE, Cochrane, PEDro, CINAHL, Index to chiropractic literature, ProQuest, Physical Therapy, Journal of Physiotherapy, Canadian Physiotherapy and Physiotherapy Theory and Practice) were searched from inception until January 2020. Included studies evaluated the validity and reliability of NS-CLBP diagnostic classification systems in adults. Risk of bias was assessed using a Critical Appraisal Tool.

Results

Twenty-two studies were eligible: Five investigated inter-rater reliability, and 17 studies analyzed validity of O’Sullivan’s classification system (OCS, n = 15), motor control impairment (MCI) test battery (n = 1), and Pain Behavior Assessment (PBA, n = 1). Evidence from multiple low risk of bias studies demonstrates that OCS has moderate to excellent inter-rater reliability (kappa > 0.4). Also, two low risk of bias studies support of OCS-MCI subcategory. Three tests within the MCI test battery show acceptable inter- and intra-rater reliability for clinical use (the "sitting knee extension," the “one leg stance,” and the “pelvic tilt” tests). Evidence for the reliability and validity of the PBA is limited to one high bias risk study.

Conclusions

Multiple low risk of bias studies demonstrate strong inter-rater reliability for OCS classification specifically OCS-MCI subcategory. Future studies with low risk of bias are needed to evaluate reliability and validity of the MCI test battery and the PBA.

Appendices

Appendix 1

Search strategies of the searched databases and journals

Database/journal	Last citation no	Keywords
PubMed	364	(((((Non specific OR non-specific OR nonspecific OR mechanical))) AND ((low back pain OR simple backache OR lumbar strain OR spinal degeneration))) AND ((clinical test OR clinical examination OR clinical sign))) AND ((valid* OR reliabl*)) simple search
EMbase	738	(clinical) AND (test* OR exam* OR sign*) AND (non-specific OR nonspecific OR 'non specific' OR mechanical OR simple) AND (low back pain OR back pain OR LBP) AND (reliab* OR valid*) in English only and limited to human plus searching in EMbase only
Cochrane	92	(Non specific or non-specific or nonspecific or mechanical) and (low back pain or simple backache or lumbar strain or spinal degeneration) and (clinical test or clinical examination or clinical sign) and (valid* or reliabl*) in search manager choose in Trials, Methods Studies, Technology Assessments and Economic Evaluations (Word variations have been searched)
PEDro	226	Non specific low back pain (abstract and title) in advanced search (method clinical trials)
CINHAL	286	(Non specific OR non-specific OR nonspecific OR mechanical) AND (low back pain OR simple backache OR lumbar strain OR spinal degeneration) AND (clinical test OR clinical examination OR clinical sign) AND (valid* OR reliabl*) in advanced search
ProQuest	358	(("non specific" OR "non-specific" OR "nonspecific" OR "mechanical back pain") AND ("back pain" OR "lumbar strain" OR "simple backache") AND ("clinical test" OR "clinical examination" OR "clinical sign") AND ("valid*" OR "reliab*")) AND la.exact("ENG")
Physical therapy journal	460	"non specific" "non-specific" "nonspecific" "mechanical back pain" "back pain" "lumbar strain" "simple backache" "clinical test" "clinical examination" "clinical sign" "valid*" "reliab*"
Chiroindex	79	"non specific" "non-specific" "nonspecific" "mechanical back pain" "back pain" "lumbar strain" "simple backache" "clinical test" "clinical examination" "clinical sign" "valid*" "reliab*"
Australian journal of physiotherapy	54	Non specific in Title/Abs/Keywords OR nonspecific inTitle/Abs/Keywords OR non-specific in Title/Abs/Keywords AND Low Back Pain inTitle/Abs/Keywords OR Mechanical low back pain in Title/Abs/Keywords OR simple backache in Title/Abs/Keywords
Canadian physiotherapy In advanced search	113	Non specific OR non-specific OR nonspecific OR mechanical AND low back pain AND clinical tests OR clinical examination OR clinical sign AND valid* OR reliabl*
physiotherapy theory and practice journal	113	Non specific OR non-specific OR nonspecific OR mechanical AND low back pain AND clinical test OR clinical examination OR clinical sign AND valid* OR reliabl*

Appendix 2

Systematic review critical appraisal tool (Reproduced from Brink and Louw (2011))

Item 1: If human subjects were used, did the authors give a detailed description of the sample of subjects used to perform the (index) test on?

Why the criterion should be evaluated: The validity and reliability of a test will be affected by the sample characteristics or composition, and therefore, the study has to report on the sample characteristics because the validity and reliability scores will then only be applicable to that particular population. A study does not contribute to validity and reliability testing if the subjects were not recruited appropriately

This item can be scored yes if:

1 the sample characteristics (e.g., height, weight, age, diagnosis and symptom status) were described or the manner of recruiting subjects was stated or if selection criteria were applied

If none of the above have been described or if insufficient information was provided, select “no.” If inhuman or inanimate objects were used, select N/A

Item 2: Did the authors clarify the qualification, or competence of the rater(s) who performed the (index) test?

Why the criterion should be evaluated: The amount of experience of the rater(s), performing the (index) test, will influence the validity and reliability scores and needs to be explained

This item can be scored yes if:

1 the rater(s) characteristics (e.g., qualification, specialization and amount of experience using the instrument under investigation) have been described

If the above have not been described or insufficient information was provided, select “no”

Item 3: Was the reference standard explained?

Why the criterion should be evaluated: The index test scores need to be compared to the scores obtained from the reference standard in order to test validity, and therefore, the reference standard needs to be explained appropriately

This item can be scored yes if:

1 the reference standard is likely to produce correct measurements;

2 the reference standard is the best method available; and

3 details (name of the instrument, references to the accuracy of the instrument) of the reference standard are reported

If none of the above is applicable to the reference standard’s description, then select “no”

Item 4: If inter-rater reliability was tested, were raters blinded to the findings of other raters?

Why the criterion should be evaluated: When raters have access to the findings of other raters, it compromises the quality of the reliability testing procedure by inflating the agreement among the raters, and therefore, blinding needs to be performed

This item can be scored yes if:

1 it is stated that the raters were blinded to each other’s findings or if a description that implies that the raters were blinded was reported

If no information is provided, then select “no.” If intra-rater reliability was examined, then select “N/A”

Item 5: If intra-rater reliability was tested, were raters blinded to their own prior findings of the test under evaluation?

Why the criterion should be evaluated: If raters have knowledge of their prior own findings, it will influence the findings of their repeated measurements and could inflate the rater agreement, and therefore, appropriate measures, depending on the characteristics or the study design of the research study, need to be applied to ensure blinding

This item can be scored yes if:

1 rater(s) has/have examined the same subjects on more than one occasion, it should be stated whether the rater(s) was/were blinded to the subjects they have examined previously

If insufficient information is provided, then select “no.” If inter-rater reliability was examined, then select “N/A”

Item 6: Was the order of examination varied?

Why the criterion should be evaluated: If the order is varied, in which the raters examine the subjects when inter-rater reliability is tested, it reduces the risk of systematic bias. If the order is varied in which subjects are examined by one rater when intra-rater reliability is tested, it reduces the risk of the rater recalling the previous test scores and reduces bias

This item can be scored yes if:

1 the order in which subjects were tested varied between raters if inter-rater reliability was tested;

2 the order of subjects was varied when intra-rater reliability was tested

If insufficient information is provided, then select “no.” If varied order of examination is unnecessary or impractical (e.g., rater(s) digitizing or reading X-rays) then select “N/A”

Item 7: If human subjects were used, was the time period between the reference standard and the index test short enough to be reasonably sure that the target condition did not change between the two tests?

Why the criterion should be evaluated: The index test and the reference standard should be performed at the same time; however, this is not always possible. It becomes important to know whether it is possible that the test variable did not change between the two tests, otherwise it will affect the index test’s validity performance

This item can be scored yes if:

1 result from the index test and the reference standard were collected on the same subjects at the same time;

2 a delay between measurements occurs, it is important that the target condition should not change between measurements

If the time period between performing the index test and the reference standard was sufficiently long that the target condition may have changed between the two tests or if insufficient information is provided, then select “no.” If inhuman or inanimate objects were used, then select N/A

Item 8: Was the stability (or theoretical stability) of the variable being measured considered when determining the suitability of the time interval between repeated measures?

Why the criterion should be evaluated: For reliability, the test variable should not change between repeated measures, otherwise it will decrease the amount of agreement obtained between and within the rater(s)

This item can be scored yes if:

1 the stability of the variable is known or reported, and reviewers then decide on an appropriate time interval between repeated measures (stability of a test variable can only be determined if there is a reference standard);

2 there is no reference standard, then the reviewers should agree upon the theoretical stability of the variable and decide on an appropriate time interval between repeated measures

If insufficient information is provided, then select “no”

Item 9: Was the reference standard independent of the index test?

Why the criterion should be evaluated: If the reference standard and the index test are not independently performed, then the index test cannot replace the reference standard on its own

This item can be scored yes if:

1 it is clear from the study that the index test did not form part of the reference standard

If it appears that the index test formed part of the reference standard, then select “no”

Item 10: Was the execution of the (index) test described in enough detail to permit replication of the test?

Why the criterion should be evaluated: Variations in the execution of the reference standard and the (index) test might affect the agreement between the two tests and it is also important to be able to replicate the same study procedure in another setting when needed

This item can be scored yes if:

1 the study reported a clear description of the measurement procedure (e.g., the positioning of the instrument or rater and execution sequence of events);

2 citations of methodology were supplied

The extent to which details is expected to be reported depends on the ability of different procedures to influence the results and on the type of instrument or test under evaluation

If insufficient information is provided, then select “no”

Item 11: Was the execution of the reference standard described in enough detail to permit its replication?

Why the criterion should be evaluated: For the same reason as item 10

This item can be scored yes if:

1 the study reported a clear description of the measurement procedure (e.g., the positioning of the instrument or rater and execution sequence of events);

2 citations were supplied

If insufficient information is provided, then select “no”

Item 12: Were withdrawals from the study explained?

Why the criterion should be evaluated: The sample composition will influence the validity and reliability performance of the (index) test; therefore, it is important to know whether any withdrawals from the sample might have changed the composition of the sample

This item can be scored yes if:

1 it is clear what happened to all subjects who entered the study;

2 subjects who entered but did not complete the study are considered

If it appears that subjects who entered but did not complete the study were not accounted for or if insufficient information is provided, then select “no.” If inhuman or inanimate objects were used, then select N/A

Item 13: Were the statistical methods appropriate for the purpose of the study?

Why the criterion should be evaluated: The aim of validity and reliability studies is to report on an estimate of validity and reliability for the particular test and appropriate statistical methods need to be implemented in order to produce this estimate

This item can be scored yes if:

1 the analysis is appropriate in terms of the type of data (e.g., categorical, continuous and dichotomous);

2 statistical analysis for validity studies incorporates, for example means, differences between measurements, 95% confidence interval and ANOVA; and

3 statistical analysis for reliability studies incorporates, for example, interclass correlation coefficient and 95% confidence interval

If the analysis is not appropriate or if insufficient information was provided, then select “no”

Appendix 3

Classification processes of OCS