To improve standardization of headache disorder diagnoses, the International Classification of Headache Disorders (ICHD) was created and consists of chapters on primary headache disorders, secondary headache disorders, and painful cranial neuralgias/facial pain. However, the clinical application of the ICHD may be challenging for physicians who are inexperienced in headache medicine, the researchers noted.
Neurophysiological tests, neuroimaging, and blood-based biomarkers have also been used to aid diagnoses of primary headache disorders, although these methods have not replaced clinical interviews.
To aid in avoiding biases attributed to human error, data-driven approaches using machine learning have been recently tested in the medical field. In the current study, the researchers used real-world questionnaires from over 2000 patients to develop a machine learning approach specifically intended for use in headache disorders.
The questionnaire used in the study collected data on headache characteristics, disease course, associated symptoms, aura, medication use for headaches, past medical history, and social history. Information on participant age, sex, and body mass index (BMI) were also included.
Data from participants who answered the questionnaire in 2018 (n = 1286) were used as the training cohort, while data collected from participants in 2017 (n = 876) were used as the test cohort.
All headache disorders were classified into 7 groups: migraine, tension-type headache (TTH), trigeminal autonomic cephalalgia (TAC), epicranial headache (including primary stabbing headache and occipital neuralgia), thunderclap headache (TCH; including primary and secondary causes of TCH), other primary headache disorders, and secondary headaches other than those causing TCHs.
“We adopted a stacked model that consisted of 4 layers of binary XGBoost classifiers,” the researchers explained. “The first layer classified between migraine and others, the second layer classified between tension-type headache (TTH) and others, and the third layer classified between trigeminal autonomic cephalalgia (TAC) and others, and the fourth layer classified between epicranial and thunderclap headaches,” for a total of 5 subtypes.
Analyses revealed that for migraine, TTH, TAC, epicranial headache, and TCH, respectively:
- Stacked XGBoost classifier using the selected features attained an accuracy of 82%; sensitivity of 87%, 66%, 85%, 65%, and 64%; and specificity of 94%, 54%, 58%, 63%, and 57% for the 5 subtypes in the training cohort
- Baseline accuracy (ie, assigning all cases as the dominant subtype) was 67% in the training cohort
- Stacked XGBoost classifier using the selected features led to an accuracy of 81%; sensitivity of 88%, 69%, 65%, 53%, and 51%; and specificity of 95%, 55%, 46%, 48%, and 51% for the 5 subtypes in the test cohort
- Baseline accuracy (i.e., assigning all cases as the dominant subtype) was 68% in the test cohort
Researchers then compared XGBoost with other classifiers and found that although there were small differences in performances, XGBoost outperformed Random Forest, support vector machine, and k-nearest neighbor.
Despite the fact the approach’s accuracy in classifying headache disorders other than migraine was inferior to that in classifying migraine, the authors argued “our automated classification results could be still meaningful as the gold standard for the diagnosis of headache as a manual skillful application of the current classification criteria.”
Samples of each headache disorder other than migraine and TTH were insufficient for the training cohort. Therefore, headache disorders or syndromes other than migraine and TTH were merged into broader categories such as epicranial headaches or TCHs, which was not ideal for the detailed classification of second- or third-digit ICHD codes, marking a limitation to the study.
The authors cautioned the machine learning approach cannot replace physician-based diagnoses. But, “our results might be used to inform or assist physicians by pre-screening with the most important factors of the stacked classifier or increasing the accuracy of less-specialized provider,” they concluded.
Reference
Kwon J, Lee H, Cho S, et al. Machine learning-based automated classification of headache disorders using patient-reported questionnaires. Sci Rep. Published online August 20, 2020. doi:10.1038/s41598-020-70992-1