BACKGROUND
Trauma remains a leading cause of mortality in children worldwide.1 The development of trauma registries has contributed to improving trauma outcomes in a variety of settings, including resource-poor environments.2 However, the impact of trauma registries is highly dependent on the quality of the data within.3 Although data quality can be evaluated according to a multifactorial paradigm, missing data is a basic parameter that can be used to easily assess the quality of data collection and effectiveness of a given data collection tool.3-6
We aim to establish a clinical and research partnership between HSR and MUHC for pediatric injury surveillance. This initiative entails the development and deployment of a PTR that can be used for the purposes of epidemiologic analysis, research, quality improvement, outcomes benchmarking and decision-making for resource allocation. In this pilot study, we set out to evaluate the readiness of HSR for PTR deployment. To do so, our objectives were to describe the epidemiology of pediatric trauma at HSR, to identify gaps in data collection using existing HSR health record systems, and to use the information from this pilot study to inform our future efforts to optimize the utility of a prospective PTR within our partnership. We hypothesize that missing data is significant in existing health records at HSR which may result in bias and negatively impact health care resource allocation decisions based on incomplete data.
MATERIALS AND METHODS
Study Setting
Hospital Dr. Sotero del Rio is a large urban referral center for pediatric trauma. It is located in South-Eastern Santiago, Chile, and serves a population of approximately 2 million people. The HSR is not a stand-alone pediatric hospital. Two main pavilions are located on the hospital campus, one of which is dedicated to the care of children. The pediatric pavilion has a separate emergency department, for patients 15 years-old and younger. The Chilean trauma system is regionalized such that HSR serves several districts of South-Eastern Santiago. This institution's emergency department uses free-text style open charts, rather than predetermined admission, history, and physical examination templates. Full pediatric general surgery and orthopedics coverage is available, while pediatric neurosurgery consultants provide coverage 6 days per week, on average. In addition, HSR is not a dedicated burn center. Electronic health records are used at HSR. A patient-specific medical record number can be used to browse multiple data interfaces which captures information on the different phases of patient care, such as, emergency department visits, hospitalizations, and ambulatory clinic visits.
Data Collection
After obtaining approval from the HSR institutional research Ethics Review Board, we obtained access to the electronic health records of all trauma patients who were seen at the HSR pediatric emergency department in the 6-month period between January 1st, 2015 to June 30th, 2015. Retrospective chart review was performed for patients who met inclusion criteria: Patients aged between 0 and 15 years who were seen by pediatric general surgery, orthopedics or neurosurgery for traumatic injury and who required hospitalization or transfer to another institution for definitive care (including return to hospital of origin with planned follow-up visit at HSR). Patients were excluded if they met the following exclusion criteria: Age greater than 15 or discharge home from ER. Variables were collected from existing health records according to a pilot PTR adapted from the McGill Centre for Global Surgery iTrauma Trauma Registry. The “iTrauma”, which is a trauma registry designed to capture essential data on adult trauma patients in differently-resourced environments. Though it was originally developed as a paper-based registry (Appendix 1), it has since been made available as an application available on iOS, the mobile operating system for Apple devices, and functions with or without an internet connection.7 The pilot PTR was adapted from iTrauma to capture relevant data for pediatric trauma patients and finally translated into Spanish and approved by pilot institution (Appendix 2). The variables collected fall into four major categories: (1) Demographics and logistics, (2) mechanism, (3) injury, and (4) outcomes. Missing data were recorded as not-available. After pilot data was collected, this was presented at the weekly pediatric surgery service rounds and qualitative feedback was sought for discussion.
Analysis
Summary statistics are reported as appropriate for all variables, including missing data. For a given variable, data missingness is expressed as a fraction of missing data points over total observations. For a given patient, data missingness is expressed as a fraction of missing data points over total number of variables. Potential predictors of individual patient data missingness were explored using univariate linear regression. These included season, weekend, gender, consultant and age. Variables that correlated with individual patient data missingness on univariate analysis (p < 0.25) were evaluated using multiple linear regression.
RESULTS
Descriptive Statistics
We included 351 patients, of which 234 (66%) were female. Median age was 5 (2—9). Approximately 48% of the patients were seen by pediatric general surgery, 28% were seen by neurosurgery and 24% were seen by orthopedics. A majority of the injuries (60.1%) occurred within the Puente Alto district, where HSR is located. The remaining 39.9% of patients were injured in surrounding five districts, including Pirque, San José de Maipo, La Granja, La Florida, La Pintana, and San Ramon. Approximately half the patients who presented, sustained a fall. The mechanisms and disposition of patients are presented in Graphs 1 and 2 respectively. Of all patients who were hospitalized, 83% underwent an operative intervention, and 26% were admitted to the intensive care unit at some point during their stay.
Per-variable Data Missingness
The range of per-variable missing data was 0.6 to 100%. Demographic data regarding means of hospital arrival and time of trauma were missing in upwards of 80% of cases. Physiologic variables were also missing in large proportions. Percentages of missing data are presented in Graph 3. Basic hemodynamic variables that one might expect to find recorded in any patient's chart, not to mention a trauma patient's chart, such as, blood pressure and pulse rate were missing in 72% of patients. Temperature, respiratory rate and need for supplemental oxygen were missing in upwards of 90% of patients. The least frequently missing physiologic variables were glasgow coma scale and oxygen saturation, at 61 and 37% respectively. However, anatomic injury data only had a 0.6% overall rate of missingness. Not surprisingly, follow-up data at 2 weeks was also problematic. It was identifiable in only 54% of patients despite integrated electronic health records.
Table 1
Coefficients from simple linear regression, representing expected change in percentage of per-patient missing data, along with p-values
Table 2
Coefficients from multiple linear regression model including variables achieving p < 0.1 on simple linear regression, representing expected change in percentage of per-patient missing data after adjustment for all other variables, along with p-values
| Variable | Coefficient | p-value |
|---|---|---|
| Gender (female vs. male) | −0.008 | 0.1512 |
| Season (winter vs. summer) | 0.077 | <0.01 |
| Consultant specialty (neurosurgery vs. general pediatric surgery) | −0.064 | <0.01 |
Predictors of Per-patient Data Missingness
Univariate linear regression analysis was performed using the following potential predictors of missing data: Gender, age, season, consultant specialty or weekend. Results of the univariate linear regression analysis are shown in Table 1. Because they were identified as potential independent predictors of per-patient missingness, the following variables were included in the multiple linear regression: Gender, season, and consultant specialty (Table 2). Among these, season and consultant specialty were found to be independent predictors of proportion of per-patient missing data. Specifically, a patient presenting in winter compared to summer, with all else being equal, is expected to have a 7.7% increase in proportion of missing data (p < 0.01). Similarly, a patient seen by neurosurgery upon presentation, with all else being equal, is expected to have a 6.4% decrease in proportion of missing data (p < 0.01).
DISCUSSION
We identified several possible explanations for the large amounts of missing data. We can speculate that the multiple interfaces of the electronic health record system, capturing patient data at various phases of care may be partly responsible, as their integration is not yet seamless. For example, emergency department records on one interface may be left incomplete by the consulting physician who plans to include a summarized version of the findings in her admission note on another interface. Furthermore, feedback at service rounds from local collaborators included anecdotal explanations that fundamental physiologic data are not necessarily recorded in the patient chart, even though they are consistently measured. Often, this is due to inadequate time in an overcrowded emergency department, particularly in the winter months when the number of visits for respiratory illness increases drastically. A quarterly administrative report produced by HSR indicates that although the number of emergency visits almost doubles in the winter months, the average wait time for patients to see a physician is only minimally increased, which reflects the expedited triage practices that this particular emergency department has adopted. The quantitative analysis we performed corroborates this subjective experience of the providers. In addition to season, we found that the specialty of the consultant who evaluated the child had a significant effect on the percentage of missing data recorded in the chart. One can only speculate as to the cause of the increased meticulousness of neurosurgeons in data recording, although liability concerns or lower relative volume of nontraumatic emergency consults may have contributed. Other potential predictors of data missingness, such as patient gender, day of the week (school day vs. weekend), and patient age were not significant on linear regression.
As trauma registries proliferate in LMICs,8,9 there have been a growing number of studies examining the quality of data within these registries.3,10-12 As stated by Porgo, et al, problems of data quality affect the reliability of said data in inform clinical decision-making, resource-allocation planning, and any quality improvement initiative.3 There has been widespread recognition that some existing and well-established trauma registries, such as, the International Trauma Registry in Latin America and the Caribbean regions, could benefit from integrating more rigorous data quality measures.10,11 It is important to note, however, that definitions of data quality vary significantly within the published literature, and methods described for evaluating such quality in trauma registries are inconsistent.12
In the growing field of global surgery, there is no doubt that trauma is a central driver of morbidity and mortality among the young and productive members of society, across all income levels.13 In 2015, the seminal work by the Lancet Commission on Global Surgery (LCoGS)14 reaffirmed the primordial importance of essential surgical services as a health care priority. This position statement, reinforced by the Disease Control Priorities 3rd edition volume on Essential Surgery15 and the World Health Assembly resolution A68.15,16 resulted in many studies describing the classification and measurement of unmet surgical disease.17-19 However, many of these studies and foundational documents, including the LCoGS estimates of surgical burden, DALYs and economic impacts are estimates based on models derived from sparse and incomplete data. As Margaret Chan of the World Health Organization aptly warned, “the real need is to close data gaps, especially in low- and middle-income countries, so that we no longer have to rely so heavily on statistical modeling for data on disease burden”.20 Indeed, the more high quality data can be gathered on trauma processes, care and outcomes in LMICs, the more empowered local health-care providers and stakeholders will be to make impactful improvements.
The present study has its limitations. In particular it is a retrospective chart review and therefore, conclusions about the readiness of HSR to collect data prospectively within the context of a new injury surveillance program cannot be definitively asserted. If the local providers and stakeholders view the implementation of a new PTR as intrinsically valuable, then the hope is that the necessary resources will be allocated to allow its successful maintenance through continued high quality data collection. Furthermore, there may be specific challenges at the pilot site which contributed to the issues of data completeness and therefore results may not be generalizable to other institutions. It is possible that the proportion of missing data would be less significant in other hospitals where admission templates are used, resulting in more uniform and complete data collection. These admission, history, and physical examination templates are widespread in many Latin-American hospitals, and may already contribute to improvements in data collection in those institutions.
CONCLUSION
The study has shown that multiple local factors can decrease the providers' ability to record even the most basic of data variables that would be expected in patient health records, regardless of whether or not it was part of a quality improvement or research project. Most prevalent of all such factors appears to be time, which is why we believe that a new PTR must have the appropriate number of variables, without being overly simplistic or too complex, and allow short per-patient completion time. Furthermore, we argue that such a new PTR should include data points that can be easily completed at point-of-care, given that existing health records show inadequate detail for retrospective TR completion.