In an assessment of the Department of Homeland Security’s (DHS’s) annually required reporting on 43 measures of border security effectiveness, the U.S. Government Accountability Office (GAO) found data limitations and issues with statistical models.
It should be noted: most of what GAO found was good, and their report specifically states that DHS analysis generally used data and methods that produced reliable results. However, the context for DHS reports was lacking. GAO found that recorded border-related data can vary location to location, and DHS fails to provide a complete picture of information, making it difficult for Congress or the general public to fully understand what is being reported to them.
The GAO report, based on DHS reporting for 2017, found that half of the 35 metrics used that year varied by scope or calculation from those elements called for by the National Defense Authorization Act (NDAA). Limitations on data were also not likely to be disclosed by DHS in certain areas, such as apprehensions assisted by unmanned aerial systems, nor were some of DHS’s assumptions in their reported validated with concrete analyses. Part of the issue, in GAO’s view, is that DHS needs to develop and implement a process for systematic review of their data’s reliability, then communicate those limitations to those beyond their doors.
As a result of this, GAO made four recommendations for DHS. Those recommendations include developing and implementing a process of reviewing the data used in its Border Security Metrics Report and identify any limitations with the data and methodologies used as well as ensuring the communication of identified limitations through annual review. Additionally, DHS should include results of sensitivity analyses for key assumptions in statistical models of unlawful entry estimates and include measures of statistical uncertainty for metrics based on statistical model-based estimates in that same report.
DHS has concurred with the recommendations given.