RECENT HIGHLIGHTED PUBLICATIONS

Association of Hospital Public Quality Reporting With Electronic Health Record Medication Safety Performance with David W Bates in JAMA Network Open.

Using a regression discontinuity design, we evaluate the causal impact of hospitals receiving negative publicly reported feedback on EHR-based clinical decision support system performance at prevent adverse drug events on performance in the subsequent year. We find that hospitals who receive negative public feedback improve significantly more in the following year, suggesting that public reporting of specific process quality outcomes can be a useful tool for policymakers seeking to incentivize improvement.

Growth in Health Information Exchange With ACO Market Penetration with Nate Apathy and Rachel Werner in The American Journal of Managed Care.

Using newly developed difference-in-differences estimators for staggered treatment design, we evaluate the causal impact of hospitals joining an accountable care organization (ACO) on the breadth of their health information exchange (HIE) data sharing networks broadly and stratified by ACO penetration within their local market. We find that HIE breadth increased by 30.7% with ACO participation. In stratified analyses, this effect was larger for hospitals in high–ACO penetration markets (32.0% increase) and smaller for hospitals in low–ACO penetration markets (24.8% increase). We found dynamic effects of ACO adoption illustrating an immediate effect in high–ACO penetration markets and a 2-year delayed effect in low–ACO penetration markets.

Assessment of Electronic Health Record Use Between US and Non-US Health Systems with N Lance Downing, David W Bates, Tait Shanafelt, Arnold Milstein, Topher Sharp, David Cutler, Robert Huckman, and Kevin Schulman in JAMA Internal Medicine.

Taking advantage of a novel dataset of EHR audit log measures across the entire Epic Systems customer base, we compare data from 371 health systems in the United States and abroad. We find that compared to their non-US peers, US clinicians spent more time per day actively using the EHR , documented using more automated note text via copy/paste and templates, and received more inbox messages per day. The median US clinician spent as much time actively using the EHR per day as a non-US clinician in the 99th percentile of active EHR use .


See below for a full list of peer-reviewed research publications:

Peer Reviewed journal articles

  1. Liu, T., Zhu, Z., Holmgren, A. J., & Ellimoottil, C. (2024) National Trends in Billing Patient Portal Messages as E-Visit Services in Traditional Medicare. Health Affairs Scholar. Volume 2, Issue 4, April 2024, qxae040, doi:10.1093/haschl/qxae040.

  2. Apathy, N. C., Patel, V. P., Rolle, T. L., & Holmgren, A. J. (In Press) Physicians in ACOs Report Greater Documentation Burden. American Journal of Managed Care.

  3. Holmgren, A. J., Oakes, A. H., Miller, A., Adler-Milstein, J., & Mehrotra, A. (2024) National Trends in Billing Secure Messages as E-Visits. JAMA: Journal of the American Medical Association. Published online January 10, 2024. doi:10.1001/jama.2023.26584

  4. Rotenstein, L. S., Holmgren, A. J., Horn, D. M., Lipsitz, S., Phillips, R., Gitomer, R., & Bates, D. W. (2023) System-Level Factors and Time Spent on Electronic Health Records by Primary Care Physicians. JAMA Network Open. 2023;6(11):e2344713. doi:10.1001/jamanetworkopen.2023.44713

  5. Holmgren, A. J., Thombley, R., Sinsky, C., & Adler-Milstein, J. (2023) Changes in Physician Electronic Health Record Use With the Expansion of Telemedicine. JAMA Internal Medicine. Published online October 30, 2023. doi:10.1001/jamainternmed.2023.5738

  6. Pantell, M. S., Holmgren, A. J., Iott, B., Neuhas, J., Adler-Milstein, J., & Gottlieb, L. M. (2023) Social and Medical Care Integration Practices Among Children’s Hospitals. Hospital Pediatrics. October 2023; 13 (10): 886–894. doi:10.1542/hpeds.2023-007246

  7. Sinha, S., Holmgren, A. J., Hong, J. C., & Rotenstein, L. S. (2023) Ctrl-C: A Cross-Sectional Study of the EHR Usage Patterns of US Oncology Clinicians. JNCI Cancer Spectrum. 2023;, pkad066, doi:10.1093/jncics/pkad066.

  8. Rotenstein, L. S.*, & Holmgren, A. J.* (2023) COVID Exacerbated the Gender Disparity in Physician Electronic Inbox Burden. Journal of the American Medical Informatics Association. Published online July 12, 2023:ocad141. doi:10.1093/jamia/ocad141

  9. Cross, D. A., Holmgren, AJ., & Apathy, N. C. (2023) The Role of Organizations in Shaping Physician Use of Electronic Health Records (EHRs). Health Services Research. doi:10.1111/1475-6773.14203

  10. Holmgren, A. J., Esdar, M., Hüsers, J, & Coutinho-Almeida, J. (2023) Health Information Exchange: Understanding the Policy Landscape and Future of Data Interoperability. IMIA Yearbook of Medical Informatics. Published online July 6, 2023. doi:10.1055/s-0043-1768719.

  11. Hendrix, N., Bazemore, A., Holmgren, A. J., Rotenstein, L. S., Eden, A. R., Krist, A. H., Phillips, R. L. (2023) Variation in Family Physicians’ Experiences Across Different Electronic Health Record Platforms: a Descriptive Study. Journal of General Internal Medicine. doi:10.1007/s11606-023-08169-5

  12. Holmgren, A. J., Byron, M. E., Grouse, C. K., & Adler-Milstein, J. (2023) Association Between Billing Patient Portal Messages as E-Visits and Patient Messaging Volume. JAMA: Journal of the American Medical Association. Published online January 06, 2023. doi:10.1001/jama.2022.24710

  13. Grouse, C. K., Waung, M. W., Holmgren, A. J., Mongan, J., Neinstein, A., Josephson, A. S., & Khanna, R. R. (2022) Behavioral “Nudges” in the Electronic Health Record to Reduce Waste and Misuse: Three Interventions. Journal of the American Medical Informatics Association. ocac238, doi:10.1093/jamia/ocac238.

  14. Apathy, N.C., Rotenstein, L. S., Bates, D. W., & Holmgren, A. J. (2022) Documentation Dynamics: EHR note composition, burden, and physician efficiency. Health Services Research. doi:10.1111/1475-6773.14097

  15. Holmgren, A. J., & Apathy, N. C. (2022.) Trends in US Hospital Electronic Health Record Vendor Market Concentration, 2012 – 2021. Journal of General Internal Medicine. doi:10.1007/s11606-022-07917-3.

  16. Tang, M., Holmgren, A. J., McElerath, E. E., Bhatt, A. S., Varshney, A. S., Lee, S. G., Vaduganathan, M, Adler, D. S., & Huckman, R. S. (2022) Investigating the Association Between Telemedicine Use and Timely Follow-up Care after Acute Cardiovascular Hospital Encounters. JACC: Advances. Dec, 1 (5) 1–12. doi:10.1016/j.jacadv.2022.100156

  17. Rotenstein, L. S., Apathy, N. C., Holmgren, A. J., & Bates, D. W. (2022) Physician Note Composition Patterns and Time on the EHR Across Specialty Types: a National, Cross-sectional Study. Journal of General Internal Medicine. doi:10.1007/s11606-022-07834-5

  18. Rotenstein, L. S., Holmgren, A. J., Healey, M. J., Horn, D., Ting, D. Y., Lipsitz, S., Salmasian, H., Gitomer, R., & Bates, D. W. (2022) Association Between Electronic Health Record Time and Quality of Care Metrics in Primary Care. JAMA Network Open. 2022;5(10):e2237086. doi:10.1001/jamanetworkopen.2022.37086

  19. Cross, D. A., Adler-Milstein, J., & Holmgren, A. J. (2022) Management Opportunities and Challenges after Achieving Widespread Health System Digitization. Advances in Health Care Management. 2022:67-87. doi:10.1108/S1474-823120220000021004

  20. Holmgren, A. J., & Apathy, N. C. (2022) Assessing the Impact of Patient Access to Electronic Health Information on Clinician Electronic Health Record Documentation. Journal of the American Medical Informatics Association. ocac120, doi:10.1093/jamia/ocac120.

  21. Kutza, J. O., Hubner, U, Holmgren, A. J., & Esdar, M. (2022) Patient Safety Informatics: Criteria development for assessing the maturity of digital patient safety in hospitals. Studies in Health Technology and Informatics. IOS Press; 2022. doi:10.3233/SHTI220618.

  22. Holmgren, A. J.*, Rotenstein, L. S.*, Downing, N. L., Bates, D. W., & Schulman, K. (2022) Association Between State-level Medical Malpractice and Clinician Electronic Health Record Use. Journal of the American Medical Informatics Association. ocac034, doi:10.1093/jamia/ocac034

  23. Arvisais-Anhalt, S., Lau, M., Lehmann, C. U., Holmgren, A. J., Medford, R. J., Ramirez, C. M., & Chen, C. (2022) The 21st Century Cures Act and Multi-User Electronic Health Record Access. Journal of Medical Internet Research. 2022;24(2):e34085. doi:10.2196/34085

  24. Holmgren, A. J., Everson, J., & Adler-Milstein, J. (2022) Association of Hospital Interoperable Data Sharing and Alternative Payment Model Participation. JAMA Health Forum. 2022;3(2):e215199. doi:10.1001/jamahealthforum.2021.5199

  25. Apathy, N. C., Holmgren, A. J., & Werner, R. M. (2022) Growth in Health Information Exchange with ACO Market Penetration. American Journal of Managed Care. 2022;28(1):e7-e13. doi:10.37765/ajmc.2022.88815

  26. Holmgren, A. J.*, Downing, N. L.*, Tang, M., Sharp, C., Longhurst, C., & Huckman, R. S. (2021) Assessing the Impact of the COVID-19 Pandemic on Clinician Electronic Health Record Use. Journal of the American Medical Informatics Association. ocab268, https://doi.org/10.1093/jamia/ocab268

  27. Holmgren, A. J., & Bates, D. W. (2021) Association of Hospital Public Quality Reporting With Electronic Health Record Medication Safety Performance. JAMA Network Open. 2021;4(9):e2125173. doi:10.1001/jamanetworkopen.2021.25173

  28. Holmgren, A. J., Lindeman, B., & Ford, E. W. (2021) Resident Physician Experience and Duration of Electronic Health Record Use. Applied Clinical Informatics. 12(4):8. doi:10.1055/s-0041-1732403

  29. Holmgren, A. J., Kuznetsova, M, Classen, D. C., & Bates, D. W. (2021) Assessing Hospital Electronic Health Record Vendor Performance Across Publicly Reported Quality Measures. Journal of the American Medical Informatics Association. ocab120, doi.org/10.1093/jamia/ocab120

  30. Holmgren, A. J., Cutler, D., Mehrotra, A. (2021) The Increasing Role of Physicians as Bill Collectors: Destined for Failure. JAMA: Journal of the American Medical Association. Published online July 30, 2021. doi:10.1001/jama.2021.12191

  31. Rotenstein, L. S.*, Holmgren, A. J.*, Downing, N. L., Longhurst, C. A., & Bates, D. W. (2021) Differences in EHR Use Between Adult and Pediatric Primary Care Ambulatory Clinicians. JAMA Network Open. 2021;4(7):e2116375. doi:10.1001/jamanetworkopen.2021.16375

  32. Apathy, N. C., Holmgren, A. J., & Adler-Milstein, J. (2021) A Decade Post-HITECH: Critical Access Hospitals have EHRs but Struggle to Keep Up with Other Advanced Functions. Journal of the American Medical Informatics Association. ocab102, doi.org/10.1093/jamia/ocab102

  33. Holmgren, A. J., Phelan, J., Jha, A. K., & Adler-Milstein, J. (2021) Hospital Organizational Strategies Associated With Advanced EHR Adoption. Health Services Research. 2021;00:1–11. https://doi.org/10.1111/1475-6773.13655

  34. Rotenstein, L.S., Holmgren, A. J., Downing, N. L., & Bates, D. W. (2021) Differences in Total and After-Hours Electronic Health Record Time Across Ambulatory Specialties. JAMA Internal Medicine. Published online March 22, 2021. doi:10.1001/jamainternmed.2021.0256

  35. Co, Z., Holmgren, A. J., Classen, D. C., Newmark, L. P., Seger, D., Cole, J. M., Zimmer, K. P., Pon, B., & Bates, D. W. (2021) The Development and Piloting of the Ambulatory Electronic Health Record (EHR) Evaluation Tool: Lessons Learned. Applied Clinical Informatics. doi:10.1055/s-0041-1722917

  36. Thompson, S. C., Holmgren, A. J., & Ford, E. W. (2021) Information System Use Antecedents of Nursing Employee Turnover in a Hospital Setting. Health Care Management Review.

  37. Holmgren, A. J.,* Downing, N. L.,* Bates, D. W., Shanafelt, T., Milstein, A., Sharp, C., Cutler, D., Huckman, R. S, & Schulman, K. (2020) Assessment of Electronic Health Record Use Between US and Non-US Health Systems. JAMA Internal Medicine. Published online December 14, 2020. doi:10.1001/jamainternmed.2020.7071. Featured Editorial: Sinsky, Christine. (2020) Advancing Practice Science With Electronic Health Record Use Data.

  38. Co, Z., Holmgren, A. J., Classen, D. C., Newmark, L. P., Seger, D., Danforth, M., & Bates, D. W. (2020) The Tradeoffs Between Safety and Alert Fatigue: Data from a National Evaluation of Hospital Medication-related Clinical Decision Support. Journal of the American Medical Informatics Association. ocaa098, https://doi.org/10.1093/jamia/ocaa098

  39. Holmgren, A. J., & Apathy, N. C. (2020) Evaluation of Prescription Drug Monitoring Program Integration with Hospital Electronic Health Records by US County-level Opioid Prescribing Rates. JAMA Network Open. 2020;3(6):e209085. doi:10.1001/jamanetworkopen.2020.9085

  40. Holmgren, A. J., Botelho, A., & Brandt, A. M. (2020) A History of Prescription Drug Monitoring Programs in the United States: Political Appeal and Public Health Efficacy. American Journal of Public Health. no.(): pp. e1-e7. doi:10.2105/AJPH.2020.305696

  41. Classen, D. C., Holmgren, A. J., Co, Z., Newmark, L. P., Seger, D., Danforth, M., & Bates, D. W. (2020) National Trends in the Safety Performance of Electronic Health Record Systems: 2009-2018. JAMA Network Open. 2020;3(5):e205547. doi:10.1001/jamanetworkopen.2020.5547. Featured Editorial: Ratwani, Raj. (2020) Modest Progress on the Path to Electronic Health Record Medication Safety.

  42. Holmgren, A. J., Apathy, N. C., & Adler-Milstein, J. (2020) Barriers to Hospital Public Health Reporting and Implications for the COVID-19 Pandemic. Journal of the American Medical Informatics Association ocaa112, https://doi.org/10.1093/jamia/ocaa112

  43. Levine, D. M., Co, Z., Newmark, L.P., Groisser, A. R., Holmgren, A. J., Haas, J.S., & Bates, D.W. (2020) Design and Testing of a Mobile Health Application Rating Tool. npj Digital Medicine. 3, 74 (2020). https://doi.org/10.1038/s41746-020-0268-9

  44. Apathy, N. C., & Holmgren, A. J. (2020) Opt-in Consent Policies: Potential Barriers to Hospital Health Information Exchange. American Journal of Managed Care, 26(1), e14-e20. Health IT Special Issue.

  45. Holmgren, A. J., & Apathy, N. C. (2019) Hospital Adoption of API-Enabled Patient Data Access. Healthcare. (2019/08/27/ 2019): 100377. doi:10.1016/j.hjdsi.2019.100377

  46. Holmgren, A. J., Co, Z., Newmark, L. P., Danforth, M., Classen, D. C., & Bates, D. W. (2019) Assessing the Safety of Electronic Health Records: A National Longitudinal Study of Medication-Related Decision Support. BMJ Quality & Safety, Published Online First: 18 July 2019. doi: 10.1136/bmjqs-2019-009609. Featured Editorial: Sheikh, Aziz. (2019) Realising the Potential of Health Information Technology to Enhance Medication Safety.

  47. Holmgren, A. J., Adler-Milstein, J., & Chen, L.M. (2018) Participation in a Voluntary Bundled Payment Program by Organizations Providing Care After an Acute Hospitalization. JAMA: Journal of the American Medical Association, 320(4), 402-404. doi:10.1001/jama.2018.8666.

  48. Holmgren, A. J., & Ford, E.W. (2018) Assessing the Impact of Health System Organizational Structure on Hospital Electronic Data Sharing. Journal of the American Medical Informatics Association, 25(9), 1147-1152. doi:10.1093/jamia/ocy084

  49. Holmgren, A. J., Adler-Milstein, J., & McCullough, J. (2018). Are all certified EHRs created equal? Assessing the relationship between EHR vendor and hospital meaningful use performance. Journal of the American Medical Informatics Association, 25(6), 654-660. doi:10.1093/jamia/ocx135. Awarded Editor’s Choice.

  50. Holmgren, A. J., Patel, V., & Adler-Milstein, J. (2017) Progress in Interoperability: Measuring US Hospital Engagement in Sharing Patient Data. Health Affairs, 36(10), 1820-1827. 

  51. Adler-Milstein, J., Holmgren, A. J., Kralovec, P., Worzala, C., Searcy, T., & Patel, V. (2017) Electronic Health Record Adoption in US Hospitals: The Emergence of a Digital “Advanced Use” Divide. Journal of the American Medical Informatics Association. ocx080, doi: 10.1093/jamia/ocx080. Awarded Editor’s Choice.

  52. Holmgren, A. J., & Adler-Milstein, J. (2017) Health Information Exchange in U.S. Hospitals: The Current Landscape and a Path to Improved Information Sharing. Journal of Hospital Medicine, 12(3), 193-198. doi:10.12788/jhm.2704

  53. Holmgren, A. J., Pfeifer, E., Manojlovich, M., & Adler-Milstein, J. (2016). A Novel Survey to Examine the Relationship between Health IT Adoption and Nurse-Physician Communication. Applied Clinical Informatics, 7(4), 1182-1201. doi:10.4338/ACI-2016-08-RA-0145

  54. Holmgren, A. J., Patel, V., Charles, D., & Adler-Milstein, J. (2016). US hospital engagement in core domains of interoperability. American Journal of Managed Care, 22(12), e395-e402. Health IT Special Issue.

*Indicates co-first authorship.

OTHER RESEARCH PUBLICATIONS

  1. Gale, B., Mossburg, S., Holmgren, A. J., & McBride, S. (2023) Technology as a Tool for Improving Patient Safety. AHRQ Patient Safety Network Annual Perspective.

  2. Holmgren, A. J. (2023) The Health Information Technology Special Issue: Tackling New Challenges to Digital Transformation. The American Journal of Managed Care.

  3. Holmgren, A. J. (2019) The Prescribing Panopticon: A History of Prescription Drug Monitoring Programs. The Opioid Epidemic in Historical Perspective: An Anthology.

  4. Holmgren, A. J. & Adler-Milstein, J. (2019) Does Electronic Health Record Consolidation Follow Hospital Consolidation? Health Affairs Blog.

  5. Holmgren, A. J., & Ford, E.W. (2018) Health System Organizational Structure and Hospital Electronic Data Sharing. Proceedings of the Seventy-Eighth Annual Meeting of the Academy of Management.

TEXTBOOK CHAPTERS

  1. Dixon, B.E., Holmgren, A. J., Adler-Milstein, J, & Grannis, S. J. (2022) Health Information Exchange and Interoperability. In: Finnell, J. T.; Dixon, B. E. Editors. Clinical Informatics Study Guide. Springer Nature Switzerland. 2022. doi:10.1007/978-3-030-93765-2_14, ISBN 978-3-030-93764-5