Seattle-Denver Center of Innovation (COIN)

VA Collaborative Evaluation Center (VACE)

Background | Current Projects | Team | Publications | Questions

VA Collaborative Evaluation Center (VACE)

A highly collaborative VA “virtual center” composed of quantitative, qualitative, and mixed methods evaluation experts.

Evaluate implementation of VA initiatives to assess effectiveness and barriers and facilitators to implementation, as well as unintended consequences.

Work collaboratively with operational partners to ensure evaluation outcomes are relevant and useful for program improvements and sustainability.

Benefits of the collective VACE experience

Highly responsive to needs of program office.

Able to quickly address emerging evaluation questions.

Leverage cumulative experience from conducting multiple studies with overlapping foci (e.g. access to care).

Bring knowledge and methodological rigor from one study to another.

Provide multi-specialty perspective on evaluation (e.g., primary care and specialty care).

Background

Group began in 2011 through a competitive award from VA Quality Enhancement Research Initiative (QUERI) and Office of Specialty Care (OSC) to conduct partnered research and evaluation while meeting the operational needs of the OSC

Developed a longitudinal, mixed methods evaluation of the recently launched national initiatives to improve delivery of specialty care to Veterans:

• Electronic Consults (E-consults)
• Specialty Care Access Networks-Extension for Community Healthcare Outcomes (SCAN-ECHO)
• Specialty Care Mini-Residency Program (SC-MRP)
• Specialty Care Neighborhood (SCN)

Return to Top

Current Projects

ED Opioid Safety Initiative (OSI) Program Evaluation

Assess VA ED OSI program to understand impact on prescribing rates and provider prescribing practices

Highly Rural Veteran Transportation EWI Evaluation

Assess the implementation of Highly Rural Transportation services and the impact on Veterans’ access to VA and VA-approved community care.

Mobile Prosthetic and Orthotic Care EWI Evaluation

Collect and evaluate data pertaining to need for mobile O&P services throughout the Enterprise

PT In PACT Mixed Methods Evaluation

Evaluate the implementation, impact, and effectiveness of integrating Physical Therapy (PT) in primary care settings to treat musculoskeletal complaints.

RCI Evaluation

Quantitatively and qualitatively assess the national implementation of the Referral Coordination Initiative (RCI) within cardiology, gastroenterology, and pulmonary for the Office of Integrated Veterans Care (IVC).

Return to Top

VACE Team

Return to Top

VACE-Related Publications

Stryczek KC, Honsberger M, Ball SL, Barnard JG, Young JP, Felker B, Au DH, Ho PM, Kirsh SR, Sayre GG. VA Outreach Is an Essential Area for Improving Veterans' Health Care Accessibility. Mil Med. 2023 Feb 15:usad019. doi: 10.1093/milmed/usad019. Epub ahead of print.
PMID: 36790439

Carey E, Chen HP, Baker D, Blankenhorn R, Munro S. Non-Ventilator Associated Hospital Acquired Pneumonia (NV-HAP) Risk Among Hospitalized Veterans Before and During the COVID-19 Pandemic. Am J Infect Control. 2023 Jan 2:S0196-6553(22)00888-4. doi: 10.1016/j.ajic.2022.12.012. Epub ahead of print. PMCID: PMC9807267.
PMID: 36603808.
Learn more about this publication on Dimensions

Stevenson LD, Munro S, Klocko R, Sayre G. A mixed-methods evaluation of the national implementation of the Hospital-Acquired Pneumonia Prevention by Engaging Nurses (HAPPEN) initiative. Infect Control Hosp Epidemiol. 2022 Aug 30:1-8. doi: 10.1017/ice.2022.99. Epub ahead of print.
PMID: 36039946
Learn more about this publication on Dimensions.

Mog AC, Moldestad M, Kenney R, Stevenson L, Lee M, Ho PM, Sayre GG. "I was Unsure at First": A Qualitative Evaluation of Patient Perceptions of VA Clinical Video Telehealth Visits in the V-IMPACT Program. J Patient Exp. 2022 Jul 3;9:23743735221107237. doi: 10.1177/23743735221107237. PMCID: PMC9260561. PMID: 35813242
Learn more about this publication on Dimensions.

Carey E, Chen HP, Baker D, Blankenhorn R, Vega RJ, Ho M, Munro S. The Association between Non-Ventilator Associated Hospital Acquired Pneumonia and Patient Outcomes Among U.S. Veterans. Am J Infect Control. 2022 Feb 26:S0196-6553(22)00102-X. doi: 10.1016/j.ajic.2022.02.023. Epub ahead of print.
PMID: 35231564
Learn more about this publication on Dimensions

Kenney RR, Haverhals LM, Stryczek KC, Fehling KB, Ball SL. Site visit Standards Revisited: A Framework for Implementation. American Journal of Evaluation. February 2022.
doi:10.1177/10982140221079266.

Leonard C, Kenney RR, Lee M, Greene P, Whittington M, Kirsh S, Ho PM, Sayre G, Simonetti J. Common Ground: Primary Care and Specialty Clinicians' Perceptions of E-Consults in the Veterans Health Administration. Fed Pract. 2022 Jan;39(1):42-47b. doi: 10.12788/fp.0214. Epub 2022 Jan 12. PMCID: PMC8849031.
PMID: 35185320

Subramaniam S, Stryczek KC, Ball S, Stevenson L, Ho PM, Aron DC. Workforce Development to Improve Access to Pain Care for Veterans: A Qualitative Analysis of VA-ECHO Participant Experiences. Health Serv Res Manag Epidemiol. 2022 Sep 7;9:23333928221124806. doi: 10.1177/23333928221124806. PMCID: PMC9452798.
PMID: 36093259
Learn more about this publication on Dimensions.

Subramaniam S, Chen J, Wilkerson TL, Stevenson L, Kincaid C, Firestone C, Ball SL. Refining the Implementation of a Hub-and-Spoke Model for TelePain Through Qualitative Inquiry. J Technol Behav Sci. 2022 Dec 6:1-11. doi: 10.1007/s41347-022-00288-w. Epub ahead of print. PMCID: PMC9734451.
PMID: 36530381
Learn more about this publication at Dimensions

Stevenson LD, Munro S, Klocko R, Sayre G. A mixed-methods evaluation of the national implementation of the Hospital-Acquired Pneumonia Prevention by Engaging Nurses (HAPPEN) initiative. Infect Control Hosp Epidemiol. 2022 Aug 30:1-8. doi: 10.1017/ice.2022.99. Epub ahead of print.
PMID: 36039946
Learn more about this publication on Dimensions.

Donovan LM, PalenBN, Fernandes LA, Whittington MD, Feser WJ, Blankenhorn R, Gunzburger E,Grunwald GK, Gamache J, Liao JM, Kirsh S, Au DH. The Referral Coordination Team: A Redesign of Specialty Care to Enhance Service Delivery and Value in Sleep Medicine. NEJM Catalyst Innovations in Care Delivery Vol. 2 No. 2 February 2021.
DOI:https://doi.org/10.1056/CAT.20.0403

Carey E, Blankenhorn R, Chen P, Munro S. Non-Ventilator Associated Hospital Acquired Pneumonia Incidence and Health Outcomes Among U.S. Veterans from 2016-2020. Am J Infect Control. 2021 Jun 8:S0196-6553(21)00399-0. doi: 10.1016/j.ajic.2021.06.001. Epub ahead of print.
PMID: 34116083

Donovan LM, Mog AC, Blanchard KN, Magid KH, Syed AS, Kelley LR, Palen BN, Parsons EC, McCall CC, Thompson W, Charlton M, Spece LJ, Kirsh S, Au DH, Sayre GG. Patient experiences with telehealth in sleep medicine: a qualitative evaluation. J Clin Sleep Med. 2021 Mar 23. doi: 10.5664/jcsm.9266. Epub ahead of print.
PMID: 33755010.

Donovan LM, Mog A, Blanchard K, Magid K, Syed A, Kelley L, Palen BN, Parsons BC, McCall C, Thompson WH, Charlton M, Spece LJ, Kirsh S, Au DH, Sayre G. (2021). Patient Experiences with Telehealth in Sleep Medicine: A Qualitative Evaluation.
https://doi.org/10.1164/ajrccm-conference.2021.203.1_MeetingAbstracts.A1716

Kelley L, Ho M, Zanetti C, McKeever M, Na S, Showell S, Fehling K, Kenney R. Primary Care Evolution: Clinic Staff and Patient Perspectives of the Rapid Upscale of Virtual Primary Care during the COVID‐19 Pandemic. Health Serv Res. 2021 Sep;56(Suppl 2):32. doi: 10.1111/1475-6773.13762. Epub 2021 Sep 15. PMCID: PMC8441387.

Donovan LM, Palen BN, Syed A, Blankenhorn R, Blanchard K, Feser WJ, Magid K, Gamache J, Spece LJ, Feemster LC, Fernandes L, Kirsh S, Au DH. Nurse-led triage of new sleep referrals is associated with lower risk of potentially contraindicated sleep testing: a retrospective cohort study. BMJ Qual Saf. 2021 Jul;30(7):599-607. doi: 10.1136/bmjqs-2020-011817. Epub 2020 Dec 22. PMCID: PMC8315816.
PMID: 33443226
Learn more about this publication on Dimensions

Marshall V, Stryczek KC, Haverhals L, Young J, Au DH, Ho PM, Kaboli PJ, Kirsh S, Sayre G. The Focus They Deserve: Improving Women Veterans' Health Care Access. Womens Health Issues. 2021 Feb 10:S1049-3867(20)30157-2. doi: 10.1016/j.whi.2020.12.011. Epub ahead of print.
PMID: 33582001

Whittington MD, Ho PM, Kirsh SR, Kenney RR, Todd-Stenberg J, Au DH, Simonetti J. Cost savings associated with electronic specialty consultations. Am J Manag Care. 2021 Jan 1;27(1):e16-e23. doi: 10.37765/ajmc.2021.88579.
PMID: 33471464

Moldestad M, Stryczek KC, Haverhals L, Kenney R, Lee M, Ball S, Au D, Kirsh S, Sayre G, Young J. Competing Demands: Scheduling Challenges in Being Veteran-centric in the Setting of Health System Initiatives to Improve Access. Mil Med. 2021 Nov 2;186(11-12):e1233-e1240. doi: 10.1093/milmed/usaa520.
PMID: 33289838

LeRouge C, Sangameswaran S, Frogner B, Snyder C, Rubenstein L, Kirsh S, Sayre G. The Group Practice Manager in the VHA: A View From the Field. Fed Pract. 2020 Feb;37(2):79-85. PMCID: PMC7138344.
PMID: 32269470

Lee M, Leonard C, Greene P, Kenney R, Whittington MD, Kirsh S, Ho PM, Sayre G, Simonetti J. Perspectives of VA Primary Care Clinicians Toward Electronic Consultation-Related Workload Burden: A Qualitative Analysis. JAMA Netw Open. 2020 Oct 1;3(10):e2018104. doi: 10.1001/jamanetworkopen.2020.18104. PMCID: PMC7599439.
PMID: 33125494

Ball S, Stryczek K, Stevenson L, Hearns R, Au DH, Ho PM, Aron DC. A Qualitative Evaluation of the Pain Management VA-ECHO Program Using the RE-AIM Framework: The Participant's Perspective. Front Public Health. 2020 May 15;8:169. doi: 10.3389/fpubh.2020.00169. PMCID: PMC7242719.
PMID: 32500053

Sayre GG, Haverhals LM, Ball S, Stevenson L, Battaglia C, Aron DC, Kirsh S, Helfrich CD, Au D. Adopting SCAN-ECHO: The providers' experiences. Healthc (Amst). 2017 Mar;5(1-2):29-33. doi: 10.1016/j.hjdsi.2016.04.006. Epub 2016 May 18.
PMID: 28668200

Ball SL, Stevenson LD, Ladebue AC, McCreight MS, Lawrence EC, Oestreich T, Lambert-Kerzner AC. Adaptation of Lean Six Sigma Methodologies for the Evaluation of Veterans Choice Program at 3 Urban Veterans Affairs Medical Centers. Med Care. 2017 Jul;55 Suppl 7 Suppl 1:S76-S83. doi: 10.1097/MLR.0000000000000721.
PMID: 28410337
Learn more about this publication on Dimensions.

Stevenson L, Ball S, Haverhals LM, Aron DC, Lowery J. Evaluation of a national telemedicine initiative in the Veterans Health Administration: Factors associated with successful implementation. J Telemed Telecare. 2018 Apr;24(3):168-178. doi: 10.1177/1357633X16677676. Epub 2016 Nov 30.
PMID: 27909208

Haverhals LM, Sayre G, Helfrich CD, Battaglia C, Aron D, Stevenson LD, Kirsh S, Ho M, Lowery J. E-consult implementation: lessons learned using consolidated framework for implementation research. Am J Manag Care. 2015 Dec 1;21(12):e640-7. PMCID: PMC4717483.
PMID: 26760426

Battaglia C, Lambert-Kerzner A, Aron DC, Sayre GG, Ho PM, Haverhals L, Stevenson L, Kirsh S, Au DH, Helfrich CD. Evaluation of E-Consults in the VHA: Provider Perspectives. Fed Pract. 2015 Jul;32(7):42-48. PMCID: PMC6363293.
PMID: 30766078

Return to Top

Questions

If you have any questions please email:
kelty.fehling@va.gov
michael.ho@va.gov

return to top