Transforming Organizational Training: A Mixed Methods Change Initiative in the U.S. Defense Sector

Introduction

In an increasingly complex global defense environment, traditional leadership and training models no longer suffice. This project outlines a real-world organizational change project conducted within a U.S. defense branch to assess training effectiveness, define a strategic vision, and guide leadership development. Using a mixed methods approach, this initiative gathered insights from personnel at every level, ensuring voices across shifts, roles, and specialties were represented. Here’s how we did it, what we discovered, and why it matters.

Why Change Was Needed

The report begins by acknowledging the scale and complexity of the U.S. defense industry, especially in the context of globalization and emerging threats like climate change. These macro-level shifts require changes not only in defense strategy but also within the internal operations and leadership paradigms of defense organizations. The call is clear: organizational growth must center around people development and adaptability.

How We Designed the Project

A steering team was created to guide the change effort. Each member brought a unique operational perspective.

The model used defined a framework to evaluate training effectiveness and future needs.

Figure 1: A model for assessing perceptions of training effectiveness and future training needs simultaneously.

Project timelines and roles were meticulously tracked, as seen in Table 2.

Collecting Insights from the Organization

Quantitative surveys assessed perceived training effectiveness and leadership engagement. Qualitative focus groups supplemented this with emotional and contextual insights. Personnel responded with candor, sharing frustrations, critiques, and occasionally strong language that revealed deeply held concerns.

Bridging the Gap

Phase III focused on identifying future training needs in alignment with the newly defined vision. Cross-functional focus groups and ranking surveys helped prioritize these needs. Survey ties were broken through targeted follow-ups to ensure clarity.

Reporting and Recommendations

The report’s quantitative data was presented using descriptive statistics and Likert-type scales.

Where distributions were bimodal, annotations clarified interpretation. Leadership engagement scores were visualized through bar charts and qualitative findings were categorized and supplemented with direct quotes. The major takeaway: Invest in leadership development, especially for new managers.

Figure 2: A demonstration, not based on actual data collected for this project, of what the reported distributions looked like.

Reflecting on the Process

Most of the project adhered closely to the original plan, with only minor changes made to accommodate real-world constraints. Flexibility in Phase II focus group methods was necessary due to operational realities, leading to a successful data collection effort across all shifts.

Results & Organizational Impact

The project achieved all three core goals: defining a vision, assessing current training, and prioritizing future training. The report prompted organizational action: 18 new action items were developed by branch managers based on the findings.

Lessons in Leadership and Equity

Managers adapted to reach all crews fairly, even working weekends and night shifts to ensure full representation. Ethical leadership was practiced through participatory methods, reflecting a deep respect for diversity across teams.

Conclusion

This project didn’t just evaluate a training program, it modeled a respectful, inclusive, and research-driven path to organizational improvement. When people are heard, leaders listen, and action is based on evidence, even the most complex systems can undergo transformation and improve operational performance.

All References Used in the Project

Astalin, P.K. (2013) Qualitative Research Designs: A Conceptual Approach. International Journal of Social Science and Interdisciplinary Research. 2, 118-124.
Brooks, S.G. (2007) Producing security. New Jersey: Princeton University Press
Colman, A., Norris, E., & Preston C. (1997). Comparing Rating Scales of Different Lengths: Equivalence of Scores From 5-Point and 7-Point Scales. Psychological Reports. 80, pp. 355-362.
Creswell, J. W. (2014) Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: SAGE Publications, Inc.
Donalek, J. G. (2004). Demystifying Nursing Research: Phenomenology as a Qualitative Research Method. Urologic Nursing, 24, 516–517.
English, A. D. (2004). Understanding Military Culture: A Canadian Perspective. Montreal, Que: MQUP.
Eschenfelder, B.E. (2010). Using Community Based Assessments to Strengthen Nonprofit-Government Collaboration and Service Delivery. Journal of Health Services Administration, 32(4), 405-446.
Finstad, K. (2010) Response Interpolation and Scale Sensitivity: Evidence Against 5-Point Scales. Journal of Usability Studies. 5(3). pp. 104-110
Kaufman, R. (1998) Strategic thinking: A guide to identifying and solving problems. revised. Washington, DC & Alexandria, VA: The International Society for Performance Improvement and ASTD Press.
Kaufman, R. Guerra-Lopez, I. (2013) Needs assessment for organizational success. Alexandria, VA: American Society for Training and Development Press.
Hajjar, R. M. (2014). Emergent Postmodern US Military Culture. Armed Forces & Society, 40(1), 118-145.
Heifetz, R., Grashow, A., & Linsky, M. (2009) The practice of adaptive leadership. Boston: Harvard Business Press.
Hough, P. (2013) Understanding global security. New York: Routledge
Lewis, James R. (1993) Multipoint Scales: Mean and Median Differences and Observed Significance Levels. International journal of Human-Computer Interaction. 5(4). pp. 383-392