CAPABILITIES BASED TEST & EVALUATION (CBTE) KENNETH SENECHAL NAWCAD

CAPABILITIES BASED TEST & EVALUATION (CBTE) KENNETH SENECHAL NAWCAD Release 2019-277. Distribution Statement A - Approved for public release; distribution is unlimited.

What is CBTE? Warfare Analysis Top Down Red Forces and Blue Forces CONEMPs System of Systems Rqmts ENGAGE SURVEIL Integration with COTF's Mission Based Test Design (MBTD) Weapons Schools TRACK AA MISSILE F-35 F/A-18 E-2D Bottom Up DETECT T O N ID LAUNCH WEAPON ASSESS L A ION CBTE evaluates the effects chain in a mission context Add Mission Platform Context Rqmts (CDD, CPD, (CONOPs, ROC / POE) TEMP, specs) NAWCAD Distribution Statement A - Approved publicdistribution release; isdistribution NAWCADRelease Release 2019-277. 2018-274. Distribution Statement A - Approved for publicfor release; unlimited. is unlimited. 2

NDIA Tabletop Exercise Jointly hosted by CBTE & NDIA Objectives Emphasized an open forum and non-attributional dialogue Generate baseline RFP and Proposal Guidebook Participation Navy Enterprise COTF, NAVSEA, NAVAIR Major Primes Boeing, Lockheed Martin, Raytheon, Northrop Grumman Representing industry not their specific company Facilitated R-Squared Solutions – Bill “Roto” Reuter Pivotal Insight – Cary Paul & Emily Knight NAWCAD Distribution Statement A - Approved publicdistribution release; isdistribution NAWCADRelease Release 2019-277. 2018-274. Distribution Statement A - Approved for publicfor release; unlimited. is unlimited. 3

Tabletop Early Takeaways Moment of Truth Feedback Extremely positive feedback on the event from all participants Consensus that CBTE is a highly effective approach Product Delivery Comment/Feedback adjudication is still in progress What contractors need: Provide previous test data as GFI to industry to help them VV&A their models Need to stop doing things the current way and start doing them the new way Level setting contractor bids so that each prime approaches with same assumptions What CBTE needs: Contractor understanding of govt LVC infrastructure Using a mission focus to incentivize contracts Define overarching test strategy that includes LVC (LVC isn’t a stand alone strategy) MBTD / Mission thread focused (security issues) What we both need: Need a common vernacular and industry understanding of our tools/process/policy Identified new specific SOO/SOW elements Pairing a weapon system SME with a DoE SME is critical for success Balance of defining need/requirement so it doesn’t stifle innovation Proper balancing of spec requirement verification in a mission construct NAWCAD Distribution Statement A - Approved publicdistribution release; isdistribution NAWCADRelease Release 2019-277. 2018-274. Distribution Statement A - Approved for publicfor release; unlimited. is unlimited. 4

NDIA Way Ahead Continue Data & Feedback Adjudication NDIA T&E Conference Out-brief (13-15 May 2019) N94 Sponsored NAVAIR hosted Cross SYSCOM TIM (5-6 Jun 2019) Common processes across Navy SYSCOMs Follow on Tabletop Sessions desired Notionally planning for end of Summer Product Delivery Projecting Year End CBTE RFP and Proposal Guidebook Brief guidelines to NAVAIR/PEO Leadership NAWCAD Distribution Statement A - Approved publicdistribution release; isdistribution NAWCADRelease Release 2019-277. 2018-274. Distribution Statement A - Approved for publicfor release; unlimited. is unlimited. 5

Back Up NAWCAD Distribution Statement A - Approved publicdistribution release; isdistribution NAWCADRelease Release 2019-277. 2018-274. Distribution Statement A - Approved for publicfor release; unlimited. is unlimited. 6

CBTE Enablers Live-Virtual-Constructive (LVC) Mission Based Test Design (MBTD) Design of Experiments (DoE) Mission T&E During System Development System of Systems Testing Human Performance Mission Analysis Mission Training NAWCAD Distribution Statement A - Approved publicdistribution release; isdistribution NAWCADRelease Release 2019-277. 2018-274. Distribution Statement A - Approved for publicfor release; unlimited. is unlimited. 7

CBTE “Strategy” Test Program Introduction Document Mission Based Test Design Engineering / Test Collaboration Test Planning Strategy Meeting - 5.1 / 5.2 / 5.3 / 5.4 / COTF/DOT&E - Early identification of LVC, range, and target needs NAWCAD Release 2019-277. Distribution Statement A - Approved for public release; distribution is unlimited. NAWCAD Release 2018-274. Distribution Statement Approved for release; distribution is unlimited. NAWCAD Release 2019-277. Distribution Statement AA- -Approved forpublic public release; distribution is unlimited. 8

Capabilities-Based Acquisition Integrated Warfare Analysis Fleet operators work with engineering to develop concepts of employment (CONEMPS), effects-chains & required attributes Capabilities-Based T&E System Model Systems are developed in a model-based integrated digital environment (SE Transformation) Industry Models 001010 1010 0001 010 Virtual Constructive Digital Modeling 010 1010 00010 010010 SoS Models Data Feedback Loop SoS Model Integration Laboratory Live Flight Test Installed Systems Test Facility Single Source of Truth CONEMPS and effects-chains are modeled at the System of Systems (SoS) level Hardware In the loop System models form “Constructive” Basis for LVC M&S environment LVC-based Training Required for Fleet Integrated Ops Results Results in optimized capability capability development development document document NAWCAD Distribution Statement A - Approved publicdistribution release; isdistribution NAWCADRelease Release 2019-277. 2018-274. Distribution Statement A - Approved for publicfor release; unlimited. is unlimited. 9