Evaluating Predictive Mathematical Models of Shipboard Operational Decision-Making Performance Using Scenario-Based Assessment

Abstract

Shipboard operational decision making requires seafarers to interpret numerical information, identify operational risks, apply procedures, use onboard technology, and coordinate actions under changing conditions. This study evaluated predictive mathematical models of shipboard operational decision-making performance using scenario-based assessment. Data were obtained from 78 experienced seafarers enrolled in Recognition of Prior Learning programs in Nautical Studies and Marine Engineering. Five self-reported constructs were examined: operational data understanding, seafaring experience, SOP and checklist use, shipboard technology utilization, and communication and coordination. The outcome was the mean of four discipline-specific scenario scores. During manuscript revision, all scenario responses were re-evaluated with the predefined 1–5 rubric by discipline-specific subject-matter lecturers, and all analyses were repeated using these expert-assessed scores. OLS with HC3 robust standard errors was used as the inferential baseline, while Ridge regression was assessed using repeated nested stratified five-fold cross-validation. The OLS model yielded R² = 0.198 and adjusted R² = 0.130; X1–X5 were not jointly significant after program adjustment (p = 0.231). The best predictive model, Ridge with X1–X5 and program, achieved mean cross-validated R² = 0.075 (SD = 0.030), RMSE = 1.099, and MAE = 0.950. These findings indicate modest out-of-sample prediction and support cautious use of self-reported operational constructs as predictors of scenario-demonstrated performance.