1.1 New additions to this version (1.0.0)

• Assessment models (Assess class functions)
  • Statistical Catch at Age (SCA and SCA2)
  • Delay Difference (DD_TMB)
  • State-Space Delay Difference (DD_SS)
  • Surplus Production (SP)
  • State-Space Surplus Production (SP_SS)
• Visualization functions
  • Standardized assessment reports (plot and summary)
  • Diagnostics for assessment models (profile_likelihood and retrospective)
  • Diagnostics for assessment models within closed-loop simulation (AM, assessment in MSE) (prelim_AM, diagnostic_AM, and retrospective_AM)
• Other features
  • Harvest control rules (HCR class functions: HCR_MSY, HCR40_10, and HCR60_20)
  • make_MP function conveniently combines a Assess and HCR function to create a MP function
  • simmov function creates movement matrices for parameterizing multiple area operating models (age-independent)
  • iSCAM2OM and iSCAM2Data functions import from iSCAM assessments. The MCMC outputs of iSCAM models are now used to characterise operating model uncertainty
  • SS2OM and SS2Data functions import Stock Synthesis assessments using the r4ss package version 1.24.0 (latest CRAN version)

1.2 Coming soon

• More data-rich assessment models
  • Virtual Population Analysis (VPA)
  • Kalman filter assessment approximation

• Stock projections for assessment models

• Age-varying movement matrices for simmov to condition operating models

• Multi-stock and multi-fleet operating models

2.1 Overview of MSEtool

To facilitate a standardized design for data-rich MPs, MSEtool creates new classes to build out the MP class. MSEtool uses functions of class Assess and HCR for an assessment model and harvest control rule, respectively. An MP class function can be made from these functions to be tested in a MSE within the DLMtool framework.

Additionally, Assess functions can be used in an assessment-only context (external to management strategy evaluations). Output of an Assess function can be used to provide standardized stock assessment reporting and diagnostics, such as Kobe plots, likelihood profiles for parameter estimates, and retrospective analyses of time series estimates of biomass, fishing mortality, recruitment, etc.

Figure 2: Design of MSEtool package for building management procedures from assessment models and harvest control rules for management strategy evaluations with the DLMtool package. Assess and HCR functions and the Assessment object are unique to MSEtool, but work within the framework of DLMtool objects. Separate functions are also available for standardized reporting of output from assessment models in a stock assessment setting.

3.1 Reporting and diagnostic functions

Assessment results and summary, along with diagnostic procedures are available for each assessment model. Generic functions summary and plot are available for Assessment objects. The summary function returns a list of estimated current stock status (\(F/F_{MSY}\) , \(B/B_{MSY}\) in the terminal year), input parameters to the model, derived quantities, and parameter estimates. The plot function generates time series figures of data, fitted data, and estimated quantities (e.g., biomass and recruitment). By default, these figures are save to disk and can be viewed with a HTML webpage generated by the function.

summary(DD_result)
plot(DD_result, save_figure = TRUE, save_dir = tempdir()) # By default, also saves figures in a temporary directory for viewing through a HTML browser. The directory can be changed to a user's filespace of choice.

Two diagnostic functions are available: retrospective for retrospective analyses (re-running the model by sequentially removing terminal years of data) and profile_likelihood for profiling the likelihood function over a grid of parameters.

retrospective(DD_result, nyr = 5) # Retrospective analysis going back 5 years from current year
profile_likelihood(DD_result, R0 = seq(0.75, 1.25, 0.025), h = seq(0.95, 1, 2.5e-3)) # Joint profile over grid of R0 and steepness

Figure 3: Joint profile of R0 and steepness from Delay-Difference assessment model (DD_TMB function). Contours indicate change in likelihood values from minimum (red point).

5.1 Diagnostic of assessment models in MSE

Three functions are designed to evaluate the performance of Assess models in MSE: prelim_AM, diagnostic_AM, and retrospective_AM (where AM stands for “assessment in MSE”).

prelim_AM is designed to evaluate the configuration of the assessment model before running the closed-loop simulation. Given an operating model, this function generates data from the historical period of the MSE, applies the assessment model to those data, then returns the convergence rate of the model. A list of Assessment objects are returned and can be used for troubleshooting. Additional arguments are passed to the Assess function via ...:

prelim_AM(DLMtool::testOM, DD_TMB, ...)
#> Generating Hist object from OM object via runMSE:
#> Loading operating model
#> Optimizing for user-specified movement
#> Optimizing for user-specified depletion
#> Calculating historical stock and fishing dynamics
#> Calculating MSY reference points
#> Calculating B-low reference points
#> Calculating reference yield - best fixed F strategy
#> Returning historical simulations
#> Running DD_TMB with 48 simulations for DLMtool::testOM.
#> Assessments complete.
#> Total time to run 48 assessments: 1.2 seconds
#> 2 of 48 simulations (4.2%) did not converge.
#> See simulation number: 11 21

By default, assessment model results are not saved during the MSE (called by DLMtool::runMSE). However, this feature can be turned on with argument diagnostic in make_MP:

DD_MSY <- make_MP(DD_TMB, HCR_MSY, diagnostic = "full")

For each MSEtool MP, two diagnostic objects can be saved and reported in the DLMtool::DLMenv environment: convergence diagnostics and the Assessment objects from each assessment call. The first can be obtained with diagnostic = "min", while both are returned with diagnostic = "full".

MSEtool MPs may be memory-intensive and cause performance issues or crashes. With large number of simulations, a solution may be to run DLMtool::runMSE() in several batches with different seed numbers. MSE performance over all batches can be calculated by stitching the class MSE objects together with DLMtool::joinMSE().

The diagnosic_AM function plots the convergence information, and retrospective_AM plots the assessment model’s retrospective patterns (for a single simulation in the MSE).

DD_MSY <- make_MP(DD_TMB, HCR_MSY, diagnostic = "full")
DD_4010 <- make_MP(DD_TMB, HCR40_10)
myMSE <- DLMtool::runMSE(OM = DLMtool::testOM, MPs = c("FMSYref", "DD_MSY", "DD_4010"))
diagnostic_AM(myMSE)
retrospective_AM(myMSE, sim = 1, MP = "DD_MSY")

Figure 6. Convergence diagnostics (return by TMB objects and nlminb function) for DD_MSY MP during the MSE. Here, the assessment model was applied beginning in year 50 of the MSE and repeated every 4th year. In MSEtool, convergence is defined by a positive-definite Hessian matrix (top middle figure). In the case of non-convergence, the previous management recommendation is used.

Figure 7. Retrospective analysis of the Delay-Difference model in the DD_MSY MP during the MSE for simulation #1. Operating model (true) values of spawning biomass (SSB), relative spawning biomass (SSB_SSBMSY), spawning biomass depletion (SSB_SSB0), fishing mortality (F), relative F (F_FMSY), and vulnerable bimoass (VB) are in dark black. Colored lines indicate model estimates sequentially over the projection years. Note that this assessment model assumes the vulnerable biomass to be the spawning biomass.

Once it is determined that the MSEtool assessment models are performing adequately (no major convergence issues), then the performance of the MPs that use these models can be evaluated with the DLMtool package.

5.2 Obtaining management recommendations

Like all other output MPs, DLMtool::TAC() can be used to obtain a catch limit:

DD_MSY <- make_MP(DD_TMB, HCR_MSY)
DD_4010 <- make_MP(DD_TMB, HCR40_10)
snapper_TAC <- TAC(Red_snapper, MPs = c("DD_MSY", "DD_4010"))
plot(snapper_TAC)

Figure 8. Stochastic TAC recommendations from the Delay-Difference model with two harvest control rules. When the 40-10 harvest control rule is applied, the prescribed TAC is close to zero because the estimated terminal year biomass is 0.13 in the assessment model.