Method Replacement Strategy
Source:vignettes/Results_Method_Replacement.Rmd
Results_Method_Replacement.RmdOverview
To handle cases where methods fail to converge, a method replacement strategy is implemented. This approach emulates what a data analyst might do in practice: if a sophisticated method fails to converge, they fall back to a simpler, more robust method. The replacement is applied sequentially: if the first replacement method also fails, the next method in the sequence is tried, and so on.
This strategy allows method performance to be evaluated in a more realistic scenario where non-convergence is handled pragmatically. However, it should be noted that results with replacement no longer reflect “pure” method performance, as they may combine estimates from multiple different methods.
Replacement Specification
The table below shows the replacement sequence for each method. When
a method fails to converge, the methods in the “Replacement Sequence”
column are applied in order until convergence is achieved or all
replacement options are exhausted. An em-dash — indicates
no replacement (the method is assumed to always converge).
| Method | Replacement Sequence |
|---|---|
| mean (default) | — |
| FMA (default) | — |
| RMA (default) | FMA (default) |
| PET (default) | WLS (default) → FMA (default) |
| PEESE (default) | WLS (default) → FMA (default) |
| PETPEESE (default) | WLS (default) → FMA (default) |
| WLS (default) | FMA (default) |
| WILS (default) | WLS (default) → FMA (default) |
| WAAPWLS (default) | WLS (default) → FMA (default) |
| trimfill (default) | RMA (default) → FMA (default) |
| SM (3PSM) | RMA (default) → FMA (default) |
| SM (4PSM) | SM (3PSM) → RMA (default) → FMA (default) |
| puniform (default) | RMA (default) → FMA (default) |
| puniform (star) | RMA (default) → FMA (default) |
| pcurve (default) | RMA (default) → FMA (default) |
| EK (default) | WLS (default) → FMA (default) |
| AK (AK1) | RMA (default) → FMA (default) |
| AK (AK2) | AK (AK1) → RMA (default) → FMA (default) |
| RoBMA (PSMA) | RMA (default) → FMA (default) |
The choice of replacement methods is based on:
- Similarity: Replacement methods should be conceptually similar to the original method when possible.
- Robustness: Simpler methods are generally more robust to convergence issues.
- Practicality: The sequence reflects what applied researchers might reasonably do when faced with convergence failures.