Comparisons between Resampling Techniques in Linear Regression: A Simulation Study

Anwar Fitrianto, Punitha Linganathan

Abstract


The classic methods used in estimating the parameters in linear regression need to fulfill some assumptions. If the assumptions are not fulfilled, the conclusion is questionable. Resampling is one of the ways to avoid such problems. The study aims to compare resampling techniques in linear regression. The original data used in the study is clean, without any influential observations, outliers and leverage points.  The ordinary least square method was used as the primary method to estimate the parameters and then compared with resampling techniques. The variance, p-value, bias, and standard error are used as a scale to estimate the best method among random bootstrap, residual bootstrap and delete-one Jackknife. After all the analysis took place, it was found that random bootstrap did not perform well while residual and delete-one Jackknife works quite well. Random bootstrap, residual bootstrap, and Jackknife estimate better than ordinary least square. Is was found that residual bootstrap works well in estimating the parameter in the small sample. At the same time, it is suggested to use Jackknife when the sample size is big because Jackknife is more accessible to apply than residual bootstrap and Jackknife works well when the sample size is big.

Keywords


jackknife; linear; regression; resampling; resampling

Full Text:

PDF

References


M. Alrasheedi, “Parametric and non-parametric bootstrap: A simulation study for a linear regression with residuals from a mixture of Laplace distributions,” European Scientific Journal, vol. 9, no. 12, 2013.

R. F. Gunst and R. L. Mason, Regression analysis and its application: a data-oriented approach. CRC Press, 2018.

A. Althubaiti, “Information bias in health research: definition, pitfalls, and adjustment methods,” Journal of multidisciplinary healthcare, vol. 9, p. 211, 2016.

M. R. Chernick, “Resampling methods,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 2, no. 3, pp. 255–262, 2012.

R. E. McRoberts, S. Magnussen, E. O. Tomppo, and G. Chirici, “Parametric, bootstrap, and jackknife variance estimators for the k-Nearest Neighbors technique with illustrations using forest inventory and satellite image data,” Remote Sensing of Environment, vol. 115, no. 12, pp. 3165–3174, 2011.

R. G. Clark and S. Allingham, “Robust resampling confidence intervals for empirical variograms,” Mathematical Geosciences, vol. 43, no. 2, pp. 243–259, 2011.

S. Sahinler and D. Topuz, “Bootstrap and jackknife resampling algorithms for estimation of regression parameters,” Journal of Applied Quantitative Methods, vol. 2, no. 2, pp. 188–199, 2007.

J. Ma et al., “Probabilistic forecasting of landslide displacement accounting for epistemic uncertainty: a case study in the Three Gorges Reservoir area, China,” Landslides, vol. 15, no. 6, pp. 1145–1153, 2018.

C. Wan, Z. Xu, Y. Wang, Z. Y. Dong, and K. P. Wong, “A hybrid approach for probabilistic forecasting of electricity price,” IEEE Transactions on Smart Grid, vol. 5, no. 1, pp. 463–470, 2013.

G. A. Nelson, “Cluster sampling: a pervasive, yet little recognized survey design in fisheries research,” Transactions of the American Fisheries Society, vol. 143, no. 4, pp. 926–938, 2014.

P. Phaladiganon, S. B. Kim, V. C. P. Chen, J.-G. Baek, and S.-K. Park, “Bootstrap-based T 2 multivariate control charts,” Communications in Statistics—Simulation and Computation®, vol. 40, no. 5, pp. 645–662, 2011.

X. Li, W. Wong, E. L. Lamoureux, and T. Y. Wong, “Are linear regression techniques appropriate for analysis when the dependent (outcome) variable is not normally distributed?,” Investigative ophthalmology & visual science, vol. 53, no. 6, pp. 3082–3083, 2012.

Z. Y Algamal and K. B Rasheed, “Re-sampling in linear regression model using jackknife and bootstrap,” IRAQI JOURNAL OF STATISTICAL SCIENCES, vol. 10, no. 18, pp. 59–73, 2010.

J. Shao and D. Tu, The jackknife and bootstrap. Springer Science & Business Media, 2012.

U. Beyaztas and A. Alin, “Sufficient jackknife-after-bootstrap method for detection of influential observations in linear regression models,” Statistical Papers, vol. 55, no. 4, pp. 1001–1018, 2014.

D. C. Montgomery, E. A. Peck, and G. G. Vining, Introduction to linear regression analysis. John Wiley & Sons, 2021.




DOI: https://doi.org/10.18860/ca.v7i3.14550

Refbacks

  • There are currently no refbacks.


Copyright (c) 2022 Anwar Fitrianto, Punitha Linganathan

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Editorial Office
Mathematics Department,
Universitas Islam Negeri Maulana Malik Ibrahim Malang
Gajayana Street 50 Malang, East Java, Indonesia 65144
Faximile (+62) 341 558933
e-mail: cauchy@uin-malang.ac.id

Creative Commons License
CAUCHY: Jurnal Matematika Murni dan Aplikasi is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.