Skip to Main content Skip to Navigation
Journal articles

Ultra-high dimensional variable screening via Gram–Schmidt orthogonalization

Abstract : Independence screening procedure plays a vital role in variable selection when the number of variables is massive. However, high dimensionality of the data may bring in many challenges, such as multicollinearity or high correlation (possibly spurious) between the covariates, which results in marginal correlation being unreliable as a measure of association between the covariates and the response. We propose a novel and simple screening procedure called Gram–Schmidt screening (GSS) by integrating the classical Gram–Schmidt orthogonalization and the sure independence screening technique, which takes into account high correlations between the covariates in a data-driven way. GSS could successfully discriminate between the relevant and the irrelevant variables to achieve a high true positive rate without including many irrelevant and redundant variables, which offers a new perspective for screening method when the covariates are highly correlated. The practical performance of GSS was shown by comparative simulation studies and analysis of two real datasets.
Document type :
Journal articles
Complete list of metadatas

https://hal-cnam.archives-ouvertes.fr/hal-02557084
Contributor : Gilbert Saporta <>
Submitted on : Tuesday, April 28, 2020 - 2:27:08 PM
Last modification on : Thursday, April 30, 2020 - 1:20:25 AM

Identifiers

Collections

Citation

Huiwen Wang, Ruiping Liu, Shanshan Wang, Zhichao Wang, Gilbert Saporta. Ultra-high dimensional variable screening via Gram–Schmidt orthogonalization. Computational Statistics, Springer Verlag, 2020, ⟨10.1007/s00180-020-00963-7⟩. ⟨hal-02557084⟩

Share

Metrics

Record views

28