Regression model
目次
Classification of Regression models
Independent variable (exposure) | ||||
---|---|---|---|---|
Univariable (single variable) | Multivariable (multiple variables) | How to derive coefficients [math]\displaystyle{ b_i }[/math] | ||
Dependent variable (outcome) |
Continuous |
|
|
Least squares method |
Binary |
|
|
Maximum likelihood estimation method | |
Multinominal ≥ 3 |
|
|
Maximum likelihood estimation method | |
Ordinal |
|
|
Maximum likelihood estimation method | |
Rate ratio |
|
Maximum likelihood estimation method | ||
Survival time |
|
Maximum likelihood estimation method |
†'Multivariable' can be rephrased as 'Multiple'; Multivariable is NOT equal to 'Multivariate'!!
How to convert coefficient of binary logistic regressoin to odds ratio
About a binary logistic regression formula,
- [math]\displaystyle{ \begin{align} & \log Y = \\ & \log \left ( \frac{p}{1-p} \right ) = a + b_1X_1 + b_2X_2 + b_3X_3 + \cdots \end{align} }[/math]
adding 1 to [math]\displaystyle{ X_1 }[/math] makes probablity [math]\displaystyle{ p\prime }[/math] as,
- [math]\displaystyle{ \log \left ( \frac{p\prime}{1-p\prime} \right ) = a + b_1(X_1 + 1) + b_2X_2 + b_3X_3 + \cdots }[/math]
Subtraction of above two equations makes,
- [math]\displaystyle{ \begin{align} \log \left ( \frac{p\prime}{1-p\prime} \right ) - \log \left ( \frac{p}{1-p} \right ) & = b_1(X_1 + 1) - b_1X_1 \\ & = b_1 \\ \Leftrightarrow \frac {\left ( \dfrac{p\prime}{1-p\prime} \right )}{ \left ( \dfrac{p}{1-p} \right ) } & = e^{b_1} \end{align} }[/math]
Because [math]\displaystyle{ \frac {\left ( \dfrac{p\prime}{1-p\prime} \right )}{ \left ( \dfrac{p}{1-p} \right ) } }[/math] is the odds ratio of the probablity [math]\displaystyle{ p\prime }[/math] when 1 is added to [math]\displaystyle{ X_1 }[/math] to the probability [math]\displaystyle{ p }[/math],
[math]\displaystyle{ \color{red}{e^{b_1}} }[/math] or [math]\displaystyle{ \color{red}{\exp (b_1)} }[/math] is the conversion of coefficient [math]\displaystyle{ b_1 }[/math] to obtain the odds ratio of outocome probabilities before and after variable [math]\displaystyle{ X_1 }[/math] gains 1.
Generalized linear model
Penalized multivariable logistic regression model
- Penalized Logistic Regression Essentials in R: Ridge, Lasso and Elastic Net
- 罰則付き・正則化回帰モデルについて(About penalized/regularized regression model)