Regression model

Basics & Definition

Epidemiology

Odds in statistics and Odds in a horse race

Collider bias

Data distribution

Statistical test

Regression model

Multivariate analysis

Marginal effects

Prediction and decision

Table-related commands in STATA

Missing data and imputation

Classification of Regression models

		Independent variable (exposure)
		Univariable (single variable)	Multivariable (multiple variables)	How to derive coefficients [math]\displaystyle{ b_i }[/math]
Dependent variable (outcome)	Continuous	Single linear regression [math]\displaystyle{ Y = a + bX }[/math]	Multivariable† linear regression [math]\displaystyle{ Y = a + b_1X_1 + b_2X_2 + b_3X_3 + \cdots }[/math]	Least squares method
	Binary	Single binary logistic regression [math]\displaystyle{ \log Y = a + bX }[/math] where [math]\displaystyle{ Y }[/math] is odds of outcome [math]\displaystyle{ \frac{p}{1-p} }[/math]	Multivariable† binary logistic regression [math]\displaystyle{ \log Y = a + b_1X_1 + b_2X_2 + b_3X_3 + \cdots }[/math] where [math]\displaystyle{ Y }[/math] is odds of outcome [math]\displaystyle{ \frac{p}{1-p} }[/math]	Maximum likelihood estimation method
	Multinominal ≥ 3	Single multinominal logistic regression	Multivariable† multinominal logistic regression	Maximum likelihood estimation method
	Ordinal	Single ordinal logistic regression	Multivariable† ordinal logistic regression	Maximum likelihood estimation method
	Rate ratio		Multivariable Poisson regression [math]\displaystyle{ \log Y = a + b_1X_1 + b_2X_2 + b_3X_3 + \cdots }[/math] where [math]\displaystyle{ Y }[/math] is rate ratio [math]\displaystyle{ \frac{events_1/person \text{-} time_1}{events_2/person \text{-} time_2} }[/math]	Maximum likelihood estimation method
	Survival time		Multivariable proportional hazard regression = Cox hazard regression [math]\displaystyle{ \log h(T) = \log h_0(T) + b_1X_1 + b_2X_2 + b_3X_3 + \cdots }[/math] where [math]\displaystyle{ h(T) }[/math] is the hazard at time [math]\displaystyle{ T }[/math] and [math]\displaystyle{ h_0(T) }[/math] is the baseline hazard at time [math]\displaystyle{ T }[/math]	Maximum likelihood estimation method

†'Multivariable' can be rephrased as 'Multiple'; Multivariable is NOT equal to 'Multivariate'!!

How to convert coefficient of binary logistic regressoin to odds ratio

About a binary logistic regression formula,

[math]\displaystyle{ \begin{align} & \log Y = \\ & \log \left ( \frac{p}{1-p} \right ) = a + b_1X_1 + b_2X_2 + b_3X_3 + \cdots \end{align} }[/math]

adding 1 to [math]\displaystyle{ X_1 }[/math] makes probablity [math]\displaystyle{ p\prime }[/math] as,

[math]\displaystyle{ \log \left ( \frac{p\prime}{1-p\prime} \right ) = a + b_1(X_1 + 1) + b_2X_2 + b_3X_3 + \cdots }[/math]

Subtraction of above two equations makes,

[math]\displaystyle{ \begin{align} \log \left ( \frac{p\prime}{1-p\prime} \right ) - \log \left ( \frac{p}{1-p} \right ) & = b_1(X_1 + 1) - b_1X_1 \\ & = b_1 \\ \Leftrightarrow \frac {\left ( \dfrac{p\prime}{1-p\prime} \right )}{ \left ( \dfrac{p}{1-p} \right ) } & = e^{b_1} \end{align} }[/math]

Because [math]\displaystyle{ \frac {\left ( \dfrac{p\prime}{1-p\prime} \right )}{ \left ( \dfrac{p}{1-p} \right ) } }[/math] is the odds ratio of the probablity [math]\displaystyle{ p\prime }[/math] when 1 is added to [math]\displaystyle{ X_1 }[/math] to the probability [math]\displaystyle{ p }[/math],

[math]\displaystyle{ \color{red}{e^{b_1}} }[/math] or [math]\displaystyle{ \color{red}{\exp (b_1)} }[/math] is the conversion of coefficient [math]\displaystyle{ b_1 }[/math] to obtain the odds ratio of outocome probabilities before and after variable [math]\displaystyle{ X_1 }[/math] gains 1.

Penalized multivariable logistic regression model

• Penalized Logistic Regression Essentials in R: Ridge, Lasso and Elastic Net
• 罰則付き・正則化回帰モデルについて（About penalized/regularized regression model）

Restricted cubic spline

• Restricted cubic splinesをStataで実行してみる