The Origins of adjusted R², AIC and BIC: who invented them, how, and why they look the way they do
课件给了三个准则的用法，但没说它们为什么长这样。本文回答四个问题：谁发明的？在什么问题上被逼出来的？推导的关键一步是什么？为什么惩罚项偏偏是 p/(n−p−1)、2(p+1) 和 (p+1)·log n？
The lecture slides give the rules of use but not the reasoning behind the formulas. This booklet answers four questions: who invented each criterion, what problem forced it into existence, what the key step of the derivation is, and why the penalty is exactly p/(n−p−1), 2(p+1) and (p+1)·log n.
一句话剧透——三个准则其实在估计三个不同的东西：adjusted R² 估计总体 R²，AIC 估计样本外预测的信息损失，BIC 估计模型的后验概率。目标不同，惩罚自然不同。抓住这条主线，后面所有公式都不再是死记。
Spoiler: the three criteria estimate three different quantities — adjusted R² estimates the population R², AIC estimates the out-of-sample information loss, and BIC estimates the posterior probability of a model. Different targets, different penalties.
The common problem
R² 的致命缺陷不是「算得不准」，而是「问错了问题」：它衡量的是模型对已经用过的数据拟合得多好。而加一个变量总能让拟合变好——哪怕这个变量纯属噪声——因为最小二乘会利用这份数据里的一切偶然波动。
The flaw of R² is not inaccuracy but that it answers the wrong question: how well the model fits the data it was fitted on. Adding any variable improves the fit, even a pure-noise variable, because least squares exploits every chance fluctuation in that dataset.
于是所有惩罚项都在做同一件事：把「样本内的乐观」扣回去。三个准则的差别，只在于它们如何度量这份乐观、以及想还原到哪个目标。
Every penalty does the same job: subtract the in-sample optimism. The three criteria differ only in how they measure that optimism and which target they restore.
| 准则 | 真正在估计什么 | 乐观量的来源 | 惩罚形式 |
|---|---|---|---|
| adjusted R² | 总体 R²（ρ²）：模型在新样本上能解释的方差比例 | 用同一份数据既挑变量又评估 | R² 上的乘性收缩：R²adj = 1 − (1−R²)(n−1)/(n−p−1) |
| AIC | 样本外预测的 KL 信息损失 ×(−2) | 极大似然估计「迁就」了这份数据 | 加性 2(p+1) |
| BIC | 模型的后验概率 ×(−2) | 忽略了参数空间体积随 n 的增长 | 加性 (p+1)·log n |
左列是三个准则，中列是它们的「靶子」。当有人问「到底该用哪个」时，正确的回答不是「AIC 好」或「BIC 好」，而是「你要估计什么」：要预测 → AIC；要挑出真变量（稀疏、可解释）→ BIC；要一个能与 F 检验、t 检验对话的方差解释比例 → adjusted R²。
The middle column is the target. “Which one should I use?” is answered not by “AIC is better” but by “what are you estimating?”: prediction → AIC; identifying the true sparse model → BIC; a variance-explained measure that talks to F- and t-tests → adjusted R².
adjusted R² 是三个准则里最早的，诞生于 1930 年前后的心理测量学与相关分析传统，而不是回归诊断传统。通常归于 Mordecai Ezekiel（1930 年《Methods of Correlation Analysis》一书），其中给出的修正就是今天被直接称为「adjusted R²」的那一个；次年 R. J. Wherry（1931）从「多元相关系数的收缩（shrinkage）」角度给出了等价公式，因此这一族修正有时也叫 Wherry 公式、shrunken R²、cross-validity 修正 [1, 2]。
Adjusted R² is the oldest of the three and came out of the psychometric/correlation tradition of around 1930 rather than out of regression diagnostics. It is usually credited to Mordecai Ezekiel (1930, Methods of Correlation Analysis); R. J. Wherry (1931) derived an equivalent formula from the “shrinkage” of the multiple correlation coefficient, which is why this family of corrections is also called the Wherry formula, shrunken R², or the cross-validity correction.
他们当时面对的不是「选模型」，而是一个更尖锐的问题：研究者从 p 个变量里挑出与 y 相关最强的组合，算出 R² = 0.6。可是如果换一批样本，这个 R² 还会是 0.6 吗？
Their problem was not model selection but something sharper: a researcher picks the combination of p variables most correlated with y and reports R² = 0.6. Would that 0.6 survive on a fresh sample?
答案是：必然下降。因为「挑」这个动作本身就在消耗数据里的偶然性。这种下降当时被称为收缩（shrinkage），而收缩量正是今天所谓的选择偏差（selection bias）。
No — it must fall, because the act of choosing consumes the luck embedded in the sample. That fall was called shrinkage, and it is exactly what we now call selection bias.
R² = 1 − SSE/SST。分子 SSE 是「模型没解释掉的部分」，但拟合时我们恰恰是让 SSE 尽可能小。p+1 个参数会主动去贴合这份数据里的 p+1 个方向，因此即使模型全是噪声，SSE 也会比它「应有的」小一些。这个人为的缩小量，就是乐观量。
In R² = 1 − SSE/SST the numerator is what the model failed to explain, and fitting deliberately makes SSE as small as possible. The p+1 parameters actively fit p+1 directions of this particular sample, so even a noise model ends up with an SSE smaller than it deserves. That artificial reduction is the optimism.
用 n 个数据点去估计 p+1 个参数，等于花掉了 p+1 个自由度。残留的自由度只有 n−p−1 个。所以凡是「用方差来打分」的准则，分母都得除以 n−p−1 而不是 n。
Estimating p+1 parameters from n points spends p+1 degrees of freedom, leaving n−p−1. Any criterion built on a variance estimate must therefore divide by n−p−1, not by n.
把 R² 写成两个方差估计之比。若模型设定正确，E[SSE/(n−p−1)] = σ²（残差方差的无偏估计），E[SST/(n−1)] = σy² = Var(系统部分) + σ²。于是
也就是总体 R²：如果有无穷多的数据、模型结构不变，这个模型在总体上能解释的方差比例。
Write R² as a ratio of two variance estimates. If the model is correct, SSE/(n−p−1) is unbiased for σ² and SST/(n−1) is unbiased for the total variance. Adjusted R² therefore estimates the population R², ρ²: the fraction of variance the model would explain with unlimited data.
分母用的是 SST/(n−1)：SST 是相对样本均值 ȳ 的离差平方和，而 ȳ 本身也花掉了 1 个自由度，所以除以 n−1。分子用的是 SSE/(n−p−1)：模型花掉了 p+1 个自由度（p 个斜率加 1 个截距），所以除以 n−p−1。两个都是「每个自由度分摊的平方和」，即均方（mean square）。R²adj = 1 − MSE/MST 因此本质上是两个无偏方差估计之比，这正是它「长得像 R²、却又比 R² 公道」的原因。
The denominator divides SST by n−1, because SST is measured about the sample mean, which itself costs one degree of freedom. The numerator divides SSE by n−p−1, because the model spends p+1 degrees of freedom (p slopes plus the intercept). Both are mean squares, so R²adj = 1 − MSE/MST is a ratio of two unbiased variance estimates.
把上面两式相除，等价写法是
(n−1)/(n−p−1) 就是收缩因子：它把「样本里的未被解释比例」1−R² 放大，等价于把 R² 缩小。展开最后一式可得课件里那个更好用的形式：
每个自变量要交的「过路费」是 (1−R²)/(n−p−1)：模型本来解释得越差（1−R² 越大）、数据越少（n−p−1 越小），加变量的代价就越高。这就把「惩罚项」讲成了一句人话。
Equivalently 1 − R²adj = (1−R²)(n−1)/(n−p−1); the factor is a shrinkage factor. Expanding gives the slide’s form R²adj = R² − (1−R²)p/(n−p−1), so each predictor costs a toll of (1−R²)/(n−p−1) — higher when the model explains little and when the sample is small.
这一条最能说明「惩罚项长这样」的含义。向模型里加一个变量时，可以证明：
其中 t 正是该变量的 t 统计量（等价地，偏 F = t² > 1）。
The cleanest reading of the penalty: adding one variable raises adjusted R² if and only if |t| > 1 for that variable (equivalently, the partial F = t² > 1).
注意这个门槛不是 1.96。也就是说，用 adjusted R² 选模型，等价于「只要 t 的绝对值超过 1 就留下这个变量」——比 5% 显著性宽松得多。这解释了为什么 adjusted R² 选出的模型往往比 F 检验看上去该有的更大，也预告了后面 AIC（门槛 √2 ≈ 1.41）和 BIC（门槛 √log n）的差别（见 §5.3）。
The threshold is not 1.96: selecting by adjusted R² keeps any variable whose t-statistic exceeds one in absolute value, far looser than a 5% test. This is why adjusted R² tends to retain more variables, and it sets up the comparison with AIC (threshold √2 ≈ 1.41) and BIC (threshold √log n) in §5.3.
设加入变量前后残差平方和为 SSEk 与 SSEk+1 = SSEk − ΔSSE。R²adj 上升 ⟺ SSEk+1/(n−k−1) < SSEk/(n−k) ⟺ (n−k)SSEk+1 < (n−k−1)(SSEk+1 + ΔSSE) ⟺ ΔSSE > SSEk+1/(n−k−1)，而右边正是 t² 的分母形式。
Write SSEk+1 = SSEk − ΔSSE; the inequality reduces to ΔSSE > SSEk+1/(n−k−1), whose left-hand ratio is t².
AIC 由日本统计学家赤池弘次（Hirotugu Akaike, 1927–2009）提出。它最初的名字就是朴素的「一个信息准则」（an information criterion）。英文世界最早听到它，是 Akaike 在 1971 年布达佩斯国际信息论研讨会上的报告；会议论文集 1973 年出版，但那一篇只是概念的非正式陈述。正式论文是 1974 年发表在 IEEE Transactions on Automatic Control 上的《A new look at the statistical model identification》，这也是今天引用最多的那一篇 [5, 6]。
AIC was formulated by the Japanese statistician Hirotugu Akaike (1927–2009). Its original name was simply “an information criterion”. It was first announced in English at the 1971 International Symposium on Information Theory in Budapest; the proceedings appeared in 1973 but were only an informal presentation of the concepts. The first formal publication was the 1974 paper “A new look at the statistical model identification” in IEEE Transactions on Automatic Control, still the standard citation.
后续的重要补丁：Takeuchi (1976) 把推导所需的强假设大幅放宽（得到 TIC）；Sugiura (1978) 针对线性回归给出小样本修正（AICc）；Hurvich & Tsai (1989) 把它推广到更多情形 [8, 11, 12]。
Later refinements: Takeuchi (1976) weakened the assumptions (TIC); Sugiura (1978) gave the small-sample correction for linear regression; Hurvich & Tsai (1989) extended it.
设数据由某个未知过程 f 生成，我们手里有候选模型 g₁, g₂, …。衡量一个模型有多接近真相，最自然的尺子是 Kullback–Leibler 散度（Kullback & Leibler, 1951）：
第一项只与未知的 f 有关，比较模型时是常数；所以「选最接近真相的模型」等价于「选 Ef[log g] 最大的模型」。问题在于：f 未知，这个期望算不出来，只能用样本内的 log L(θ̂) 去估——而它恰好是乐观的。AIC 的全部内容，就是把这个乐观量扣掉。
The first term depends only on the unknown f and is a constant across models, so choosing the closest model means maximising Ef[log g]. That expectation cannot be computed, and the in-sample log L(θ̂) that replaces it is optimistic. The whole of AIC is the correction of that optimism.
先在最容易算的场合把答案算出来：正态误差线性模型 y = Xβ + ε，ε ∼ N(0, σ²I)，参数个数 k = p+1（截距 + p 个斜率），σ² 视为已知或并入常数项（这是 R 里 AIC() 的约定）。
Compute the answer in the one case where everything is exact: the Gaussian linear model with k = p+1 regression coefficients, treating σ² as known or absorbing it into the constant (the convention used by R’s AIC()).
注意这里已经是 n−k 而不是 n——拟合「用掉」了 k 个自由度，模型因此显得比它真实水平更好。这个 −k 就是乐观量的具体形式。
Note the n−k rather than n: fitting consumed k degrees of freedom, making the model look better than it is. That −k is the optimism.
取一个新观测点 x*（不在训练集里），其响应为 y*。杠杆的定义给出
对所有新点平均，杠杆的平均值是 h̄ = k/n（因为 Σᵢhᵢᵢ = k）。于是 n 倍的平均样本外平方误差为
Averaging the leverage over new points gives h̄ = k/n since Σᵢhᵢᵢ = k, so n times the average out-of-sample squared error is (n+k)σ².
把 2k 加回去，就得到「样本外表现」的（近似）无偏估计：
2k 不是谁拍脑袋定的常数，而是最小二乘用掉 k 个自由度的价格。这也解释了为什么 AIC 与 R²adj 在精神上是亲戚：两者都在补偿被拟合吃掉的自由度，只是一个用加法（对偏差加 2k），一个用除法（把方差除以 n−k）。
Adding 2k back gives an approximately unbiased estimate of out-of-sample performance. The 2k is not arbitrary — it is the price of the k degrees of freedom that least squares consumes. This also explains the kinship with adjusted R²: both compensate for spent degrees of freedom, one additively, one by division.
上面的算法用到了正态性与线性。Akaike 的一般结论是（f 在候选模型类内、n 足够大）：
即「样本内偏差」比「样本外偏差」系统性地小约 2k。因此 AIC 是期望样本外偏差的近似无偏估计，等价地说，AIC 估计了 2·DKL(f ‖ ĝθ̂) 加上一个所有模型共有的常数。Akaike 的原话精神是：用可计算的样本内似然，加上参数个数这一个便宜的修正，就能逼近不可计算的样本外信息损失——这就是它成为二十世纪最有影响力的模型选择工具的原因 [6, 18]。
Akaike’s general result is that the in-sample deviance is systematically smaller than the out-of-sample deviance by about 2k, so AIC is an approximately unbiased estimator of the expected out-of-sample deviance (equivalently of twice the KL divergence plus a common constant). His insight was that a cheap correction — one term in the number of parameters — turns a computable in-sample quantity into a good proxy for an incomputable out-of-sample one.
把似然换成「偏差」（deviance）D = −2 log L，是为了两件事：一是让「最优」变成「最小」（−2 log L 越小拟合越好）；二是让正态模型的偏差与 RSS/σ² 直接对应。在这个尺度下，每个参数的价格恰好是 2——因为上一步算出的乐观量是 2k。若改用 log L 本身，AIC 就写成 log L − k；若用 χ² 尺度，价格是 1。「2」是单位为「负二倍对数似然」的结果，不是额外的常数。
Working with the deviance D = −2 log L makes “best” mean “smallest” and aligns the Gaussian case with RSS/σ². On that scale each parameter costs exactly 2, because the optimism is 2k. In log-likelihood units AIC would read log L − k; on a χ² scale it would be 1. The “2” is a unit convention, not an extra constant.
把正态似然代入 −2 log L + 2k，得
这正是课件上 AIC = −2 log(L) + 2(p+1) 的线性模型版本（k = p+1）。比较模型时两边常数相同，可以直接比较。注意：如果把 σ² 也当作待估参数，k 要再 +1，两种约定相差一个常数，不影响同一批数据的模型排序，但会影响不同论文之间数字的对照。
Substituting the Gaussian likelihood gives the linear-model forms shown, matching the slide’s AIC = −2 log L + 2(p+1) with k = p+1. If σ² is also counted as estimated, k increases by one; the two conventions differ by a constant, so rankings are unaffected but numbers across papers may not match.
2k 的修正是渐近的。样本量小时，实际的乐观量大于 2k，AIC 因此倾向选出参数过多的模型（过拟合）。Sugiura (1978) 对线性回归算出了精确的有限样本修正，Hurvich & Tsai (1989) 把它推广 [11, 12]：
多出来的这一项：n→∞ 时趋于 0（回到 AIC）；分母 n−k−1 与 adjusted R² 的分母一模一样——它们都在补偿同一件事，只是 AICc 是二阶才补上。经验规则：当 n/k < 40 时应优先用 AICc。
The correction vanishes as n→∞ and has the same denominator as adjusted R². Rule of thumb: prefer AICc when n/k < 40.
BIC 由 Gideon E. Schwarz（当时在贝尔实验室）在 1978 年提出，论文题为《Estimating the dimension of a model》，发表在 The Annals of Statistics 第 6 卷第 2 期 461–464 页 [9]。它出奇地短，也不谈「模型选择」这个大题目，只解决一个具体问题：如何估计模型的维数（例如多项式的次数）。方法是在贝叶斯框架下，用拉普拉斯方法近似模型证据（marginal likelihood），得到一个带 log n 惩罚的准则。
BIC was introduced by Gideon E. Schwarz (then at Bell Labs) in 1978 in the four-page paper “Estimating the dimension of a model”, The Annals of Statistics 6(2), 461–464. It addresses one narrow question — how to estimate a model’s dimension (say, the degree of a polynomial) — by approximating the Bayesian model evidence with Laplace’s method, which produces a penalty of log n.
因此它又叫 Schwarz 准则（SIC、SBC、SBIC 都是它）。同一种惩罚形式此前/同期也独立出现过：Akaike (1977) 在「熵最大原理」一文中给出过 log n 版本，Rissanen (1978) 的 MDL（最短描述长度）原理也导出同样形式 [7, 10]。所以严格说，「BIC 是 Schwarz 提出的」是就这篇论文而言，思想是多人同时到达的。
It is therefore also called the Schwarz criterion (SIC, SBC, SBIC). The same penalty appeared independently around the same time: Akaike (1977) gave a log n version from the entropy maximisation principle, and Rissanen’s (1978) minimum description length principle leads to the same form. Schwarz’s paper is the canonical citation, not the only origin of the idea.
AIC 问的是「哪个模型预测新数据最好」；BIC 问的是一个完全不同的问题：看了这些数据之后，哪个模型最可能是产生数据的那个？这是贝叶斯问题，答案就是模型的后验概率：
分母对所有候选模型都一样，所以比较模型只需比较模型证据 p(y | Mj)——也就是似然在参数先验上的加权平均（marginal likelihood）。两个模型证据之比就是贝叶斯因子，而 BIC 正是 −2 log 尺度上的贝叶斯因子的大样本近似。
AIC asks which model predicts new data best; BIC asks a different question: given these data, which model is most likely to have generated them? That is the posterior model probability, and comparing models reduces to comparing the model evidence p(y | Mj), the likelihood averaged over the prior. The ratio of two evidences is the Bayes factor, and BIC is its large-sample approximation on the −2 log scale.
把证据写成 p(y | M) = ∫ eℓ(θ) dθ，其中 ℓ(θ) = log[p(y | θ, M)·π(θ | M)]。当 n 大时，被积函数在众数 θ̃ 附近急剧集中，于是可以用二阶泰勒展开（拉普拉斯方法）：
其中 k 是参数个数，I 是 Fisher 信息矩阵。第二步是关键：观测信息量随样本量线性增长，因此行列式含因子 nk。两边取 −2 log：
丢掉 O(1) 项（它们在 n→∞ 时被 k log n 支配），就得到
Write the evidence as an integral of eℓ(θ) and apply Laplace’s method: the integrand concentrates near the mode, the second-derivative matrix scales like n times the Fisher information, so the determinant contributes nk. Taking −2 log gives −2 log L + k log n plus terms that stay bounded as n grows; dropping those leaves BIC.
惩罚项来自参数空间的体积。参数越多，模型能「伸到」的区域维度越高；而随着数据增多，每个参数方向的估计精度以 1/√n 提高，相应的体积以 nk/2 缩小。取 −2 log 之后，每个参数恰好贡献 log n——所以惩罚按参数个数线性、按样本量对数增长。对比 AIC：AIC 用的是固定的每参数价格 2，因为它的推导只保留到 O(1)（不关心体积），这也正是两者在 n 大时行为分道扬镳的根源。
The penalty comes from the volume of parameter space: more parameters means a higher-dimensional region to search, and as n grows the estimation uncertainty in each direction shrinks like 1/√n, shrinking that volume like nk/2. After −2 log, each parameter contributes exactly log n: linear in the number of parameters, logarithmic in the sample size. AIC instead charges a flat price of 2 per parameter because its derivation keeps only O(1) terms — which is exactly why the two diverge as n grows.
既然 AIC 的惩罚是常数 2、BIC 是 log n，中间自然还有空间。Hannan–Quinn 准则（HQC, 1979）取
惩罚比 BIC 轻、比 AIC 重，并具有较弱的相合性 [14]。它说明「惩罚强度」不是二选一，而是一条连续谱。
Between AIC’s constant 2 and BIC’s log n sits the Hannan–Quinn criterion, −2 log L + 2k log log n, with a lighter penalty than BIC and a heavier one than AIC. Penalty strength is a continuum, not a binary choice.
把上面所有推导并排放，会发现三个准则（加上 Cp、HQC）都是同一个骨架：
差别只在惩罚的形式、来源，以及它们各自想估计的目标。
Placed side by side, all of them share one skeleton: a fit term plus a penalty that grows with the number of parameters. They differ in the form and origin of the penalty, and in the target they estimate.
| 准则 | 拟合项 | 惩罚 | 等价 |t| 门槛 | 估计目标 | 相合性 |
|---|---|---|---|---|---|
| adjusted R² | R²（或 SSE/(n−k)） | −(1−R²)·p/(n−p−1) | > 1 | 总体 R² | 不适用 |
| AIC | −2 log L | 2k | > √2 ≈ 1.414 | 样本外 KL 信息损失 | 否（efficient） |
| AICc | −2 log L | 2k + 2k(k+1)/(n−k−1) | 略高于 √2 | 同上（小样本） | 否 |
| BIC | −2 log L | k·log n | > √log n | 后验模型概率 | 是（consistent） |
| HQC | −2 log L | 2k·log log n | 介于两者 | 折中 | 弱相合 |
| Mallows’ Cp | SSEp/σ̂²full | −(n−2k) | ≈ AIC | 预测平方误差 SSPE | 否 |
2k 与 k log n 的大小关系只取决于 n：log n > 2 ⟺ n > e² ≈ 7.39。也就是说，只要样本量超过 8 个，BIC 对每个参数的惩罚就已经比 AIC 重，而且差距随 n 越来越大：
The comparison of 2k with k log n depends only on n: log n > 2 if and only if n > 7.39. For any sample larger than eight, BIC already penalises each parameter more heavily, and the gap widens with n.
| n | 20 | 50 | 100 | 500 | 1,000 | 10,000 |
|---|---|---|---|---|---|---|
| AIC 每参数惩罚 | 2 | 2 | 2 | 2 | 2 | 2 |
| BIC 每参数惩罚 log n | 3.00 | 3.91 | 4.61 | 6.21 | 6.91 | 9.21 |
| HQC 每参数惩罚 2 log log n | 2.18 | 2.74 | 3.05 | 3.50 | 3.62 | 3.74 |
第 2.5 节证明了「加一个变量，adjusted R² 上升 ⟺ |t| > 1」。同样的推演可以套到 AIC 与 BIC 上（用 AIC ≈ n log(RSS/n) + 2k）：
这解释了所有「玄学」：
| n | 20 | 50 | 100 | 500 | 1,000 |
|---|---|---|---|---|---|
| BIC 门槛 √log n | 1.73 | 1.98 | 2.15 | 2.49 | 2.63 |
| 对应双侧 p 值 | 0.084 | 0.048 | 0.032 | 0.013 | 0.009 |
(The p-values use the large-sample normal approximation; exact values would use tn−k−1.)
设 n = 100，SST = 200，五个候选模型的 RSS 依次为 120, 100, 95, 92, 91.5（RSS 随变量增加而下降，这是必然的）。按三种准则分别打分：
Take n = 100 and SST = 200, with five candidate models whose RSS values are 120, 100, 95, 92, 91.5. Score them with the three criteria:
| p（自变量个数） | RSS | R² | adjusted R² | AIC | BIC | 备注 |
|---|---|---|---|---|---|---|
| 1 | 120.0 | 0.4000 | 0.3939 | 22.2322 | 27.4425 | |
| 2 | 100.0 | 0.5000 | 0.4897 | 6.0000 | 13.8155 | |
| 3 | 95.0 | 0.5250 | 0.5102 | 2.8707 | 13.2914 | ← BIC 选它 |
| 4 | 92.0 | 0.5400 | 0.5206 | 1.6618 | 14.6877 | ← adjusted R² 与 AIC 选它 |
| 5 | 91.5 | 0.5425 | 0.5182 | 3.1169 | 18.7479 | 两个准则都开始变差 |
不能。它们不是检验统计量，没有零分布。ΔAIC 的经验刻度（2–4 有些支持、4–7 支持很弱、> 10 基本可弃）只是启发式 [18]。
No: they are not test statistics and have no null distribution. The usual ΔAIC scale is a heuristic.
不能。它们的绝对值含 n log(RSS/n) 等项，只能比较同一份数据、同一个响应变量下的模型。跨 y 的变换（y 与 log y）也不可比。
No: only models for the same data and the same response can be compared, and never across transformations of y.
自然对数。用 log₁₀ 会改变惩罚强度（log₁₀ n 更小），结果不同。
Natural logarithm. Using base 10 changes the penalty and the answer.
看约定。R 的 AIC() 对 lm 用 k = 系数个数（即 p+1），本讲义与课件沿用这一约定；若把 σ² 也算进去则 k = p+2。二者只差常数，不影响排序。
A matter of convention: R’s AIC() for lm uses k = p+1 coefficients; counting σ² gives k = p+2. Rankings are unaffected.
不是。两个无偏估计之比仍有偏；更好的选择是 Olkin–Pratt 估计量 [3]。
No — a ratio of unbiased estimators is still biased; the Olkin–Pratt estimator is preferable.
因为 2k 是渐近修正；n 小时真实乐观量更大，AIC 会过拟合。n/k < 40 时优先用 AICc。
Because 2k is asymptotic; with small n the true optimism is larger and AIC overfits.
不保证也不追求。AIC 追求预测效率（efficiency），BIC 追求识别正确（consistency）。两者不能兼得——Yang (2005) 证明了在一般情况下无法同时达到最优 [17]。
AIC does not aim at identifying the true model — it aims at predictive efficiency, while BIC aims at consistency. Yang (2005) shows the two goals cannot both be attained in general.
接近但不相同。Cp 按「SSEp + 2kσ̂²」的尺度排序，adjusted R² 按「SSEp/(n−k)」排序，两者的惩罚尺度不同，排序可能不一致（σ² 已知时才严格等价于 AIC）。
Close but not identical: Cp ranks on an additive scale while adjusted R² ranks on a multiplicative one, so they can disagree.
不一定，通常说明信号处于边界——即那个变量带来的改善恰好落在各准则的门槛之间（例如本例 |t| 落在 1 与 √log n 之间）。这时应如实报告并说明取舍，而不是挑一个好看的。
Not necessarily — it usually means the evidence sits exactly between the thresholds. Report both and state the trade-off rather than picking the nicer answer.
不是。AIC 只在渐近意义上等价于 LOO-CV，且假设模型族正确；CV 对错误设定更稳健，也更容易向非技术读者解释。用 CV 复核是最稳妥的收尾。
No: AIC is only asymptotically equivalent to LOO-CV and assumes the model family is correct; CV is more robust and easier to explain.
| adjusted R² | AIC | AICc | BIC | |
|---|---|---|---|---|
| 年份 Year | 1930 | 1973 / 1974 | 1978 / 1989 | 1978 |
| 发明人 Who | Ezekiel（Wherry 1931 等价形式） | Akaike | Sugiura；Hurvich & Tsai | Schwarz |
| 出身 Origin | 相关分析、收缩问题 | 信息论、KL 散度 | AIC 的有限样本修正 | 贝叶斯模型证据 |
| 惩罚 Penalty | 乘性收缩 | 2k | 2k + 2k(k+1)/(n−k−1) | k log n |
| 方向 Direction | 越大越好 | 越小越好 | 越小越好 | 越小越好 |
| 目标 Target | 总体 R² | 预测 | 小样本预测 | 识别真模型 |
| 嵌套要求 Nested? | 不需要 | 不需要 | 不需要 | 不需要 |
This page was generated by DeepSeek 4.1 Flash from the course booklet (originally in PDF); the bilingual text and derivations follow the source document.
本页由 DeepSeek 4.1 Flash 依据课程讲义 PDF 生成，中英文内容与推导均按原讲义整理。