{"id":561,"date":"2014-09-08T01:22:18","date_gmt":"2014-09-08T05:22:18","guid":{"rendered":"http:\/\/zhanxw.com\/blog\/?p=561"},"modified":"2014-09-12T18:39:20","modified_gmt":"2014-09-12T22:39:20","slug":"laplaces-method","status":"publish","type":"post","link":"https:\/\/zhanxw.com\/blog\/2014\/09\/laplaces-method\/","title":{"rendered":"\u62c9\u666e\u62c9\u65af\u65b9\u6cd5(Laplace’s Method)"},"content":{"rendered":"

\u62c9\u666e\u62c9\u65af\u65b9\u6cd5(Laplace’s Method)<\/p>\n

[mathjax]<\/p>\n

\u62c9\u666e\u62c9\u65af\u65b9\u6cd5\u53c8\u79f0\u4e3a\u62c9\u666e\u62c9\u65af\u8fd1\u4f3c\uff08Laplace Approximation\uff09\u3002\u5b83\u53ef\u4ee5\u7528\u6765\u8ba1\u7b97\u4e00\u5143\u6216\u591a\u5143\u79ef\u5206\uff3b1\uff3d\u3002<\/p>\n

\u4e3e\u4f8b\u6765\u8bf4\uff0c\u5047\u8bbe $latex f(x)$ \u662f\u4e00\u7ef4\u51fd\u6570\uff0c\u6211\u4eec\u8981\u8ba1\u7b97
\n[latex] \\int_{-\\infty}^{\\infty} f(x) \\mathrm{d} x [\/latex].<\/p>\n

\u5982\u679c$latex f(x)$\u5f62\u5f0f\u5f88\u590d\u6742\uff0c\u6211\u4eec\u5f80\u5f80\u627e\u4e0d\u5230\u5b9a\u79ef\u5206\u7684\u516c\u5f0f\uff08Close form\uff09\u3002
\n\u5982\u679c\u7528\u6570\u503c\u65b9\u6cd5\u6765\u8ba1\u7b97\u79ef\u5206\uff0c\u8ba1\u7b97\u91cf\u53c8\u5f88\u5927\u3002
\n\u6240\u4ee5\u8981\u60f3\u4e2a\u529e\u6cd5\uff0c\u5f97\u5230\u4e00\u4e2a\u6bd4\u8f83\u7cbe\u786e\u7684\u79ef\u5206\u7ed3\u679c\u3002<\/p>\n

\u62c9\u666e\u62c9\u65af\u65b9\u6cd5\u53ef\u4ee5\u9002\u7528\u4e8e\u8fd9\u79cd\u60c5\u51b5\u3002\u6211\u4eec\u5148\u7528\u6cf0\u52d2\u5c55\u5f00\uff08Taylor Expansion\uff09\uff1a<\/p>\n

$$ f(x) \\approx f(x_0) + f'(x_0)(x-x_0) + \\frac{1}{2}f”(x_0) (x-x_0)^2 $$
\n\u5982\u679c\u9009\u53d6$latex x_0$\u4f7f\u5f97$latex f'(x_0) = 0$\uff0c\u5219\u53ef\u4ee5\u8fdb\u4e00\u6b65\u7b80\u5316\u4e3a
\n$$ f(x) \\approx f(x_0) + \\frac{1}{2}f”(x_0) (x-x_0)^2 $$<\/p>\n

\u518d\u5f15\u5165\u4e00\u4e2a\u5047\u8bbe\uff0c$latex \\int_{-\\infty}^{\\infty} f(x) \\mathrm{d} x $
\n\u53ef\u4ee5\u88ab\u5199\u6210 $latex \\int_{-\\infty}^{\\infty} e^{f(x)} \\mathrm{d} x $ \u7684\u5f62\u5f0f\uff0c\u90a3\u4e48\u5c31\u6709\uff1a<\/p>\n

$$ \\begin{align}
\n\\int_{-\\infty}^{\\infty} e^{f(x)} \\mathrm{d} x
\n&\\approx \\int_{-\\infty}^{\\infty} e^{f(x_0) + \\frac{1}{2}f”(x_0) (x-x_0)^2} \\mathrm{d} x \\\\
\n&= e^{f(x_0)} \\int_{-\\infty}^{\\infty} e^{- \\frac{1}{2}|f”(x_0)| (x-x_0)^2} \\mathrm{d} x \\\\
\n&= e^{f(x_0)} \\sqrt{\\frac{2\\pi}{|f”(x_0)|}}
\n\\end{align}
\n$$<\/p>\n

\u6ce8\u610f\u4e0a\u5f0f\u4e2d$latex f”(x_0)$\u8981\u53d6\u7edd\u5bf9\u503c\u3002\u56e0\u4e3a$latex f'(x_0) = 0$\uff0c$latex x_0$\u662f$latex f(x)$\u7684\u6781\u503c\u70b9\u3002
\n\u5bf9\u4e8e\u6982\u7387\u5bc6\u5ea6\u51fd\u6570\u800c\u8a00\uff0c\u4e00\u822c\u5b83\u4e5f\u662f\u6700\u5927\u503c\u70b9\uff08mode\uff09\uff0c \u56e0\u6b64$latex f”(x_0) < 0$.\n\n\u4ece\u51e0\u4f55\u4e0a\u8bb2\uff0c\u62c9\u666e\u62c9\u65af\u65b9\u6cd5\u662f\u8981\u7528\u4e00\u4e2a$latex e^{-x^2}$\u5f62\u5f0f\u7684\u51fd\u6570\u8fd1\u4f3c$latex f(x)$.\n\u6216\u8005\u8bf4\uff0c\u8981\u7528\u4e00\u4e2a\u957f\u7684\u50cf\u6b63\u6001\u51fd\u6570\u7684\u51fd\u6570\u53d6\u8fd1\u4f3c\uff0c\u8fd9\u6837\u7684\u51fd\u6570\u670d\u4ece$latex N(x_0, 1\/f''(x_0))$.\n\n\u4e0b\u9762\u4e3e\u4e2a\u4f8b\u5b50\u6765\u8ba1\u7b97\n$$\\int_{-\\infty}^{\\infty} x^2 e^{-x^2\/2} \\mathrm{d} x $$.\n\u8fd9\u4e2a\u79ef\u5206\u7684\u7ed3\u679c\u662f$latex \\sqrt{2\\pi}$\uff0c\u8fd9\u53ef\u4ee5\u8fd9\u6837\u8ba1\u7b97\u51fa\u6765\u3002\n\u5047\u8bbe$latex Z$\u662f\u4e00\u4e2a\u6807\u51c6\u6b63\u6001\u5206\u5e03\uff0c\u90a3\u4e48\uff1a\n$$\\int_{-\\infty}^{\\infty} x^2 e^{-x^2\/2} \\mathrm{d} x = \\sqrt{2\\pi} E(Z^2) = \\sqrt{2\\pi} Var(Z) = \\sqrt{2\\pi} = 2.507$$.\n\n\u7528\u62c9\u666e\u62c9\u65af\u65b9\u6cd5\uff1a\n$$\\int_{-\\infty}^{\\infty} x^2 e^{-x^2\/2} \\mathrm{d} x = \\int_{-\\infty}^{\\infty} e^{-x^2\/2 + 2 \\log{x}} \\mathrm{d} x $$.\n\u5bf9$latex f(x)$\u6c42\u5bfc\uff1a\n$$\n\\begin{align}\nf(x) &= -\\frac{x^2}{2} + 2 \\log{x} \\\\\nf'(x) &= -x + \\frac{2}{x} \\\\\nf''(x) &= -1 - \\frac{2}{x^2} \n\\end{align}\n$$\n\n\u4ee4$latex f'(x) = 0$\uff0c\u53ef\u4ee5\u89e3\u51fa$latex x_0 = \\sqrt{2}$.\n\u6211\u4eec\u6709$latex f(x_0) = -1 + \\log{2}$ \u548c $latex f''(x_0) = -2$.\n\u79ef\u5206\u7ed3\u679c\u4e3a\uff1a\n$$\ne^{f(x_0)} \\sqrt{\\frac{2\\pi}{|f''(x_0)|}} = \ne^{-1 + \\log{2}} * \\sqrt{\\frac{2\\pi}{|-2|}} = \\frac{2}{e} \\sqrt{\\pi} = 1.304\n$$\n\n\u6bd4\u8f832.507 \u548c1.304\uff0c\u6211\u4eec\u53d1\u73b0\u7406\u8bba\u7ed3\u679c\u548c\u62c9\u666e\u62c9\u65af\u65b9\u6cd5\u7684\u7ed3\u679c\u51e0\u4e4e\u5dee\u4e86\u4e00\u500d\u3002\n\u4e3a\u5565\u5dee\u522b\u90a3\u4e48\u5927\u5462\uff1f\n\u770b\u4e0b\u9762\u7684\u56fe\uff1a\n\n[caption id=\"attachment_564\" align=\"alignnone\" width=\"300\"]\"Demo<\/a> Demo Laplace Method[\/caption]<\/p>\n

\u6211\u4eec\u53d1\u73b0\u8981\u6c42\u7684\u79ef\u5206\u503c\u662f\u9ed1\u8272\u66f2\u7ebf\u4e0b\u7684\u9762\u79ef\u3002\u62c9\u666e\u62c9\u65af\u65b9\u6cd5\u60f3\u7528\u7070\u8272\u66f2\u7ebf\uff08$latex e^{-\\frac{x^2}{2}}$\uff09\u901a\u8fc7\u62c9\u4f38\u5f97\u5230\u7ea2\u8272\u66f2\u7ebf\uff0c\u7136\u540e\u7528\u7ea2\u8272\u66f2\u7ebf\u4e0b\u7684\u9762\u79ef\u6765\u8fd1\u4f3c\u79ef\u5206\u3002\u4f46\u662f\u7531\u4e8e\u9ed1\u8272\u66f2\u7ebf\u6709\u4e24\u4e2a\u5cf0\u503c\uff0c\u8fd9\u4e2a\u8fd1\u4f3c\u663e\u7136\u4e0d\u7b97\u6210\u529f\u3002
\n\u6240\u4ee5\u62c9\u666e\u62c9\u65af\u65b9\u6cd5\u662f\u6709\u5c40\u9650\u6027\u7684\uff1a\u88ab\u79ef\u5206\u7684\u51fd\u6570\u6709\u4e00\u4e2a\u5cf0\u503c\uff0c\u5e76\u4e14\u548c\u6b63\u6001\u66f2\u7ebf\u957f\u7684\u50cf\u3002\u8fd9\u79cd\u60c5\u51b5\u4e0b\u7684\u8fd1\u4f3c\u624d\u80fd\u6bd4\u8f83\u7cbe\u786e\u3002<\/p>\n

\u5bf9\u4e8e\u6982\u7387\u5bc6\u5ea6\u51fd\u6570\uff0c\uff08\u65e0\u8bba\u4e00\u7ef4\u8fd8\u662f\u591a\u7ef4\uff09\uff0c\u5927\u90e8\u5206\u90fd\u662f\u4e00\u4e2a\u5cf0\u503c\u3002\u6216\u8005\u56e0\u4e3a\u4e2d\u5fc3\u6781\u9650\u5b9a\u7406\uff0c\u7edf\u8ba1\u91cf\u5747\u503c\u7684\u5206\u5e03\u548c\u6b63\u6001\u51fd\u6570\u5f88\u50cf\uff0c\u4e5f\u662f\u4e00\u4e2a\u5cf0\u503c\u3002\u56e0\u6b64\u62c9\u666e\u62c9\u65af\u65b9\u6cd5\u7684\u7528\u5904\u8fd8\u662f\u5f88\u591a\u7684\u3002\u6b64\u5916\uff0c\u62c9\u666e\u62c9\u65af\u65b9\u6cd5\u8ba1\u7b97\u5f88\u5feb\u3002\u6bd4\u5982\u5728\u7ebf\u6027\u6df7\u5408\u6548\u679c\u6a21\u578b\u4e2d\uff0c\u62c9\u666e\u62c9\u65af\u65b9\u6cd5\u662f\u7528\u7684\u6700\u5e7f\u7684\u65b9\u6cd5\uff0c\u4e5f\u53ef\u80fd\u662f\u552f\u4e00\u80fd\u5728\u5b9e\u9645\u4e2d\u4f7f\u7528\u7684\u65b9\u6cd5\uff3b2-4\uff3d\u3002<\/p>\n

\uff3b1\uff3d\u7ef4\u57fa\u767e\u79d1 http:\/\/en.wikipedia.org\/wiki\/Laplace’s_method
\n\uff3b2\uff3dApproximate Inference in Generalized Linear Mixed Models. N. E. Breslow and D. G. Clayton. Journal of the American Statistical Association Vol. 88, No. 421 (Mar., 1993) , pp. 9-25
\n\uff3b3\uff3dVariance component testing in generalised linear models with random effects. XIHONG LIN. Biometrika (1997) 84 (2): 309-326.
\n\uff3b4\uff3dlme4 package Douglas Bates et al. http:\/\/cran.r-project.org\/web\/packages\/lme4\/lme4.pdf (see nAGQ parameter)<\/p>\n","protected":false},"excerpt":{"rendered":"

\u62c9\u666e\u62c9\u65af\u65b9\u6cd5(Laplace’s Method) [mathjax] \u62c9\u666e\u62c9\u65af\u65b9\u6cd5\u53c8\u79f0\u4e3a\u62c9\u666e\u62c9\u65af\u8fd1\u4f3c\uff08Laplace Approximation\uff09\u3002\u5b83\u53ef\u4ee5\u7528\u6765\u8ba1\u7b97\u4e00\u5143\u6216\u591a\u5143\u79ef\u5206\uff3b1\uff3d\u3002 \u4e3e\u4f8b\u6765\u8bf4\uff0c\u5047\u8bbe $latex f(x)$ \u662f\u4e00\u7ef4\u51fd\u6570\uff0c\u6211\u4eec\u8981\u8ba1\u7b97 [latex] \\int_{-\\infty}^{\\infty} f(x) \\mathrm{d} x [\/latex]. \u5982\u679c$latex f(x)$\u5f62\u5f0f\u5f88\u590d\u6742\uff0c\u6211\u4eec\u5f80\u5f80\u627e\u4e0d\u5230\u5b9a\u79ef\u5206\u7684\u516c\u5f0f\uff08Close form\uff09\u3002 \u5982\u679c\u7528\u6570\u503c\u65b9\u6cd5\u6765\u8ba1\u7b97\u79ef\u5206\uff0c\u8ba1\u7b97\u91cf\u53c8\u5f88\u5927\u3002 \u6240\u4ee5\u8981\u60f3\u4e2a\u529e\u6cd5\uff0c\u5f97\u5230\u4e00\u4e2a\u6bd4\u8f83\u7cbe\u786e\u7684\u79ef\u5206\u7ed3\u679c\u3002 \u62c9\u666e\u62c9\u65af\u65b9\u6cd5\u53ef\u4ee5\u9002\u7528\u4e8e\u8fd9\u79cd\u60c5\u51b5\u3002\u6211\u4eec\u5148\u7528\u6cf0\u52d2\u5c55\u5f00\uff08Taylor Expansion\uff09\uff1a $$ f(x) \\approx f(x_0) + f'(x_0)(x-x_0) + \\frac{1}{2}f”(x_0) (x-x_0)^2 $$ \u5982\u679c\u9009\u53d6$latex x_0$\u4f7f\u5f97$latex f'(x_0) = 0$\uff0c\u5219\u53ef\u4ee5\u8fdb\u4e00\u6b65\u7b80\u5316\u4e3a $$ f(x) \\approx f(x_0) + \\frac{1}{2}f”(x_0) (x-x_0)^2 $$ \u518d\u5f15\u5165\u4e00\u4e2a\u5047\u8bbe\uff0c$latex \\int_{-\\infty}^{\\infty} f(x) \\mathrm{d} x $ \u53ef\u4ee5\u88ab\u5199\u6210 $latex \\int_{-\\infty}^{\\infty} e^{f(x)} […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[15],"tags":[136,135,19],"class_list":["post-561","post","type-post","status-publish","format-standard","hentry","category-statistics","tag-approximation","tag-laplace","tag-statistics-2"],"_links":{"self":[{"href":"https:\/\/zhanxw.com\/blog\/wp-json\/wp\/v2\/posts\/561","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/zhanxw.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/zhanxw.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/zhanxw.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/zhanxw.com\/blog\/wp-json\/wp\/v2\/comments?post=561"}],"version-history":[{"count":0,"href":"https:\/\/zhanxw.com\/blog\/wp-json\/wp\/v2\/posts\/561\/revisions"}],"wp:attachment":[{"href":"https:\/\/zhanxw.com\/blog\/wp-json\/wp\/v2\/media?parent=561"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/zhanxw.com\/blog\/wp-json\/wp\/v2\/categories?post=561"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/zhanxw.com\/blog\/wp-json\/wp\/v2\/tags?post=561"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}