\u6211\u4eec\u4ece\u89e3\u51b3\u8fd9\u51e0\u4e2a\u95ee\u9898\u5165\u624b\uff1a\u4f20\u7edf\u7684PCA\u5982\u4f55\u505a\uff1f\u5728\u9ad8\u7ef4\u7a7a\u95f4\u91cc\u7684PCA\u5e94\u8be5\u5982\u4f55\u505a\uff1f\u5982\u4f55\u7528Kernel Trick\u5728\u9ad8\u7ef4\u7a7a\u95f4\u505aPCA\uff1f\u5982\u4f55\u5728\u4e3b\u6210\u5206\u65b9\u5411\u4e0a\u6295\u5f71\uff1f\u5982\u4f55Centering \u9ad8\u7ef4\u7a7a\u95f4\u7684\u6570\u636e\uff1f<\/p>\n
1.1 \u4f20\u7edf\u7684PCA\u5982\u4f55\u505a\uff1f<\/strong><\/p>\n \u8ba9\u6211\u5148\u5b9a\u4e49\u5982\u4e0b\u53d8\u91cf\uff1a [latex] X= [x_1, x_2, \\ldots, x_N] [\/latex] \u662f\u4e00\u4e2a[latex] d \\times N [\/latex]\u77e9\u9635\uff0c\u4ee3\u8868\u8f93\u5165\u7684\u6570\u636e\u6709[latex] N[\/latex] \u4e2a\uff0c\u6bcf\u4e2asample\u7684\u7ef4\u6570\u662f[latex] d [\/latex]\u3002\u6211\u4eec\u505a\u964d\u7ef4\uff0c\u5c31\u662f\u60f3\u7528[latex] k [\/latex]\u7ef4\u7684\u6570\u636e\u6765\u8868\u793a\u539f\u59cb\u7684[latex]d[\/latex]\u7ef4\u6570\u636e\uff08[latex] k \\le d [\/latex]\uff09\u3002 1.2 \u5728\u9ad8\u7ef4\u7a7a\u95f4\u91cc\u7684PCA\u5e94\u8be5\u5982\u4f55\u505a\uff1f<\/strong><\/p>\n \u9ad8\u7ef4\u7a7a\u95f4\u4e2d\uff0c\u6211\u4eec\u5b9a\u4e49\u4e00\u4e2a\u6620\u5c04[latex]\\Phi : X^d \\rightarrow \\mathcal{F}[\/latex]\uff0c\u8fd9\u91cc[latex]\\mathcal{F}[\/latex]\u8868\u793aHilbert\u6cdb\u51fd\u7a7a\u95f4\u3002 1.3 \u5982\u4f55\u7528Kernel Trick\u5728\u9ad8\u7ef4\u7a7a\u95f4\u505aPCA\uff1f<\/strong><\/p>\n <\/strong><\/p>\n <\/strong> \u57281.1\u8282\u4e2d\uff0c\u901a\u8fc7PCA\uff0c\u6211\u4eec\u5f97\u5230\u4e86[latex]U[\/latex]\u77e9\u9635\u3002\u8fd9\u91cc\u5c06\u4ecb\u7ecd\u5982\u4f55\u4ec5\u5229\u7528\u5185\u79ef\u7684\u6982\u5ff5\u6765\u8ba1\u7b97\u4f20\u7edf\u7684PCA\u3002 \u8fdb\u800c\u6211\u4eec\u663e\u793aPCA\u6295\u5f71\u53ef\u4ee5\u7528\u5185\u79ef\u8fd0\u7b97\u8868\u793a\uff0c\u4f8b\u5982\u6211\u4eec\u628a[latex]x_i[\/latex]\u5411\u4efb\u610f\u4e00\u4e2a\u4e3b\u6210\u5206\u5206\u91cf[latex]u_a[\/latex]\u8fdb\u884c\u6295\u5f71\uff0c\u5f97\u5230\u7684\u662f[latex]u_a^T x_i[\/latex]\uff0c\u4e5f\u5c31\u662f[latex]x_i^T u_a[\/latex] \u3002\u4f5c\u8005\u731c\u6d4b\u5199\u6210\u8fd9\u79cd\u5f62\u5f0f\u662f\u4e3a\u4e86\u80fd\u62bd\u51fa[latex]x_i^T x_j = \\mathrm{<} x_i, x_j\\mathrm{>} [\/latex]\u7684\u5185\u79ef\u5f62\u5f0f\u3002<\/p>\n $$\\begin{align} \u5728\u4e0a\u9762\u7684\u5206\u6790\u8fc7\u7a0b\u4e2d\uff0c\u6211\u4eec\u53ea\u4f7f\u7528\u4e86\u5185\u79ef\u3002\u56e0\u6b64\u5f53\u6211\u4eec\u628a[latex]K_{ij} = x_i^T x_j[\/latex]\u63a8\u5e7f\u4e3a[latex]K_{ij} = <\\Phi(x_i), \\Phi(x_j> = \\Phi(x_i)^T \\Phi(x_j)[\/latex]\u65f6\uff0c\u4e0a\u9762\u7684\u5206\u6790\u7ed3\u679c\u5e76\u4e0d\u4f1a\u6539\u53d8\u3002<\/p>\n 1.4 \u5982\u4f55\u5728\u4e3b\u6210\u5206\u65b9\u5411\u4e0a\u6295\u5f71\uff1f<\/strong><\/p>\n \u6295\u5f71\u65f6\uff0c\u53ea\u9700\u8981\u4f7f\u7528[latex]U[\/latex]\u77e9\u9635\uff0c\u5047\u8bbe\u6211\u4eec\u5f97\u5230\u7684\u65b0\u6570\u636e\u4e3a[latex]t[\/latex]\uff0c\u90a3\u4e48[latex]t[\/latex]\u5728[latex]u_a[\/latex]\u65b9\u5411\u7684\u6295\u5f71\u662f\uff1a 1.5 \u5982\u4f55Centering \u9ad8\u7ef4\u7a7a\u95f4\u7684\u6570\u636e\uff1f<\/strong><\/p>\n \u5728\u6211\u4eec\u7684\u5206\u6790\u4e2d\uff0c\u534f\u65b9\u5dee\u77e9\u9635\u7684\u5b9a\u4e49\u9700\u8981centered data\u3002\u5728\u9ad8\u7ef4\u7a7a\u95f4\u4e2d\uff0c\u663e\u5f0f\u7684\u5c06[latex]\\Phi(x_i)[\/latex]\u5c45\u4e2d\u5e76\u4e0d\u7b80\u5355, 2. \u6f14\u793a (R code)<\/strong><\/p>\n <\/strong> KPCA\u56fe\u7247\uff1a R \u6e90\u4ee3\u7801\uff08Source Code\uff09\uff1a\u94fe\u63a5\u5230\u5b8c\u6574\u7684\u4ee3\u7801\u00a0KernelPCA<\/a><\/p>\n Kernel PCA\u90e8\u5206\u4ee3\u7801\uff1a<\/p>\n 3. \u4e3b\u8981\u53c2\u8003\u8d44\u6599<\/strong><\/p>\n \u30101\u3011A Tutorial on Principal Component Analysis ,Jonathon Shlens\u0003, Shlens03<\/a><\/p>\n \u30102\u3011Wikipedia\uff1a http:\/\/en.wikipedia.org\/wiki\/Kernel_principal_component_analysis<\/p>\n \u30103\u3011 Original KPCA Paper\uff1aKernel principal component analysis\uff0cBernhard Sch\u00f6lkopf, Alexander Smola and Klaus-Robert M\u00fcller\u00a0http:\/\/www.springerlink.com\/content\/w0t1756772h41872\/fulltext.pdf<\/a><\/p>\n
\n\u5f53\u6211\u4eec\u4f7f\u7528centered\u7684\u6570\u636e\uff08\u5373[latex]\\sum_i x_i = 0[\/latex]\uff09\u65f6\uff0c\u53ef\u5b9a\u4e49\u534f\u65b9\u5dee\u77e9\u9635[latex]C[\/latex]\u4e3a\uff1a
\n[mathjax]
\n$$C=\\frac{1}{N} x_i x_i^T = \\frac{1}{N} X X^T$$
\n\u505a\u7279\u5f81\u503c\u5206\u89e3\uff0c\u6211\u4eec\u53ef\u4ee5\u5f97\u5230\uff1a
\n$$CU = U \\Lambda \\Rightarrow C = U \\Lambda U^T = \\sum_a \\lambda_a u_a u_a^T$$
\n\u6ce8\u610f\u8fd9\u91cc\u7684[latex]C, U , \\Lambda[\/latex]\u7684\u7ef4\u6570\u90fd\u662f[latex]d \\times d[\/latex], \u4e14[latex] U=[u_1, u_2, \\ldots, u_d] [\/latex], [latex]\\Lambda = diag(\\lambda_1, \\lambda_2, \\ldots, \\lambda_d) [\/latex]\u3002
\n\u5f53\u6211\u4eec\u505a\u964d\u7ef4\u65f6\uff0c\u53ef\u4ee5\u5229\u7528\u524d[latex]k[\/latex]\u4e2a\u7279\u5f81\u5411\u91cf[latex]U_k = [u_1, u_2, \\ldots, u_k] [\/latex]\u3002\u5219\u5c06\u4e00\u4e2a[latex]d[\/latex]\u7ef4\u7684[latex]x_i[\/latex]\u5411[latex]k[\/latex]\u7ef4\u7684\u4e3b\u6210\u5206\u7684\u65b9\u5411\u6295\u5f71\u540e\u7684[latex]y_i = U_k^T x_i[\/latex] \uff08\u8fd9\u91cc\u7684\u6bcf\u4e00\u4e2a[latex]u_i[\/latex]\u90fd\u662f[latex]d[\/latex]\u7ef4\u7684\uff0c\u4ee3\u8868\u662f\u4e00\u4e2a\u6295\u5f71\u65b9\u5411\uff0c\u4e14[latex]u_i^T u_i = 1[\/latex]\uff0c\u8868\u793a\u8fd9\u662f\u4e00\u4e2a\u65cb\u8f6c\u53d8\u91cf\uff09<\/p>\n
\n\u73b0\u5728\u6211\u4eec\u7684\u8f93\u5165\u6570\u636e\u662f[latex]\\Phi(x_i), i = 1, 2, …n[\/latex]\uff0c \u4ed6\u4eec\u7684\u7ef4\u6570\u53ef\u4ee5\u8bf4\u662f\u65e0\u7a77\u7ef4\u7684\uff08\u6cdb\u51fd\u7a7a\u95f4\uff09\u3002
\n\u5728\u8fd9\u4e2a\u65b0\u7684\u7a7a\u95f4\u4e2d\uff0c\u5047\u8bbe\u534f\u65b9\u5dee\u77e9\u9635\u540c\u6837\u662fcentered\uff0c\u6211\u4eec\u7684\u534f\u65b9\u5dee\u77e9\u9635\u4e3a\uff1a
\n$$\\bar{C}=\\frac{1}{N} \\Phi(x_i) \\Phi(x_i)^T = \\frac{1}{N} \\Phi(X) \\Phi(X)^T$$
\n\u8fd9\u91cc\u6709\u4e00\u4e2a\u9677\u9631\uff0c\u6211\u8df3\u8fdb\u53bb\u8fc7\uff1a
\n\u5728\u5bf9Kernel trick\u4e00\u77e5\u534a\u89e3\u7684\u65f6\u5019\uff0c\u6211\u4eec\u5e38\u5e38\u4ece\u5f62\u5f0f\u4e0a\u8ba4\u4e3a[latex]\\bar{C} [\/latex]\u53ef\u4ee5\u7528[latex]K_{i,j} = K(x_i, x_j)[\/latex]\u6765\u4ee3\u66ff\uff0c
\n\u56e0\u6b64\u5bf9[latex]K=(K_{ij})[\/latex]\u505a\u7279\u5f81\u503c\u5206\u89e3\uff0c\u7136\u540e\u5f97\u5230[latex]K=U \\Lambda U^T[\/latex]\uff0c\u5e76\u4e14\u5bf9\u539f\u6709\u6570\u636e\u964d\u7ef4\u7684\u65f6\u5019\uff0c\u5b9a\u4e49[latex]Y_i=U_k^T X_i[\/latex]\u3002
\n\u4f46\u8fd9\u4e2a\u9519\u8bef\u7684\u65b9\u6cd5\u6709\u4e24\u4e2a\u95ee\u9898\uff1a\u4e00\u662f\u6211\u4eec\u4e0d\u77e5\u9053\u77e9\u9635[latex]\\bar{C}[\/latex]\u7684\u7ef4\u6570\uff1b\u4e8c\u662f[latex]U_k^T X_i[\/latex]\u4ece\u5f62\u5f0f\u4e0a\u770b\u4e0d\u51fa\u662f\u4ece\u9ad8\u7ef4\u7a7a\u95f4\u7684[latex]\\Phi(X_i)[\/latex]\u6295\u5f71\uff0c\u5e76\u4e14\u5f53\u6709\u65b0\u7684\u6570\u636e\u65f6\uff0c\u6211\u4eec\u65e0\u6cd5\u4ece\u7406\u8bba\u4e0a\u7406\u89e3[latex] U_k^T X_{\\mathrm{new}} [\/latex]\u662f\u4ece\u9ad8\u7ef4\u7a7a\u95f4\u7684\u6295\u5f71\u3002
\n\u5982\u679c\u5e94\u7528\u8fd9\u79cd\u9519\u8bef\u7684\u65b9\u6cd5\uff0c\u6211\u4eec\u6709\u53ef\u80fd\u5f97\u5230\u770b\u8d77\u6765\u5dee\u4e0d\u591a\u6b63\u786e\u7684\u7ed3\u679c\uff0c\u4f46\u672c\u8d28\u4e0a\u8fd9\u662f\u9519\u8bef\u7684\u3002
\n\u6b63\u786e\u7684\u65b9\u6cd5\u662f\u901a\u8fc7Kernel trick\u5c06PCA\u6295\u5f71\u7684\u8fc7\u7a0b\u901a\u8fc7\u5185\u79ef\u7684\u5f62\u5f0f\u8868\u8fbe\u51fa\u6765\uff0c\u8be6\u7ec6\u89c11.3<\/p>\n
\n\u9996\u5148\u6211\u4eec\u8bc1\u660e[latex]U[\/latex]\u53ef\u4ee5\u7531[latex]x_1, x_2, \\ldots, x_N[\/latex]\u5c55\u5f00\uff08span)\uff1a
\n$$ C u_a = \\lambda_a u_a $$
\n$$ \\begin{align}
\nu_a &= \\frac{1}{\\lambda_a} C u \\\\
\n&= \\frac{1}{\\lambda_a} (\\sum_i x_i x_i^T )u \\\\
\n&= \\frac{1}{\\lambda_a} \\sum_i x_i (x_i^T u) \\\\
\n&= \\frac{1}{\\lambda_a} \\sum_i (x_i^T u) x_i \\\\
\n&= \\sum_i \\frac{x_i^T u}{\\lambda_a} x_i \\\\
\n&= \\sum_i \\alpha_i^a x_i
\n\\end{align}$$
\n\u8fd9\u91cc\u5b9a\u4e49[latex]\\alpha_i^a=\\frac{x_i^T u}{\\lambda_a} [\/latex]\u3002
\n\u56e0\u4e3a[latex]x_i^T u[\/latex] \u662f\u4e00\u4e2a\u6807\u91cf\uff08scala\uff09\uff0c\u6240\u4ee5[latex]\\alpha_i^a[\/latex]\u4e5f\u662f\u4e00\u4e2a\u6807\u91cf\uff0c\u56e0\u6b64[latex]u_i[\/latex] \u662f\u53ef\u4ee5\u7531[latex]x_i[\/latex]\u5f20\u6210\u3002<\/p>\n
\nx_i^T C u_a &= \\lambda_a x_i^T u_a \\\\
\nx_i^T \\frac{1}{N} \\sum_j x_j x_j^T \\sum_k \\alpha_k^a x_k &= \\lambda_a x_i^T \\sum_k \\alpha_k^a x_k \\\\
\n\\sum_j \\alpha_k^a \\sum_k (x_i^T x_j) (x_j^T x_k) &= N \\lambda_a \\sum_k \\alpha_k^a (x_i^T x_k) \\\\
\n\\end{align}$$
\n\u5f53\u6211\u4eec\u5b9a\u4e49[latex] K_{ij} = x_i^T x_j [\/latex]\u65f6\uff0c\u4e0a\u5f0f\u53ef\u4ee5\u5199\u4e3a[latex] K^2 \\alpha = N \\lambda_a K \\alpha^a[\/latex]
\n\uff08\u8fd9\u91cc[latex]\\alpha^a[\/latex]\u5b9a\u4e49\u4e3a[latex][\\alpha_1^a, \\alpha_2^a, \\ldots, \\alpha_N^a]^T[\/latex].\uff09
\n\u8fdb\u4e00\u6b65\uff0c\u6211\u4eec\u5f97\u5230\u89e3\u4e3a\uff1a
\n$$K \\alpha = \\tilde{\\lambda}_a \\alpha \\quad \\mathrm{with} \\quad \\tilde{\\lambda}_a = N \\lambda_a$$
\n[latex]K[\/latex]\u77e9\u9635\u5305\u542b\u7279\u5f81\u503c[latex]\\tilde{\\lambda}[\/latex]\u548c[latex]\\alpha^a[\/latex]\uff0c\u6211\u4eec\u53ef\u4ee5\u901a\u8fc7[latex]\\alpha[\/latex]\u53ef\u4ee5\u8ba1\u7b97\u5f97\u5230[latex]u_a[\/latex]\uff0c
\n\u6ce8\u610f\u7279\u5f81\u503c\u5206\u89e3\u65f6Eigendecomposition\uff0c[latex]\\alpha^a[\/latex]\u53ea\u4ee3\u8868\u4e00\u4e2a\u65b9\u5411\uff0c\u5b83\u7684\u957f\u5ea6\u4e00\u822c\u4e3a1\uff0c\u4f46\u5728\u6b64\u5904\u4e0d\u4e3a1\u3002
\n\u8fd9\u91cc\u8ba1\u7b97\u51fa[latex]\\alpha_a[\/latex]\u7684\u957f\u5ea6\uff08\u4e0b\u9762\u5c06\u8981\u7528\u5230\uff09\uff1a
\n\u56e0\u4e3a[latex]u_a[\/latex]\u7684\u957f\u5ea6\u662f1\uff0c\u6211\u4eec\u6709\uff1a
\n$$\\begin{align}
\n1 &= u_a^T u_a \\\\
\n&= ( \\sum_i \\alpha_i^a x_i) ^T (\\sum_j \\alpha_j^a x_j ) \\\\
\n&= \\sum_i \\sum_j \\alpha_i^a \\alpha_j^a x_i^T x_j^T \\\\
\n&=(\\alpha^a)^T K \\alpha_a \\\\
\n&=(\\alpha^a)^T ( N \\lambda_a \\alpha_a) \\\\
\n&=N \\lambda_a ({\\alpha^a}^T \\alpha_a )\\\\
\n&\\Rightarrow \\quad \\lVert \\alpha^a \\rVert = 1\/\\sqrt{N \\lambda_a} = 1\/\\sqrt{\\tilde{\\lambda}_a}
\n\\end{align}$$<\/p>\n
\n$$u_a^T t = \\sum_i \\alpha_i^a x_i^T t = \\sum_i \\alpha_i^a ( x_i^T t)$$
\n\u5bf9\u4e8e\u9ad8\u7ef4\u7a7a\u95f4\u7684\u6570\u636e[latex]\\Phi(x_i),\\Phi(t)[\/latex]\uff0c\u6211\u4eec\u53ef\u4ee5\u7528Kernel trick\uff0c\u7528[latex]K(x_i, t)[\/latex]\u6765\u5e26\u5165\u4e0a\u5f0f\uff1a
\n$$u_a^T t = \\sum_i \\alpha_i^a K(x_i, t)$$<\/p>\n
\n\u56e0\u4e3a\u6211\u4eec\u5e76\u4e0d\u77e5\u9053[latex]\\Phi[\/latex]\u7684\u663e\u793a\u8868\u8fbe\u3002\u4f46\u4ece\u4e0a\u9762\u4e24\u8282\u53ef\u4ee5\u770b\u51fa\uff0c\u6240\u6709\u7684\u8ba1\u7b97\u53ea\u548c[latex]K[\/latex]\u77e9\u9635\u6709\u5173\u3002\u5177\u4f53\u8ba1\u7b97\u5982\u4e0b\uff1a
\n\u4ee4[latex]\\Phi_i = \\Phi(x_i)[\/latex]\uff0c\u5c45\u4e2d[latex]\\Phi_i^C = \\Phi_i – \\frac{1}{N}\\sum_k \\Phi_k[\/latex]
\n$$\\begin{align}
\nK_{ij}^C
\n&= <\\Phi_i^C \\Phi_j^C> \\\\
\n&= (\\Phi_i – \\frac{1}{N}\\sum_k \\Phi_k)^T (\\Phi_j – \\frac{1}{N}\\sum_l \\Phi_l \uff09 \\\\
\n&=\\Phi_i^T\\Phi_j – \\frac{1}{N}\\sum_l \\Phi_i^T \\Phi_l – \\frac{1}{N}\\sum_k \\Phi_k^T \\Phi_j + \\frac{1}{N^2}\\sum_k \\sum_l \\Phi_k^T \\Phi_l \\\\
\n&=K_{ij} – \\frac{1}{N}\\sum_l K_{il} – \\frac{1}{N}\\sum_k K_{kj} + \\frac{1}{N^2}\\sum_k \\sum_l K_{kl}
\n\\end{align}$$
\n\u4e0d\u96be\u770b\u51fa\uff0c
\n$$K^C = K – 1_N K – K 1_N + 1_N K 1_N$$
\n\u5176\u4e2d[latex]1_N[\/latex] \u4e3a[latex]N \\times N [\/latex]\u7684\u77e9\u9635\uff0c\u5176\u4e2d\u6bcf\u4e00\u4e2a\u5143\u7d20\u90fd\u662f[latex]1\/N[\/latex]
\n\u5bf9\u4e8e\u65b0\u7684\u6570\u636e\uff0c\u6211\u4eec\u540c\u6837\u53ef\u4ee5
\n$$\\begin{align}
\nK(x_i, t)^C
\n&= <\\Phi_i^C \\Phi_t^C> \\\\
\n&= (\\Phi_i – \\frac{1}{N}\\sum_k \\Phi_k)^T (\\Phi_t – \\frac{1}{N}\\sum_l \\Phi_l \uff09 \\\\
\n&=\\Phi_i^T\\Phi_t – \\frac{1}{N}\\sum_l \\Phi_i^T \\Phi_l – \\frac{1}{N}\\sum_k \\Phi_k^T \\Phi_t + \\frac{1}{N^2}\\sum_k \\sum_l \\Phi_k^T \\Phi_l \\\\
\n&=K(x_i,t) – \\frac{1}{N}\\sum_l K_{il} – \\frac{1}{N}\\sum_k K(x_k,t) + \\frac{1}{N^2}\\sum_k \\sum_l K_{kl}
\n\\end{align}$$<\/p>\n
\n\u9996\u5148\u6211\u4eec\u5e94\u8be5\u6ce8\u610f\u8f93\u5165\u6570\u636e\u7684\u683c\u5f0f\uff0c\u4e00\u822c\u5728\u7edf\u8ba1\u4e2d\uff0c\u6211\u4eec\u8981\u6c42[latex]X[\/latex]\u77e9\u9635\u662f[latex]N \\times d[\/latex]\u7684\uff0c\u4f46\u5728\u6211\u4eec\u7684\u63a8\u5bfc\u4e2d\uff0c[latex]X[\/latex]\u77e9\u9635\u662f[latex]d \\times N[\/latex]\u3002
\n\u8fd9\u4e0e\u7edf\u8ba1\u4e0a\u7684\u6982\u5ff5\u5e76\u4e0d\u77db\u76fe\uff1a\u5728\u524d\u9762\u7684\u5b9a\u4e49\u4e0b\u534f\u65b9\u5dee\u77e9\u9635\u4e3a[latex]X^T X[\/latex]\uff0c\u800c\u5728\u540e\u9762\u7684\u5b9a\u4e49\u4e2d\u662f[latex]X X^T[\/latex]\u3002\u53e6\u5916\u8fd9\u91cc\u7684\u534f\u65b9\u5dee\u77e9\u9635\u662f\u6837\u672c\uff08Sample\uff09\u7684\u534f\u65b9\u5dee\u77e9\u9635\uff0c\u6211\u4eec\u7684\u8ba4\u4e3a\u5927\u5199\u7684X\u4ee3\u8868\u77e9\u9635\uff0c\u800c\u4e0d\u662f\u4ee3\u8868\u4e00\u4e2a\u968f\u673a\u53d8\u91cf\u3002
\n\u53e6\u5916\uff0c\u5728\u4e0b\u9762\u7684\u7ed3\u679c\u4e2d\uff0cGaussian \u6838\u51fd\u6570\uff08kernel function\uff09\u7684\u6807\u51c6\u5dee\uff08sd\uff09\u4e3a2\u3002\u5728\u5176\u4ed6\u53d6\u503c\u6761\u4ef6\u4e0b\uff0c\u6240\u5f97\u5230\u7684\u56fe\u50cf\u662f\u4e0d\u540c\u7684\u3002<\/p>\n
\n![\"\"](\"http:\/\/www.zhanxw.com\/blog\/wp-content\/uploads\/2011\/02\/20110206_1.png\" "\\"20110206_1\\"")
<\/a><\/p>\n\r\n# Kernel PCA\r\n# Polynomial Kernel\r\n# k(x,y) = t(x) %*% y + 1\r\nk1 = function (x,y) { (x[1] * y[1] + x[2] * y[2] + 1)^2 }\r\nK = matrix(0, ncol = N_total, nrow = N_total)\r\nfor (i in 1:N_total) {\r\n for (j in 1:N_total) {\r\n K[i,j] = k1(X[i,], X[j,])\r\n}}\r\nones = 1\/N_total* matrix(1, N_total, N_total)\r\nK_norm = K - ones %*% K - K %*% ones + ones %*% K %*% ones\r\nres = eigen(K_norm)\r\n\r\nV = res$vectors\r\nD = diag(res$values)\r\n\r\nrank = 0\r\nfor (i in 1:N_total) {\r\n\tif (D[i,i] < 1e-6) { break }\r\n V[,i] = V[,i] \/ sqrt (D[i,i])\r\n\trank = rank + 1\r\n}\r\nY = K_norm %*% V[,1:rank]\r\nplot(Y[,1], Y[,2], col = rainbow(3)[label], main = "Kernel PCA (Poly)"\r\n, xlab="First component", ylab="Second component")\r\n<\/pre>\n
\n<\/strong><\/p>\n