{"id":31578,"date":"2022-01-05T21:10:23","date_gmt":"2022-01-05T13:10:23","guid":{"rendered":"https:\/\/swarma.org\/?p=31578"},"modified":"2022-01-05T21:10:23","modified_gmt":"2022-01-05T13:10:23","slug":"%e4%bb%80%e4%b9%88%e6%98%af%e9%a6%99%e5%86%9c%e4%bf%a1%e6%ba%90%e7%bc%96%e7%a0%81%e5%ae%9a%e7%90%86-%e9%9b%86%e6%99%ba%e7%99%be%e7%a7%91","status":"publish","type":"post","link":"https:\/\/swarma.org\/?p=31578","title":{"rendered":"\u4ec0\u4e48\u662f\u9999\u519c\u4fe1\u6e90\u7f16\u7801\u5b9a\u7406 | \u96c6\u667a\u767e\u79d1"},"content":{"rendered":"
\n

<\/p>\n


<\/p>\n

\u201c\u96c6\u667a\u767e\u79d1\u7cbe\u9009\u201d\u662f\u4e00\u4e2a\u957f\u671f\u4e13\u680f\uff0c\u6301\u7eed\u4e3a\u5927\u5bb6\u63a8\u9001\u590d\u6742\u6027\u79d1\u5b66\u76f8\u5173\u7684\u57fa\u672c\u6982\u5ff5\u548c\u8d44\u6e90\u4fe1\u606f\u3002\u4f5c\u4e3a\u96c6\u667a\u4ff1\u4e50\u90e8\u7684\u5f00\u6e90\u79d1\u5b66\u9879\u76ee\uff0c\u96c6\u667a\u767e\u79d1\u5e0c\u671b\u6253\u9020\u590d\u6742\u6027\u79d1\u5b66\u9886\u57df\u6700\u5168\u9762\u7684\u767e\u79d1\u5168\u4e66\uff0c\u6b22\u8fce\u5bf9\u590d\u6742\u6027\u79d1\u5b66\u611f\u5174\u8da3\u3001\u70ed\u7231\u77e5\u8bc6\u6574\u7406\u548c\u5206\u4eab\u7684\u670b\u53cb\u52a0\u5165\uff01
<\/span><\/section>\n

<\/section>\n
\u672c\u6587\u662f\u5bf9\u96c6\u667a\u767e\u79d1\u4e2d\u201c\u9999\u519c\u4fe1\u6e90\u7f16\u7801\u5b9a\u7406<\/span>\u201d\u8bcd\u6761\u7684\u6458\u5f55\uff0c\u53c2\u8003\u8d44\u6599\u53ca\u76f8\u5173\u8bcd\u6761\u8bf7\u53c2\u9605\u767e\u79d1\u8bcd\u6761\u539f\u6587\u3002<\/span><\/section>\n

<\/section>\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

\u672c\u8bcd\u6761\u7531\u96c6\u667a\u4ff1\u4e50\u90e8\u4f17\u5305\u751f\u4ea7\uff0c\u96be\u514d\u5b58\u5728\u7eb0\u6f0f\u548c\u95ee\u9898\uff0c\u6b22\u8fce\u5927\u5bb6\u7559\u8a00\u53cd\u9988\u6216\u8005\u524d\u5f80\u5bf9\u5e94\u7684\u767e\u79d1\u8bcd\u6761\u9875\u9762\u8fdb\u884c\u4fee\u6539\uff0c\u4e00\u7ecf\u4fee\u6539\uff0c\u53ef\u4ee5\u83b7\u5f97\u5bf9\u5e94\u7684\u79ef\u5206\u5956\u52b1\u5662\uff01<\/em><\/span><\/p>\n


<\/p>\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

\u76ee\u5f55<\/strong><\/p>\n


<\/p>\n

\n
\u4e00\u3001\u8bf4\u660e<\/span><\/strong><\/section>\n
\u4e8c\u3001\u8bc1\u660e\uff1a\u6e90\u4ee3\u7801\u5b9a\u7406<\/strong><\/span><\/section>\n
\u4e09\u3001\u8bc1\u660e\uff1a\u7b26\u53f7\u4ee3\u7801\u7684\u6e90\u7f16\u7801\u5b9a\u7406<\/strong><\/strong><\/span><\/section>\n
\u56db\u3001\u975e\u5e73\u7a33\u72ec\u7acb\u6e90\u7684\u6269\u5c55<\/strong><\/span><\/section>\n
\u4e94\u3001<\/strong>\u7f16\u8005\u63a8\u8350<\/strong><\/strong><\/strong><\/span><\/section>\n
\u516d\u3001\u767e\u79d1\u9879\u76ee\u5fd7\u613f\u8005\u62db\u52df<\/strong><\/strong><\/strong><\/span><\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n
<\/section>\n

\u5728\u4fe1\u606f\u8bba\u4e2d\uff0c<\/span>\u9999\u519c\u4fe1\u6e90\u7f16\u7801\u5b9a\u7406 Shannon’s source coding theorem\uff08\u6216\u65e0\u566a\u58f0\u7f16\u7801\u5b9a\u7406\uff09\u5efa\u7acb\u4e86\u53ef\u80fd\u7684\u6570\u636e\u538b\u7f29 data compression \u7684\u6781\u9650\uff0c\u4ee5\u53ca\u9999\u519c\u71b5 Shannon entropy\u7684\u64cd\u4f5c\u610f\u4e49\u3002<\/span>
<\/p>\n


\u4ee5\u514b\u52b3\u5fb7\u00b7\u9999\u519c Claude Shann<\/span>on\u547d\u540d\u7684\u4fe1\u6e90\u7f16\u7801\u5b9a\u7406\u8868\u660e\uff0c\uff08\u5728\u6781\u9650\u60c5\u51b5\u4e0b\uff0c\u5f53\u72ec\u7acb\u7684\u5747\u5300\u5206\u5e03\u7684\u968f\u673a\u53d8\u91cf\uff08i.i.d.\uff09\u6570\u636e\u6d41\u7684\u957f\u5ea6\u8d8b\u4e8e\u65e0\u7a77\u5927\u65f6\uff09\u4e0d\u53ef\u80fd\u538b\u7f29\u6570\u636e\u4f7f\u7f16\u7801\u7387\uff08\u6bcf\u4e2a\u7b26\u53f7\u7684\u5e73\u5747\u6bd4\u7279\u6570\uff09\u5c0f\u4e8e\u4fe1\u6e90\u7684\u9999\u519c\u71b5\uff0c\u800c\u4e8b\u5b9e\u4e0a\u53c8\u4e0d\u80fd\u786e\u5b9a\u4fe1\u606f\u4f1a\u4e22\u5931\u3002\u7136\u800c\uff0c\u53ef\u4ee5\u4efb\u610f\u5730\u4f7f\u7801\u7387\u63a5\u8fd1\u9999\u519c\u71b5\uff0c\u635f\u5931\u7684\u6982\u7387\u53ef\u4ee5\u5ffd\u7565\u4e0d\u8ba1\u3002<\/span><\/p>\n


\u7b26\u53f7\u7801\u7684\u4fe1\u6e90\u7f16\u7801\u5b9a\u7406\u6839\u636e\u8f93\u5165\u5355\u8bcd(\u88ab\u89c6\u4e3a\u4e00\u4e2a\u968f\u673a\u53d8\u91cf)\u71b5\u548c\u76ee\u6807\u5b57\u6bcd\u8868\u5927\u5c0f\u5c06\u6700\u5c0f\u53ef\u80fd\u671f\u671b\u7801\u5b57\u957f\u5ea6\u4f5c\u4e3a\u4e00\u4e2a\u51fd\u6570\uff0c\u5e76\u8bbe\u7f6e\u4e86\u4e00\u4e2a\u4e0a\u4e0b\u754c\u3002<\/span><\/p>\n


<\/p>\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n


<\/p>\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

<\/section>\n

<\/section>\n
\n
<\/svg><\/section>\n<\/section>\n<\/section>\n
\n

\u8bf4\u660e<\/strong><\/strong><\/p>\n<\/section>\n

\n

<\/section>\n

<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n

<\/span>
<\/h2>\n

\u4fe1\u6e90\u7f16\u7801\u662f\u4ece\u4fe1\u6e90\u7b26\u53f7\u5e8f\u5217\u5230\u5b57\u6bcd\u7b26\u53f7\u5e8f\u5217(\u901a\u5e38\u662f\u6bd4\u7279)\u7684\u6620\u5c04\uff0c\u4f7f\u4fe1\u6e90\u7b26\u53f7\u80fd\u591f\u51c6\u786e\u5730\u4ece\u4e8c\u8fdb\u5236\u6bd4\u7279\u4f4d(\u65e0\u635f\u6e90\u7f16\u7801)\u6062\u590d\u6216\u5728\u67d0\u79cd\u5931\u771f(\u6709\u635f\u6e90\u7f16\u7801)\u8303\u56f4\u5185\u6062\u590d\u3002\u8fd9\u5c31\u662f\u6570\u636e\u538b\u7f29\u7684\u6982\u5ff5\u3002<\/span><\/p>\n


<\/p>\n

\u4fe1\u6e90\u7f16\u7801\u5b9a\u7406<\/span><\/strong><\/span><\/h3>\n

\u5728\u4fe1\u606f\u8bba\u4e2d\uff0c\u4fe1\u6e90\u7f16\u7801\u5b9a\u7406(Shannon\uff0c1948)):<\/span><\/p>\n

\n

N i.i.d.\u6bcf\u4e2a\u968f\u673a\u53d8\u91cf\u90fd\u6709\u71b5 H(X)\u53ef\u4ee5\u538b\u7f29\u6210\u8d85\u8fc7N\u2009H(X)\u3002\u968f\u7740N \u2192 \u221e\uff0c\u4f4d\u7684\u4fe1\u606f\u4e22\u5931\u98ce\u9669\u53ef\u4ee5\u5ffd\u7565\u4e0d\u8ba1\uff1b\u4f46\u53cd\u8fc7\u6765\u8bf4\uff0c\u5982\u679c\u5b83\u4eec\u88ab\u538b\u7f29\u6210\u5c11\u4e8eN\u2009H(X) \u4f4d\uff0c\u5219\u4fe1\u606f\u80af\u5b9a\u5c06\u4e22\u5931\u3002<\/p>\n<\/blockquote>\n


<\/p>\n

\u7b26\u53f7\u7801\u7684\u6e90\u7801\u5b9a\u7406<\/span><\/strong><\/span><\/h3>\n

\u4ee4\u03a31<\/sub>,\u03a3<\/span>2<\/span><\/sub>\u8868\u793a\u4e24\u4e2a\u6709\u9650\u5b57\u6bcd\uff0c\u5e76\u8ba9\u03a3<\/span>1<\/span><\/sub>*\u548c\u03a3<\/span>2<\/span><\/sub>*\u5206\u522b\u8868\u793a\u8fd9\u4e9b\u5b57\u6bcd\u4e2d\u6240\u6709\u6709\u9650\u8bcd\u7684\u96c6\u5408\u3002<\/span><\/p>\n

\u5047\u8bbeX\u662f\u4e00\u4e2a\u968f\u673a\u53d8\u91cf\uff0c\u53d6\u503c\u5728\u03a3<\/span>1<\/sub>\uff0c\u8ba9\u2009f\u2009 \u662f\u4e00\u4e2a<\/span>\u552f\u4e00\u53ef\u89e3\u7801\u4ee3\u7801\uff0c\u4ece\u03a3<\/span>1<\/sub>*\u5230\u03a3<\/span>2<\/span><\/sub>*\uff0c\u5176\u4e2d|\u03a3<\/span>2<\/span><\/sub>|=a\u3002\u8ba9S\u8868\u793a\u7531\u7801\u5b57\u2009f\u2009(X)\u7684\u957f\u5ea6\u7ed9\u51fa\u7684\u968f\u673a\u53d8\u91cf\u3002<\/span><\/p>\n


<\/p>\n

\u5982\u679c\u2009f\u2009\u662f\u6700\u4f18\u7684\uff0c\u56e0\u4e3a\u5b83\u5177\u6709X\u7684\u6700\u5c0f\u9884\u671f\u5b57\u957f\uff0c\u5219(Shannon 1948): X<\/span><\/p>\n


<\/p>\n

<\/p>\n


<\/p>\n

\u5176\u4e2d\u0395\u8868\u793a\u671f\u671b\u503c\u8fd0\u7b97\u7b26\u3002<\/span><\/p>\n

<\/h2>\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n


<\/p>\n

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

<\/section>\n

<\/section>\n
\n
<\/svg><\/section>\n<\/section>\n<\/section>\n
\n

\u8bc1\u660e\uff1a\u6e90\u4ee3\u7801\u5b9a\u7406<\/strong><\/strong><\/p>\n<\/section>\n

\n

<\/section>\n

<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n

<\/span>
<\/h2>\n

\u7ed9\u5b9aX\u662f\u4e00\u4e2aiid\u6e90\uff0c\u5b83\u7684\u65f6\u95f4\u5e8f\u5217X1<\/sub>, …,<\/span>X<\/span>n<\/sub>\u662f<\/span> iid \uff0c\u5728\u79bb\u6563\u503c\u60c5\u51b5\u4e0b\u71b5H(X)\u548c\u5728\u8fde\u7eed\u503c\u60c5\u51b5\u4e0b\u5fae\u5206\u71b5\u3002<\/span>\u6e90\u7f16\u7801\u5b9a\u7406\u6307\u51fa\uff0c\u5bf9\u4e8e\u4efb\u4f55\u03b5 > 0\uff0c\u5373\u5bf9\u4e8e\u5927\u4e8e\u6e90\u71b5\u7684\u4efb\u4f55\u901f\u7387H(X) + \u03b5\uff0c\u6709\u8db3\u591f\u5927\u7684n\u548c\u4e00\u4e2a\u7f16\u7801\u5668\uff0c\u8be5\u7f16\u7801\u5668\u91c7\u7528\u6e90\u7684niid \u91cd\u590d\uff0cX<\/span>1<\/sub>:n\uff0c\u5e76\u5c06\u5176\u6620\u5c04\u5230n(H(X) + \u03b5)\u4e2a\u4e8c\u8fdb\u5236\u4f4d\uff0c\u4f7f\u5f97\u6e90\u7b26\u53f7X<\/span>1<\/sub>:n\u53ef\u4ee5\u81f3\u5c111 \u2212 \u03b5.\u7684\u6982\u7387\u4ece\u4e8c\u8fdb\u5236\u4f4d\u4e2d\u6062\u590d\u3002<\/span><\/p>\n


\u53ef\u5b9e\u73b0\u6027\u8bc1\u660e\u3002\u4fee\u6b63\u4e00\u4e9b\u03b5 > 0\uff0c\u8ba9<\/span><\/p>\n


<\/p>\n

<\/p>\n


<\/p>\n

\u5178\u578b\u96c6\uff0cAn<\/sub>\u03b5<\/sup> \u5b9a\u4e49\u5982\u4e0b\uff1a<\/span><\/p>\n


<\/p>\n

<\/p>\n


<\/p>\n

\u8be5\u6e10\u8fd1\u5747\u5206\u5272\u6027\uff08AEP\uff09\u793a\u51fa\u4e86\u5bf9\u4e8e\u8db3\u591f\u5927n\uff0c\u4f7f\u5f97\u7531\u6e90\u5728\u4e8e\u5178\u578b\u7684\u96c6\u5408\u751f\u6210\u7684\u5e8f\u5217\u7684\u6982\u7387\uff0cAn<\/sub>\u03b5<\/sup> \uff0c\u5982\u5b9a\u4e49\u63a5\u8fd1\u4e00\u3002\u7279\u522b\u662f\uff0c\u5bf9\u4e8e\u8db3\u591f\u5927\u7684n\uff0cP(X<\/span>1<\/sub>,<\/span>X<\/span>2<\/sub><\/sub>,,,,,X<\/span>n<\/sub>)\u2208A<\/span>n<\/sub>\u03b5<\/sup> <\/span>)\u53ef\u4ee5\u4efb\u610f\u63a5\u8fd1 1\uff0c\u5177\u4f53\u6765\u8bf4\uff0c\u5927\u4e8e1 \u2212 \u03b5\uff08\u6709\u5173\u8bc1\u660e\uff0c\u8bf7\u53c2\u9605 AEP\uff09\u3002
<\/span><\/p>\n


<\/p>\n

\u5178\u578b\u96c6\u5408\u7684\u5b9a\u4e49\u610f\u5473\u7740\u90a3\u4e9b\u4f4d\u4e8e\u5178\u578b\u96c6\u5408\u4e2d\u7684\u5e8f\u5217\u6ee1\u8db3\uff1a<\/span><\/p>\n


<\/p>\n

<\/p>\n


<\/p>\n

\u6982\u7387:<\/span><\/p>\n