Entropy coding

百度他们在这里一同生活，互相扶持。

In information theory, an entropy coding (or entropy encoding) is any lossless data compression method that attempts to approach the lower bound declared by Shannon's source coding theorem, which states that any lossless data compression method must have an expected code length greater than or equal to the entropy of the source.^[1]

More precisely, the source coding theorem states that for any source distribution, the expected code length satisfies $\operatorname {E} _{x\sim P}[\ell (d(x))]\geq \operatorname {E} _{x\sim P}[-\log _{b}(P(x))]$ , where $\ell$ is the function specifying the number of symbols in a code word, $d$ is the coding function, $b$ is the number of symbols used to make output codes and $P$ is the probability of the source symbol. An entropy coding attempts to approach this lower bound.

Two of the most common entropy coding techniques are Huffman coding and arithmetic coding.^[2] If the approximate entropy characteristics of a data stream are known in advance (especially for signal compression), a simpler static code may be useful. These static codes include universal codes (such as Elias gamma coding or Fibonacci coding) and Golomb codes (such as unary coding or Rice coding).

Since 2014, data compressors have started using the asymmetric numeral systems family of entropy coding techniques, which allows combination of the compression ratio of arithmetic coding with a processing cost similar to Huffman coding.

Entropy as a measure of similarity

Besides using entropy coding as a way to compress digital data, an entropy encoder can also be used to measure the amount of similarity between streams of data and already existing classes of data. This is done by generating an entropy coder/compressor for each class of data; unknown data is then classified by feeding the uncompressed data to each compressor and seeing which compressor yields the highest compression. The coder with the best compression is probably the coder trained on the data that was most similar to the unknown data.

References

^ Duda, Jarek; Tahboub, Khalid; Gadgil, Neeraj J.; Delp, Edward J. (May 2015). "The use of asymmetric numeral systems as an accurate replacement for Huffman coding". 2015 Picture Coding Symposium (PCS). pp. 65–69. doi:10.1109/PCS.2015.7170048. ISBN 978-1-4799-7783-3. S2CID 20260346.
^ Huffman, David (1952). "A Method for the Construction of Minimum-Redundancy Codes". Proceedings of the IRE. 40 (9). Institute of Electrical and Electronics Engineers (IEEE): 1098–1101. doi:10.1109/jrproc.1952.273898. ISSN 0096-8390.

External links

Information Theory, Inference, and Learning Algorithms, by David MacKay (2003), gives an introduction to Shannon theory and data compression, including the Huffman coding and arithmetic coding.
Source Coding, by T. Wiegand and H. Schwarz (2011).

[1] Duda, Jarek; Tahboub, Khalid; Gadgil, Neeraj J.; Delp, Edward J. (May 2015). "The use of asymmetric numeral systems as an accurate replacement for Huffman coding". 2015 Picture Coding Symposium (PCS). pp. 65–69. doi:10.1109/PCS.2015.7170048. ISBN 978-1-4799-7783-3. S2CID 20260346.

[Huffman_1952_pp._1098–1101-2] Huffman, David (1952). "A Method for the Construction of Minimum-Redundancy Codes". Proceedings of the IRE. 40 (9). Institute of Electrical and Electronics Engineers (IEEE): 1098–1101. doi:10.1109/jrproc.1952.273898. ISSN 0096-8390.

[1]

[2]

借您吉言什么意思	什么动物吃蚊子	第一次做什么感觉	六月初九是什么日子	gop是什么
七月三号是什么日子	喘息性支气管炎吃什么药	血压高不能吃什么食物	16岁属什么	做完胃肠镜后可以吃什么
小腹痛吃什么药	和田玉对身体有什么好处	Joyce什么意思	MR医学上是什么意思	多动症是什么
食物中毒吃什么药解毒	9月27是什么星座	女孩和女人有什么区别	九月什么花开	什么牌子的氨糖好

为什么日语怎么说hcv7jop4ns5r.cn	instagram什么意思hcv8jop7ns9r.cn	五更是什么时辰hcv7jop6ns0r.cn	coach是什么牌子的包hcv7jop9ns2r.cn	武士是什么意思naasee.com
梦见自己大肚子快生了是什么意思hcv8jop4ns1r.cn	什么叫蜘蛛痣hcv7jop6ns4r.cn	noa是什么意思sscsqa.com	肠穿孔有什么症状hcv9jop2ns7r.cn	一个虫一个圣读什么hcv8jop4ns9r.cn
什么的蘑菇hcv9jop6ns9r.cn	9.25什么星座dajiketang.com	康复科是主要治疗什么hcv8jop1ns8r.cn	烦躁不安的意思是什么hcv9jop6ns2r.cn	黄酒是什么酒hcv7jop6ns7r.cn
主是什么结构的字体hcv9jop0ns6r.cn	c14呼气试验是检查什么的hcv9jop3ns9r.cn	攒是什么意思hcv9jop6ns5r.cn	女娲用什么补天shenchushe.com	梦见狗咬人是什么预兆hcv9jop5ns3r.cn

附件炎是什么原因引起的

Entropy as a measure of similarity

See also

References

External links