Unit-5

Entropy encoding: Entropy coding is also known as “zero-error coding”, “data compression” or
“lossless compression”. Entropy coding is widely used in virtually all popular international
multimedia compression standards such as JPEG and MPEG.

A complete entropy codec, which is an encoder/decoder pair, consists of the process of

“encoding” or “compressing” a random source (typically quantized transform coefficients) and
the process of “decoding” or “decompressing” the compressed signal to “perfectly” regenerate
the original random source. In other words, there is no loss of information due to the process of
entropy coding. Thus, entropy coding does not introduce any distortion, and hence, the
combination of the entropy encoder and entropy decoder faithfully reconstructs the input to the
entropy encoder.

Run Length Encoding or RLE, is a simple technique used to compress runs of identical
symbols in a data stream. Typically RLE encodes a run of symbols as a symbol and a count. The
simplicity of RLE and ease of implementation result in its frequent appearance in compression
programs, although its performance is relatively poor.

Run-Length Encoding (RLE) is a simple and widely used compression algorithm. It is particularly
effective for data with long sequences of repeated characters. This step-by-step guide will walk
you through the implementation of the RLE algorithm in JavaScript, with explanations of each
step, real examples, and analysis of time and space complexity.

The goal of Run-Length Encoding is to compress a string by replacing sequences of repeated

characters with a single character followed by the number of times it’s repeated.

Examples:
Let’s illustrate the algorithm with a couple of examples:

• Example 1:

• Input: “AAAABBBCCDAA”

• Output: “4A3B2C1D2A”

• Example 2:

• Input:

“WWWWWWWWWWWWBWWWWWWWWWWWWBBBWWWWWWWWWWWW

WWWWWWWWWWWWB”

• Output: “12WB12W3B24WB”

Statistical Encoding

A variable-length code (VLC) takes a given symbol from your input stream and assigns it a
codeword that has a variable number of bits. When you apply this to a long-enough data
set, you end up with an average bit-stream length that is close to the entropy of the data
set—as long as the probability of your symbol distribution matches the probability table of
the encoding scheme you used to build your VLC.

Examples are Huffman, Arithmetic & Lempel-Ziv coding

Huffman: 1. Huffman Coding

Huffman began with two rather obvious conditions on the code and then added a third that
allowed for
the construction of the code. The conditions were:
1. The codes corresponding to the higher probability letters could not be longer than the
code words associated with the lower probability letters.
2. The two lowest probability letters had to have code words of the same length.
He added a third condition to generate a practical
compression scheme.
3. The two lowest probability letters have codes that are identical except for the last bit.

It is easy to visualize these conditions if we think of the code words as the path through a binary
tree: a zero
bit denoting a left branch and a one bit denoting a right branch. The two lowest probability letters
would
then be the two leaves of a node at the lowest level of the tree. We can consider the parent node
of these
leaves as a letter in a reduced alphabet which is obtained as a combination of the two lowest
probability
symbols. The probability of the letter corresponding to this node would be the sum of the
probabilities of the
two individual letters. Now, we can find the two lowest probability symbols of the reduced
alphabet and treat
them as the two leaves of a common node. We can continue in this fashion until we have
completed the tree.
We can see this process best through an example.

EXAMPLE 1: HUFFMAN CODING

Suppose we have a source alphabet with five letters {a1,a2,a3,a4,a5} with probabilities of
occurrence P(a1)
= 0.15, P(a2) =0.04, P(a3) =’ 0.26, P(a4) = 0.05, and P(a5) =0.50. We can calculate the entropy
to be

If we sort the probabilities in descending order we can see that the two letters with the lowest
probabilities are
a2 and a4. These will become the leaves on the lowest level of the binary tree. The parent node
of these
leaves will have a probability of 0.09. If we consider the parent node as a letter in a reduced
alphabet it will be
one of the two letters with the lowest probability: the other one being a1. Continuing in this
manner we get
the binary tree shown in Fig. 3. The code is

It can be shown that, for a sequence of independent letters, or a memoryless source, the rate of
the Huffman
code will always be within one bit of the entropy.

H <=R <= 1

In fact we can show that (Gallagher, 1978) if pmax is the largest probability in the probability
model, then

for pmax < 0.5 the upper bound for the Huffman code is H(S) +pmax, while for pmax >= 0.5, the
upper bound
is H(S) + pmax +0.086. If instead of coding each letter separately we group the letters into blocks
containing
n letters, then the rate is guaranteed to be even closer to the entropy of the source. Using our
looser bound we can show that the bounds on the average length of the code will be

H<=R<=1/n
However, blocking letters together means an exponential increase in the size of the alphabet.
Therefore,
in many situations this particular approach is not very practical.

Arithmetic Coding

Arithmetic encoding (AE) is a lossless algorithm that uses a low number of bits to compress
data.
It’s an entropy-based algorithm One reason why AE algorithms reduce the number of bits is that
AE encodes the entire message using a single number between 0.0 and 1.0. Each symbol in the
message takes a sub-interval in the 0-1 interval, corresponding to its probability.

To calculate the probability of each symbol, a frequency table should be given as an input to the
algorithm. This table maps each character to its frequency.

The more frequent the symbol, the lower number of bits it is assigned. As a result, the total
number of bits representing the entire message is reduced. This is compared to using a fixed
number of bits for all symbols in the message as in Huffman coding.

Frequency table

To encode (= compress) a message using AE, you need these inputs:

1. The message you want to encode.

2. The frequency table of all possible symbols in the message.

The frequency table is usually calculated by analyzing different messages and concluding by the
number of times each symbol is repeated.

For example: assume that the messages to be encoded consist of just 3 symbols a, b, and c. By
analyzing some previous messages (e.g. babb, cbab, and bb), we get the frequencies of the 3
symbols:

1. a=2
2. b=7
3. c=1

Based on the frequency table, how do you encode a message like abc using AE? Before you start
encoding, you need to calculate a probability table out of the frequency table.

Probability table

Using the frequency table, we can calculate the probability of occurrence of each symbol. The
probability is calculated as follows:

Frequency of a symbol/Sum of frequencies of all symbols

Based on the frequency table, here are the probabilities of our 3 symbols:

1. p(a)=2/10=0.2
2. p(b)=7/10=0.7
3. p(c)=1/10=0.1

Given the message and the probability table, AE can start the encoding process.
Encoding

Encoding in AE works by representing the cumulative probabilities of all symbols on a line that
ranges from 0.0 to 1.0. On that line, each symbol uses a sub-range.

Given any symbol C, it starts from the value Sand ends at the value calculated using the
following equation:

S+(P(C))*R

Where:

• S: The cumulative sum of all previous probabilities.

• P(C): The probability of symbol C.
• R: The range of line which is calculated by subtracting the start from the end of the line.

In the beginning, the line starts from 0.0 and ends at 1.0, thus R=1.0.

Let’s calculate the start and end values of the 3 symbols a, b, and c.

1. For the first symbol, a, it starts from 0.0 to 0.0+(0.2)*1.0=0.2. The start value is 0.0
because it’s the first symbol. The range is (0.0:0.2).
2. For the second symbol, b, it starts from 0.2 to 0.2+(0.7)*1.0=0.9. The range is (0.2:0.9).
3. The third symbol, c, starts from 0.9 to 0.9+(0.1)*1.0=1.0. The range is (0.9:1.0).

Given the ranges of all symbols, it’s good to represent them graphically, as in this figure:

Here are some notes:

1. The first symbol starts from the same start point of the line (0.0) and the last symbol ends
at the endpoint of the line (1.0).
2. A symbol C covers a percentage of the range that corresponds to its probability. For
example, the symbol b covers 70% of the line because its probability is 0.7.

Restricting the interval

AE works by restricting the line interval, which starts from 0.0 to 1.0, through some stages equal
to the number of symbols in the message. In this example, there are just 3 symbols in the
message, so there are just 3 stages.

In each stage, the line’s interval is restricted according to the sub-interval of the current symbol.
After all symbols are processed, AE returns a single double value encoding the entire message.

Now, it’s time to look at the message to be encoded, abc, and work on the first symbol a to
restrict the line interval.Based on the previous figure, the symbol a covers the interval from 0.0
to 0.2. This interval becomes the line’s interval. In other words, the line interval changes from
0.0:1.0 to 0.0:0.2 as in the next figure. Note that symbol a starts from 0.0 and the last
symbol, c, ends at 0.2.

To calculate the interval of each symbol, we’ll use the already mentioned formula:

S+(P(C))*R

Because the line’s range has changed, R changes to 0.2-0.0=0.2. Let’s calculate the start and end
values for each symbol:

1. For the first symbol, a, it starts from 0.0 to 0.0+(0.2)*0.2=0.04. The start value is 0.0
because it’s the first symbol. The range is (0.0:0.04).
2. For the second symbol b, it starts from 0.04 to 0.04+(0.7)*0.2=0.18. The range is
(0.04:0.18).
3. The third symbol c starts from 0.18 to 0.18+(0.1)*0.2=0.2. The range is (0.18:0.2).

At the end of the first stage, here is the figure that shows the start and end values of each
symbol’s interval.

Note that each symbol takes a portion of the line equal to its probability. For example, the
probability of the symbol b is 0.7 and thus it takes 70% of the line’s interval.

The next figure summarizes the progress of AE at this point. Let’s start the second stage, which
just repeats what we did in the first stage.

The next symbol in the message is b. According to the latest line interval (0.0 to 0.2), the
symbol b starts from 0.04 to 0.18. In this stage, the interval of the line will be further restricted to
b’s interval, as shown in this figure:
Because the line’s interval changed to (0.04:0.18), R also changes. Its new value is 0.18-
0.04=0.14. Let’s calculate the start and end values for each symbol:

1. For the first symbol, a, it starts from 0.04 to 0.04+(0.2)*0.14=0.068. The range is
(0.04:0.068).
2. For the second symbol, b, it starts from 0.068 to 0.068+(0.7)*0.14=0.166. The range is
(0.068:0.166).
3. The third symbol, c, starts from 0.166 to 0.166+(0.1)*0.14=0.18. The range is
(0.166:0.18).

The next figure summarizes the intervals of the 3 symbols. This marks the end of the second
stage.

To summarize the progress made up to this point, the next figure connects the 2 completed
stages. Let’s move to the third stage.

The third and last symbol in the message is c. According to the latest stage, this symbol falls
within the range starting from 0.166 and ending at 0.18. c’s current interval will be the next
interval of the line according to the next figure. R for this new line is 0.18-0.166=0.014.

On that line, similarly to previous 2 stages, here are the intervals of the 3 symbols:

1. symbol a starts from 0.166 to 0.166+(0.2)*0.014=0.1688. The range is (0.166 :0.1688).

2. symbol b starts from 0.1688 to 0.1688+(0.7)*0.014=0.1786. The range is (0.1688
:0.1786).
3. symbol c starts from 0.1786 to 0.1786+(0.1)*0.014=0.18. The range is (0.1786:0.18).

The intervals are reflected in the next figure.

There are no more symbols in the message. The next figure summarizes the calculations from the
initial interval (0.0:1.0) to the last interval (0.166:0.18).
Given that AE completed the encoding process, the next subsection calculates the single number
that encodes the entire message.

Single value encoding the message

The latest interval reached by AE starts from 0.166 to 0.18. Within this range, any value can be
used to encode the entire message. For example, the value could be 0.17.

Through this tutorial, the value is the average of the interval which is:

(0.166+0.18)/2=0.173

So, based on the frequency table used, the message abc is encoded as the value 0.173. When the
frequency table changes, then the value that encodes the message changes too.

This marks the end of the encoding process. Now let’s discuss the decoding process.

Decoding

The inputs for decoding are:

1. The single value that encodes the message.

2. The frequency table. It should be identical to the one used for encoding.
3. The number of symbols in the original message.

In our example, the value that encodes the message is 0.173. The frequency table is [a=2, b=7,
c=1]. The number of symbols in the message is 3.

The first step of decoding is to calculate the probabilities of the symbols out of their frequency,
similarly to what we did before.

The probability table is [p(a)=0.2, p(b)=0.7, p(c)=0.1]. Based on the probability table, the
decoding process works similarly to encoding by constructing the same intervals.

First, a line starting from 0.0 and ending at 1.0 is created according to the next figure. The
intervals of the 3 symbols are calculated using the same equation used for encoding the message.

S+(P(C))*R

The value 0.173 falls within the first interval (0.0:0.2). Thus, the first symbol in the encoded
message is a. As a result, the line’s interval is restricted to the interval that starts from 0.0 to 0.2.
The value 0.173 falls within the second interval (0.04:0.18). Thus, the second symbol in the
encoded message is b. In the next figure, the interval will be restricted to the interval 0.04 to
0.18.

The value 0.173 falls within the third interval (0.166:0.18). This is why the next symbol in the
message is c.

Remember that the number of symbols in the original message is 3. After decoding 3 symbols,
the decoding process is completed. The decoded message is abc.

LZW (Lempel–Ziv–Welch) Compression technique

The LZW algorithm is a very common compression technique. This algorithm is typically used
in GIF and optionally in PDF and TIFF. Unix’s ‘compress’ command, among other uses. It is
lossless, meaning no data is lost when compressing. The algorithm is simple to implement and
has the potential for very high throughput in hardware implementations. It is the algorithm of
the widely used Unix file compression utility compress and is used in the GIF image format.
The Idea relies on reoccurring patterns to save data space. LZW is the foremost technique for
general-purpose data compression due to its simplicity and versatility.
LZW compression works by reading a sequence of symbols, grouping the symbols into strings,
and converting the strings into codes. Because the codes take up less space than the strings
they replace, we get compression. Characteristic features of LZW includes,
• LZW compression uses a code table, with 4096 as a common choice for the number of
table entries. Codes 0-255 in the code table are always assigned to represent single bytes
from the input file.
• When encoding begins the code table contains only the first 256 entries, with the remainder
of the table being blanks. Compression is achieved by using codes 256 through 4095 to
represent sequences of bytes.
• As the encoding continues, LZW identifies repeated sequences in the data and adds them to
the code table.
• Decoding is achieved by taking each code from the compressed file and translating it
through the code table to find what character or characters it represents.
Example: ASCII code. Typically, every character is stored with 8 binary bits, allowing up to
256 unique symbols for the data. This algorithm tries to extend the library to 9 to 12 bits per
character. The new unique symbols are made up of combinations of symbols that occurred
previously in the string. It does not always compress well, especially with short, diverse
strings. But is good for compressing redundant data, and does not have to save the new
dictionary with the data: this method can both compress and uncompress data.
 • Compression using LZW
• Example 1: Use the LZW algorithm to compress the string: BABAABAAA
The steps involved are systematically shown in the diagram below.

LZW Decompression
The LZW decompressor creates the same string table during decompression. It starts with the
first 256 table entries initialized to single characters. The string table is updated for each
character in the input stream, except the first one. Decoding is achieved by reading codes and
translating them through the code table being built.

Example 2: LZW Decompression: Use LZW to decompress the output sequence of

: <66><65><256><257><65><260>
The steps involved are systematically shown in the diagram below.
In this example, 72 bits are represented with 72 bits of data. After a reasonable string table is
built, compression improves dramatically.
LZW Summary: This algorithm compresses repetitive sequences of data very well. Since the
codewords are 12 bits, any single encoded character will expand the data size rather than
reduce it.

Advantages of LZW over Huffman:

• LZW requires no prior information about the input data stream.
• LZW can compress the input stream in one single pass.
• Another advantage of LZW is its simplicity, allowing fast execution.
High Compression Ratio: LZW can achieve high compression ratios, particularly for text-
based data, which can significantly reduce file sizes and storage requirements.
Fast Decompression: LZW decompression is typically faster than other compression
algorithms, making it a good choice for applications where decompression speed is critical.
Universal Adoption: LZW is widely used and supported across a variety of software
applications and operating systems, making it a popular choice for compression and
decompression.
Dynamic Compression: LZW uses a dynamic compression algorithm, meaning it adapts to
the data being compressed, which allows it to achieve high compression ratios even for data
with repetitive patterns.
Disadvantages:

Patent Issues: LZW compression was patented in the 1980s, and for many years its use was
subject to licensing fees, which limited its adoption in some applications.
Memory Requirements: LZW compression requires significant memory to maintain the
compression dictionary, which can be a problem for applications with limited memory
resources.
Compression Speed: LZW compression can be slower than some other compression
algorithms, particularly for large files, due to the need to constantly update the dictionary.
Limited Applicability: LZW compression is particularly effective for text-based data, but may
not be as effective for other types of data, such as images or video, which have different
compression requirements.

Quantization Quantization is the process of representing the output of a source with a large
(possibly infinite) alphabet with a small alphabet. It is a many-to-one mapping and therefore
irreversible. Quantization can be performed on a sample-by-sample basis or it can be performed
on a group of samples. The former is called scalar quantization and the latter is called vector
quantization. We look at each in turn

Vector Quantization (VQ) is a powerful technique used in data compression, primarily for
compressing signals, images, and speech data. It is a type of lossy compression technique where
the goal is to represent large amounts of data in a smaller, more compact form while maintaining
as much of the important information as possible.

Basic Concept

In essence, vector quantization works by partitioning the input data (which is typically high-
dimensional, like an image or speech signal) into a set of vectors and then mapping each vector
to a representative codeword from a codebook. This results in a compressed representation of
the data by substituting original vectors with these codewords.

Steps Involved in VQ:

1. Data Representation: The first step is to convert the raw data into a set of vectors. For
example, in image compression, an image can be divided into small blocks (like 8x8 or
16x16 pixel blocks), where each block becomes a vector of pixel values.
2. Training the Codebook: A codebook is created during the training phase. The
codebook is a collection of representative vectors (called codewords). The codebook is
typically generated using a clustering algorithm like k-means clustering. The goal of
training is to find a set of vectors that can best represent the data, such that each input
vector in the dataset is mapped to its nearest codeword in the codebook.
3. Quantization: After the codebook is trained, each vector from the input data is
quantized, meaning it is replaced by the closest codeword from the codebook. This step
reduces the size of the data because each vector, instead of being stored in its full high-
 dimensional form, is now represented by a much smaller index corresponding to the
closest codeword.
4. Encoding: The quantized data is then encoded using a small index (often a binary code)
for each codeword. The number of bits needed to represent each index depends on the
size of the codebook.
5. Decoding: To reconstruct the data, the index values are decoded, and the corresponding
codewords from the codebook are used to reconstruct the original vectors. Since VQ is
lossy, the reconstruction is an approximation of the original data.

Example in Image Compression

In image compression, for instance:

1. The image is divided into small non-overlapping blocks (e.g., 8x8 pixels).
2. Each block is flattened into a vector (64 values for an 8x8 block).
3. A codebook is trained using these vectors. Each codeword in the codebook represents a
typical block in the image.
4. Each 8x8 block of the image is replaced with the index of the closest codeword in the
codebook.
5. The compressed image is represented by a smaller set of indices, rather than the original
pixel values.

Advantages of Vector Quantization

• Effective Data Compression: VQ can achieve high compression ratios, especially for
large datasets such as images and speech.
• Reduction of Redundancy: VQ leverages the redundancy in the data, mapping similar
vectors to the same codeword.
• Low Complexity: Once the codebook is trained, encoding and decoding can be done
relatively efficiently.

Disadvantages

• Lossy Compression: Since VQ involves approximating the data with a set of codewords,
there is a loss of information. The quality of the compressed data can degrade, especially
if the codebook is not sufficiently large or well-trained.
• Codebook Size: The effectiveness of VQ heavily depends on the size and quality of the
codebook. A small codebook leads to poor compression, while a large codebook may
reduce compression efficiency and increase storage requirements.
• Training Cost: Training a good codebook (especially for large datasets) can be
computationally expensive, although this is generally done offline.
Applications of Vector Quantization

1. Image and Video Compression: VQ is used in various image and video compression
techniques, such as JPEG, where the image is divided into blocks, and each block is
quantized.
2. Speech Compression: In speech compression, VQ is often used to compress speech
signals by representing a series of speech frames as vectors, which are then quantized.
3. Audio Compression: VQ can be used for compressing audio signals by reducing the
number of bits required to store a set of audio features.
4. Machine Learning: VQ techniques are sometimes used in clustering and feature
extraction in machine learning tasks.

Encryption Trends

1. Post-Quantum Cryptography
o Overview: As quantum computing becomes more advanced, there is a growing
need to develop cryptographic algorithms that can withstand the computational
power of quantum machines. Quantum computers could theoretically break many
of the cryptographic schemes currently in use, such as RSA and ECC (Elliptic
Curve Cryptography), by efficiently solving mathematical problems like integer
factorization and the discrete logarithm problem.
o Trend: The shift towards Post-Quantum Cryptography (PQC), which includes
algorithms resistant to quantum attacks, is one of the most significant trends.
These include lattice-based cryptography (e.g., NTRU), hash-based cryptography,
code-based cryptography, and multivariate polynomial encryption.
o Standardization: The National Institute of Standards and Technology (NIST) is
leading efforts to standardize post-quantum cryptographic algorithms. In 2022,
NIST announced the first round of algorithms, which will be used in the future
when quantum computers become a practical threat.
2. Homomorphic Encryption (HE)
o Overview: Homomorphic encryption allows computations to be performed on
encrypted data without first decrypting it. This is ideal for scenarios where data
privacy is crucial, but computations on sensitive data are required, such as in
cloud computing and secure data analytics.
o Trend: There has been increasing interest and development in Fully
Homomorphic Encryption (FHE), which enables arbitrary computations on
encrypted data. While it is still computationally intensive and not widely adopted
for practical purposes, its potential for privacy-preserving computations is
significant, particularly in the fields of cloud computing, healthcare, and finance.
3. Zero-Knowledge Proofs (ZKPs)
o Overview: Zero-Knowledge Proofs allow one party to prove to another party that
they know a value without revealing the value itself. This is particularly useful in
privacy-preserving cryptographic protocols and authentication systems.
 o Trend: Zero-Knowledge Succinct Non-Interactive Arguments of Knowledge
(zk-SNARKs) and zk-STARKs (Scalable Transparent Arguments of Knowledge)
are gaining popularity. These techniques are increasingly used in blockchain
technology (e.g., privacy coins like Zcash) and distributed ledger technologies
(DLTs), enabling secure and private transactions without revealing any sensitive
information.
4. Blockchain and Cryptocurrencies
o Overview: The integration of encryption in blockchain technologies and
cryptocurrencies has seen exponential growth, with new encryption methods
focusing on privacy, scalability, and security.
o Trend: Privacy-focused cryptocurrencies like Monero (using Ring Signatures
and Stealth Addresses) and Zcash (using zk-SNARKs) are pushing the
boundaries of encryption in blockchain for anonymous and untraceable
transactions. Additionally, Ethereum 2.0 introduces new cryptographic
approaches (e.g., BLS signatures) for scalability and efficiency.
5. Quantum Key Distribution (QKD)
o Overview: Quantum Key Distribution is a method of transmitting encryption keys
securely using the principles of quantum mechanics. It ensures that any
interception of the key transmission can be detected, making it theoretically
secure against quantum attacks.
o Trend: There is increasing investment in the development of QKD systems for
secure communications, especially in government and military sectors. Several
commercial companies and research groups are working to make QKD a practical
technology for high-security communication networks.

Data Compression Trends

1. Deep Learning-Based Compression

o Overview: Traditional compression techniques rely on well-defined algorithms
like Huffman coding, run-length encoding, and transform-based methods (e.g.,
DCT in JPEG). However, deep learning has introduced new paradigms, where
neural networks learn to compress data in a more adaptive manner.
o Trend: Deep Learning-based Compression (such as autoencoders and
Generative Adversarial Networks (GANs)) is increasingly used to improve the
efficiency and effectiveness of compression techniques. For example, neural
network-based image compression can outperform traditional methods by
learning the most important features and compressing them in a way that reduces
artifacts and preserves quality.
o Applications: This approach is being explored in image compression (e.g.,
JPEG-XL, WebP), video compression (e.g., AV1, VVC), and audio compression
(e.g., Opus codec with machine learning enhancements).
2. Advanced Video Compression Codecs
o Overview: Video streaming and broadcasting continue to grow in importance,
and there is an increasing demand for compression technologies that offer better
quality at lower bitrates.
 o Trend: New video codecs such as AV1 and VVC (Versatile Video Coding) are
emerging as successors to H.264/AVC and HEVC/H.265. These new codecs
utilize advanced compression techniques like machine learning, motion-
compensated prediction, and transform coding to achieve superior video
quality at lower bitrates.
o Applications: AV1 is being widely adopted for web-based video streaming (e.g.,
by YouTube and Netflix), while VVC is being promoted for high-definition, 4K,
and 8K video content.
3. Lossless Compression for Big Data
o Overview: With the increasing amount of data generated by businesses and
individuals, efficient compression techniques for big data are becoming more
crucial, especially for tasks that require lossless data retention.
o Trend: New lossless compression algorithms (e.g., Zstandard (ZSTD), LZ4)
are being developed to improve the compression of large datasets without
sacrificing speed or decompression performance. These algorithms are designed
to scale well for large, real-time data streams, especially in cloud storage,
databases, and big data processing.
4. Multimedia Compression for 5G and IoT
o Overview: The advent of 5G networks and the Internet of Things (IoT) will
generate massive amounts of data that need to be efficiently compressed for
transmission and storage.
o Trend: Emerging techniques in multimedia compression are being developed to
optimize data for low-latency, high-throughput applications. For example,
lightweight codecs (e.g., HEVC for 5G video transmission) and adaptive
compression techniques (tailored for IoT devices) are being designed to manage
the high volume of diverse data traffic in 5G and IoT environments.
o Applications: Smart cities, autonomous vehicles, and remote healthcare services
require real-time, bandwidth-efficient data transmission, driving the development
of specialized compression technologies for these applications.
5. Compression for Machine Learning and AI
o Overview: Machine learning and AI models, particularly deep learning models,
have become significantly large and require efficient storage and transmission
methods.
o Trend: Research in model compression aims to reduce the size of machine
learning models without compromising their performance. Techniques like
weight pruning, quantization, knowledge distillation, and matrix
decomposition are being used to compress models so they can run on resource-
constrained devices (e.g., mobile phones, embedded systems) and be transmitted
more efficiently in edge computing and cloud environments.