Skip navigation

emCompress-Flex

  • ...
  • ...

emCompress-Flex is a compression toolkit, designed to compress data on a host computer and decompress it on a target system.

emCompress-Flex implements the lossless compression algorithm LZMA, which is used by popular compression tools, such as 7zip. emCompress-Flex enables you to use LZMA decompression even on small embedded targets. For compression on a host computer you can use standard tools or the tools and libraries included in emCompress-Flex.

Use Cases

emCompress-Flex LZMA can be used, when the system or product requires the use of a standard compression algorithm. When you freely design a new system that should use compression, have a look at emCompress-ToGo, too.

Compression with LZMA requires memory and computing power, which is usually available on computers. In contrast decompression with LZMA can be done with a lot less resources, which makes it usable even on small microcontrollers. emCompress-Flex splits LZMA into these two parts, compression on a host and decompression on the target.

emCompress-Flex can be used whenever compressed data needs to be sent to the target, such as:

  • Compress firmware images for deployment and dynamically decompress them on device update. Do faster over-the-air updates and save storage and bandwidth for the firmware images.
  • Compress larger data blobs which are sent to the target for processing. Increase the communication bandwidth.
  • Compress static files which are stored in the target's memory and decompress them on-demand. Reduce memory use on the target.

emCompress-Flex includes the compressor in a simple-to-use tool, as well as in a dynamic library to easily include compression on your tools. emCompress-Flex is also compatible with standard LZMA compression tools.

The decompressor is included in C source and can be included into any application, running on a computer or on a microcontroller. It is not limited to 32-bit architectures, but can be used even on 8-bit and 16-bit devices.

Compression on the Host

Data files, such as firmware images can easily be compressed with the compression tool included in emCompress-Flex. Once compressed, you can simply deploy the compressed file and send it to the target, for example on a USB stick or via over-the-air updates.

When you use an application to handle dynamic input data that you now want to compress, simply include the compressor library in your application. For example you developed an application to configure some parameters of the FPGA on your target which outputs the FPGA bitstream, you can instead output a compressed version of the bitstream and send it to the target.

Decompression on Target

A target system can handle any compressed input and decompress it into memory or for immediate processing.

On compressed firmware updates, the target decompresses the received data and immediately writes it to flash in small chunks. There is no need to decompress the whole firmware image first an then do the update.

When static content is stored on the device and requested to be used, the target decompresses the requested data and can store it into RAM or immediately process it. If only parts are requested, this can be handeld by the decompressor, too.

Compression ratios

The compression ratio, i.e. compressed size / original size, depends on the compressor parameters as well as on the input data. Better compression usually also requires more RAM to be available for decompression.

We have been looking at a large variety of different data types and tested different compression parameters to get some compression ratios for comparison.

As LZMA has different parameters for compression, more than 200 combinations are possible with each window size. It is not possible to find optimal compression parameters, in general, but it is possible to change the parameters and search for the combination that provides best compression. emCompress-Flex includes a tool, which can do this for you.

The table below displays the best compression for each file with different window sizes, each compressed with the best combination of compression parameters.

Type of dataDataWindow size
512 bytes1 kB2 kB4 kB
HTML file
(324751 bytes)
 27.06% (87866 bytes)22.80% (74045 bytes)18.63% (60498 bytes)13.24% (42988 bytes)
10027!CALCULATE_BAR_1!22!CALCULATE_BAR_3!18!CALCULATE_BAR_5!13!CALCULATE_BAR_7!
Blog article
(20895 bytes)
 40.31% (8423 bytes)35.64% (7448 bytes)33.21% (6939 bytes)31.00% (6478 bytes)
10040!CALCULATE_BAR_1!35!CALCULATE_BAR_3!33!CALCULATE_BAR_5!31!CALCULATE_BAR_7!
FPGA bitstream
(114618 bytes)
 23.39% (26807 bytes)23.24% (26633 bytes)23.07% (26440 bytes)20.57% (23572 bytes)
10023!CALCULATE_BAR_1!23!CALCULATE_BAR_3!23!CALCULATE_BAR_5!20!CALCULATE_BAR_7!
MIDI file
(56805 bytes)
 29.87% (16969 bytes)20.03% (11377 bytes)18.18% (10328 bytes)15.00% (8518 bytes)
10029!CALCULATE_BAR_1!20!CALCULATE_BAR_3!18!CALCULATE_BAR_5!15!CALCULATE_BAR_7!
Firmware
(218824 bytes)
 36.14% (79076 bytes)33.96% (74303 bytes)33.06% (72341 bytes)32.53% (71189 bytes)
10036!CALCULATE_BAR_1!33!CALCULATE_BAR_3!33!CALCULATE_BAR_5!32!CALCULATE_BAR_7!
FirmwareHEX
(615547 bytes)
 29.04% (178764 bytes)28.34% (174445 bytes)27.90% (171764 bytes)27.65% (170176 bytes)
10029!CALCULATE_BAR_1!28!CALCULATE_BAR_3!27!CALCULATE_BAR_5!27!CALCULATE_BAR_7!
Type of dataDataWindow size
8 kB16 kB32 kB64 kB
HTML file
(324751 bytes)
 12.63% (41028 bytes)12.32% (40006 bytes)12.01% (39005 bytes)11.72% (38064 bytes)
10012!CALCULATE_BAR_1!12!CALCULATE_BAR_3!12!CALCULATE_BAR_5!11!CALCULATE_BAR_7!
Blog article
(20895 bytes)
 30.08% (6285 bytes)29.61% (6186 bytes)29.61% (6188 bytes)29.61% (6188 bytes)
10030!CALCULATE_BAR_1!29!CALCULATE_BAR_3!29!CALCULATE_BAR_5!29!CALCULATE_BAR_7!
FPGA bitstream
(114618 bytes)
 20.17% (23117 bytes)20.11% (23046 bytes)19.83% (22726 bytes)19.64% (22514 bytes)
10020!CALCULATE_BAR_1!20!CALCULATE_BAR_3!19!CALCULATE_BAR_5!19!CALCULATE_BAR_7!
MIDI file
(56805 bytes)
 14.50% (8235 bytes)14.45% (8209 bytes)14.14% (8032 bytes)14.15% (8037 bytes)
10014!CALCULATE_BAR_1!14!CALCULATE_BAR_3!14!CALCULATE_BAR_5!14!CALCULATE_BAR_7!
Firmware
(218824 bytes)
 32.09% (70212 bytes)31.91% (69832 bytes)31.83% (69654 bytes)31.74% (69456 bytes)
10032!CALCULATE_BAR_1!31!CALCULATE_BAR_3!31!CALCULATE_BAR_5!31!CALCULATE_BAR_7!
FirmwareHEX
(615547 bytes)
 27.58% (169749 bytes)27.28% (167917 bytes)27.34% (168303 bytes)27.41% (168708 bytes)
10027!CALCULATE_BAR_1!27!CALCULATE_BAR_3!27!CALCULATE_BAR_5!27!CALCULATE_BAR_7!

Performance

... decompression performance on target ...

Resource Usage

RAM Usage

The decompressor has no static data requirement, the only memory required to decompress a stream is provided to the decompressor by the client in a context object.

The decoder context size varies according to selected emCompress-Flex LZMA decoder configuration and the specific compilation options selected to compile the decompressor.

The sizes presented here are for a 32-bit Cortex-M processor which should generally translate well to many 32-bit architectures including 32-bit “x86” targets.

It must be stressed that this computation is an estimate and the exact memory requirement can be computed with the sizeof operator applied to the context.

The following table displays the decompressor context size for each combination of LC. LP, and PB. Each size is measured in kilobytes with a zero window size, for a typical 32-bit byte-addressed machine.

LPPBLC=0LC=1LC=2LC=3LC=4LC=5LC=6LC=7LC=8
003.8 kB5.3 kB8.3 kB14.3 kB26.3 kB50.3 kB98.3 kB194.3 kB386.3 kB
014.0 kB5.5 kB8.5 kB14.5 kB26.5 kB50.5 kB98.5 kB194.5 kB386.5 kB
024.3 kB5.8 kB8.8 kB14.8 kB26.8 kB50.8 kB98.8 kB194.8 kB386.8 kB
034.9 kB6.4 kB9.4 kB15.4 kB27.4 kB51.4 kB99.4 kB195.4 kB387.4 kB
046.2 kB7.7 kB10.7 kB16.7 kB28.7 kB52.7 kB100.7 kB196.7 kB388.7 kB
105.3 kB8.3 kB14.3 kB26.3 kB50.3 kB98.3 kB194.3 kB386.3 kB770.3 kB
115.5 kB8.5 kB14.5 kB26.5 kB50.5 kB98.5 kB194.5 kB386.5 kB770.5 kB
125.8 kB8.8 kB14.8 kB26.8 kB50.8 kB98.8 kB194.8 kB386.8 kB770.8 kB
136.4 kB9.4 kB15.4 kB27.4 kB51.4 kB99.4 kB195.4 kB387.4 kB771.4 kB
147.7 kB10.7 kB16.7 kB28.7 kB52.7 kB100.7 kB196.7 kB388.7 kB772.7 kB
208.3 kB14.3 kB26.3 kB50.3 kB98.3 kB194.3 kB386.3 kB770.3 kB1538.3 kB
218.5 kB14.5 kB26.5 kB50.5 kB98.5 kB194.5 kB386.5 kB770.5 kB1538.5 kB
228.8 kB14.8 kB26.8 kB50.8 kB98.8 kB194.8 kB386.8 kB770.8 kB1538.8 kB
239.4 kB15.4 kB27.4 kB51.4 kB99.4 kB195.4 kB387.4 kB771.4 kB1539.4 kB
2410.7 kB16.7 kB28.7 kB52.7 kB100.7 kB196.7 kB388.7 kB772.7 kB1540.7 kB
3014.3 kB26.3 kB50.3 kB98.3 kB194.3 kB386.3 kB770.3 kB1538.3 kB3074.3 kB
3114.5 kB26.5 kB50.5 kB98.5 kB194.5 kB386.5 kB770.5 kB1538.5 kB3074.5 kB
3214.8 kB26.8 kB50.8 kB98.8 kB194.8 kB386.8 kB770.8 kB1538.8 kB3074.8 kB
3315.4 kB27.4 kB51.4 kB99.4 kB195.4 kB387.4 kB771.4 kB1539.4 kB3075.4 kB
3416.7 kB28.7 kB52.7 kB100.7 kB196.7 kB388.7 kB772.7 kB1540.7 kB3076.7 kB
4026.3 kB50.3 kB98.3 kB194.3 kB386.3 kB770.3 kB1538.3 kB3074.3 kB6146.3 kB
4126.5 kB50.5 kB98.5 kB194.5 kB386.5 kB770.5 kB1538.5 kB3074.5 kB6146.5 kB
4226.8 kB50.8 kB98.8 kB194.8 kB386.8 kB770.8 kB1538.8 kB3074.8 kB6146.8 kB
4327.4 kB51.4 kB99.4 kB195.4 kB387.4 kB771.4 kB1539.4 kB3075.4 kB6147.4 kB
4428.7 kB52.7 kB100.7 kB196.7 kB388.7 kB772.7 kB1540.7 kB3076.7 kB6148.7 kB

ROM Usage

... ROM Usage for decompression ...