Adaptive color space model based on dominant colors for image and video compression performance improvement

This paper describes the use of some color spaces in JPEG image compression algorithm and their impact in terms of image quality and compression ratio, and then proposes adaptive color space models (ACSM) to improve the performance of lossy image compression algorithm. The proposed ACSM consists of, dominant color analysis algorithm and YCoCg color space family. The YCoCg color space family is composed of three color spaces, which are YCcCr, YCpCg and YCyCb. The dominant colors analysis algorithm is developed which enables to automatically select one of the three color space models based on the suitability of the dominant colors contained in an image. The experimental results using sixty test images, which have varying colors, shapes and textures, show that the proposed adaptive color space model provides improved performance of 3 % to 10 % better than YCbCr, YDbDr, YCoCg and YCgCo-R color spaces family. In addition, the YCoCg color space family is a discrete transformation so its digital electronic implementation requires only two adders and two subtractors, both for forward and inverse conversions.


Introduction
Raw images and video frames recorded by highresolution cameras in information and communication technology (ICT) devices are in the form of three basic color components R (red), G (green) and B (blue). These color components are called RGB color spaces. Saving high-resolution raw images and video frames in RGB color space requires enormous memory. Image and video compression algorithms are the main solutions to reduce data size and minimize storage requirements. JPEG and JPEG2000 are two compression algorithms that are widely used in ICT devices. In general, these two algorithms have the same process flow, as shown in fig. 1 [1,2,3]. The Compression algorithm in fig. 1a consists of the following processes: color space conversion, spatial to frequency transform, quantization and coding. Contrary, the decompression algorithm ( fig. 1b) consists of: decoding, inverse quantization, frequency to spatial transform and inverse color space conversion.
In image and video compression algorithms, the choice of color space is an important part, because it directly affects the compression ratio and image quality. This means that the use of the right color space can increase the compression ratio and image quality. Generally, the color space used is in accordance with the sensitivity of the human visual system to color changes, especially the color spaces that have luminance and chrominance components. The JPEG and JPEG2000 compression al-gorithms use color spaces YC b C r and YD b D r , which Y correspond to luminance, C b and D b are blue chrominance, and C r and D r are red chrominance components [3,4,5,6]. The Color conversion process from RGB to YC b C r can be calculated through a mathematical transformation as given in Eq. (1), while the inverse conversion process from YC b C r to RGB is shown by Eq. (2). Matrix A is called a color transformation matrix to which [A -1 ] is the inverse of matrix [A] and is called an inverse color transformation matrix. Numerical implementation of RGB to YC b C r conversion requires two shift-right operations, seven multiplications and six adders/subtractors. Numerical implementation of YC b C r to RGB conversion requires four multiplications and four adders / subtractors. This color space is an irreversible color transform because it is rounded during the process.  . (2) The YD b D r color space used in the JPEG2000 compression algorithm is a variant of YC b C r [2, 7 -9]. The matrices [A] and [A -1 ] in Eqs. (3,4) respectively show the transformation matrix for RGB to YD b D r conversion and inverse transformation matrix for YD b D r to RGB conversion. This color space is reversible color transform (RCT) and its numerical implementation is simpler because it only requires shift-right (symbolized by "»"), addition and subtraction operations, and without multiplication operation. For example, 0.5×R is equal to one-bit shiftright of R value (R » 1) and 0.25×B is identical to twobits shift-right of B value (B » 2). Thus, for RGB to YD b D r conversion requires two shift-right operations and four adders / subtractors. While YD b D r to RGB inverse conversion requires three shift-right and seven addition and subtraction operations.
Another variant of the YC b C r color space is YC o Cg, which has a conversion process easier and faster than YC b C r , and it is a reversible color transform [9,10]. YC o Cg is also known as the YCgCo color model. The matrix A is used for the conversion process from RGB to YC o Cg and the inverse transformation matrix [A -1 ] is applied for the conversion process from YC o Cg to RGB which are given in Eqs. (5,6). The numerical implementation of RGB to YC o Cg conversion requires four shiftright, two addition and tow subtraction operations. Likewise for YC o Cg to RGB conversion process requires only two addition and two subtraction operations.
YC o Cg color space is then developed into YCgC-R with the aim to speed-up the conversion process by multiplying by 2 each value of the chrominance components [11,12] (see Eqs. (7,8)). Numerical implementation of RGB to YCgCo-R conversion and its inverse conversion form YCgCo-R to RGB, both only require two shift-right, two addition and two subtraction operations.
In [13], the authors developed a new family of reversible low-complexity color transformation: Y i UVj. The YD b D r and YCgC o -R color spaces are included in this family. The C o = (R-G) / 2 and Cg = (2G-(R+B))/4 as chrominance components of YC o Cg color space (Eq. 5) are not yet included. From this family, the color space components Y i , Uj and Vj, which have computational with lower complexity, are given in table 1. The performance of each color space has been compared with the focus on lossy compression and by applying an automatic color space selection algorithm based on entropy H.
The image in R, G and B color components format must first be converted into each color component of Y i , Uj, and Vj, and then the entropy value is calculated for each of H(Y i ), H(Uj), and H(Vj). It is assumed that the one having the smallest entropy value (H = H (Y i ) + H (Uj) + H (Vj)), as the components of the color space, will be selected to be applied. To avoid high computation time, an image is divided into p × q macro blocks, where the number of blocks depends on the image size. For example, an image is divided into 9 blocks (p = q = 3), in each block the same components can be applied, or may also be different, according to their entropy calculation result. Both adaptive color space and macro block models have been developed in [14,15] and implemented in 4: 4: 4 video coding.
The first three color spaces in table 1, Y 1 UV 1 , Y 2 UV 2 and Y 3 UV 3 (each for r = i = j is called YUV r : YUV 1 , YUV 2 , and YUV 3 ), can be represented as matrix multiplication and each accompanied by its inverse transformation, respectively shown by Eqs. (9,10,11,12,13,14). The YUV 1 color space is the same as the YCgC o -R color space, while YUV 2 and YUV 3 are the permutations of the YUV 1 color space. These three color spaces can be applied for lossless and lossy image compression.

Performance evaluation of existing color spaces
In terms of numerical implementation, it can be concluded that YUV 1 (= YC o Cg-R), YUV 2 and YUV 3 are simpler and faster than YC o Cg, YD b D r and YC b C r color spaces. So, what about the performance in terms of compression ratio and image quality, if each color space is implemented in the lossy image compression algorithm?
In this part, we describe performance evaluation of color spaces YC b C r , YD b D r , YC o Cg, YUV 1 , YUV 2 and YUV 3 , applied in JPEG compression algorithm. As known that image quality and compression ratio of color image compression algorithms are not only determined by the diversity of color contained in an image, but also influenced by texture and shape variations. For these reasons, in the experiments, 60 images were selected containing various colors, shapes and textures from six sources of image dataset [16,17,18,19,20,21]. Twelve of them are shown in fig. 2. For example, the Lena image has the dominant colors red, orange and red-purple, many homogeneous areas, slight in texture and shape variation. The Baboon image has random textures and shapes, and its dominant colors are green-yellow, cyan and red. The Frymire image is very rich in textures and shapes, as well as containing variant colors with almost the same composition between yellow, red, orange, green, cyan, blue and purple. The Roses image has dominant colors red, redpurple (pink), blue-cyan, blue-purple, yellow, white and black. The Woman image has a little color variation, some areas are relatively homogeneous and varying texture patterns in the hair. Yellow_orchid has a dominant homogeneous area, yellow and cyan colors, as well as various shapes and slight variations in texture.

Fig. 2. Twelve test images: (a) Lena, (b) Baboon, (c) Peppers, (d) Frymire, (e) Roses, (f) Tulips, (g) Sails, (h) Monarch, (i) Textile, (j) Woman, (k) yellow_orchid, and (l) ucid00295
Next, we outline the performance comparison of six color space YC b C r , YD b D r , YC o Cg, YUV 1 , YUV 2 and YUV 3 , regarding their compression ratio and image quality, when each of them is used in the lossy image compression algorithm. Block size 8 × 8 pixels [1], color components without sub-sampling for high quality image compression [23,24,25], discrete cosine transform (8 × 8 DCT and iDCT) [1,26,27,28,29], and Huffman statistic encoding / decoding [1,2,8,22,30] are chosen to carryout the experiments of JPEG lossy image compression. For each color space used, it is done by determining the same quality for the same image and then compares their compression ratio. In this case, the experiment was carried out by using one Photoshop's quantization matrix for medium quality (Q 9 ) as shown in Eq. (15) [3,22]. Q Lum and Q Cr are respectively quantization matrices for luminance and chrominance components, and q is a variable that determines the quantization value, so that the seven color spaces can produce the same quality for the same image, thus the compression ratio can be compared. Computer Optics, 2021, Vol. 45(3) DOI: 10.18287/2412-6179-CO-780 4 6 12 22 20 20 17 17   6 8 12 14 14 12 12 12   12 12 14 14 12 12 12 12   22 14 14 12 12 12 12 12   20 14 12 12 12 12 12 12   20 12 12 12 12 12 12 12   17 12 12 12 12 12 12 12   17 12 12 12 12 12 12 12 .   Table 2 shows the compression ratio (CR) of JPEG algorithm using color spaces: Ucid00295 images compared to the five others. YC b C r results in better CR for Roses and Tulips images, while Yellow_Orchid image has highest CR given by YUV 2 . In the two last rows of table 2, the CR average and the performance produced by each color space is given. When referring to these CR average values and comparing them to the CR average value of JPEG-standard YC b C r color space, YC o Cg provides better performance with increased CR of 3.8 % on average, while YD b D r , YUV 1 , YUV 2 and YUV 3 resulted CR averages of 1.6 %, 2.1 %, 2.1 %, and 2.5 % lower than YC b C r respectively.
The experimental results of the other forty-eight images are given in tables 4 -5. Table 4 represents the compression ratios for the same image quality and table 5 reflects the image quality for the same compression ratio. For now, we focus on the first seven columns to evaluate the compression results of the forty-eight images using YC b Cr, YC o Cg, YUV 1 , YUV 2 and YUV 3 color spaces. There are thirty-seven images (75.51 %) that have the highest compression ratio and image quality generated by the YC o Cg color space (in the fourth column), nine images (18.37 %) given by the YC b Cr color space (in the third column), and the remaining two, one and zero images respectively resulted by YUV 1 , YUV 2 and YUV 3 color spaces (in the fifth, sixth and seventh column). In the last two rows of each table, the CR average and the performance yielded by each color space is given. YC o Cg color space has the highest average value of 9.599, followed by YC b Cr = 9.252, YD b Dr = 9.101, YUV 2 = 9.059, YUV 1 = 9.055 and YUV 2 = 9.019. These CR average values show that for those forty-eight images, the use of YC o Cg in the JPEG lossy compression algorithm can increase the CR average of 3.623 %, while for YUV 1 , YUV 2 and YUV 3 there is a decrease of 3.158 %, 5.056 % and 4.693 % respectively.
These results indicate that in general YC o Cg color space provides better performance than the five other color spaces and YC b C r has a higher compression ratio than YUV 1 , YUV 2 , YUV 3 and YD b D r color spaces. It may be assumed that the compatibility between the percentage composition of each color in a color space formula and the dominant colors contained in an image can affect the quality and compression ratio. The first analysis is about the percentage of color representation in a color space formula. For example, a pixel has pure red or pure blue, in YC o Cg color space, its color will be accommodated 100 % (25 % in Y, 50 % in C o and 25 % in Cg components), while in YC b C r , YD b D r and YCgCo-R family (YUV 1 , YUV 2 , YUV 3 ) color spaces respectively accommodate 97.775 %, 125 % and 175 %. Furthermore, for pure green, it will be recorded 100 %, 250 %, 150 % and 133.695 % respectively by YC o Cg, YC b C r , YD b D r and YC o Cg-R. Ideally to get good quality and compression ratio, the color conversion formula must still record 100 % of the dynamic value for each color component R, G and B. This ideal condition is only owned by YC o Cg color space and it is proven that its performance is better than the three other color spaces. The second analysis con- The third analysis is the compatibility between the color space used and the dominant colors contained in an image. Based on the three thoughts above, this paper proposes a color dominant analysis algorithm and an adaptive color space model.

Proposed methods
The proposed methods consist of three parts: developing adaptive color space, analyzing dominant colors in an image to determine which color space will be used, and then the experimentations. The development of adaptive color space model refers to YC o Cg color space which provides better quality and compression ratio in lossy image compression (see results in table 1). It has been described above that this color space has not evenly covered all existing chrominance. This weakness has an impact on quality and compression ratio: it is best suited for images having colors according to their characteristics and decreased for images having less suitable color.

Dominant colors analysis
As described in the example of twelve images ( fig. 2), where each image has a different dominant color. Thus, there is a need to adapt the suitability of the color space used and the image to be compressed. For this reason, the dominant color analysis contained in the image must be carried out first. The colors information can be presented in the form of cylindrical coordinates hue-saturation as shown in fig. 3a. We propose to divide hue coordinates into 12 parts, so that 12 different hues will be formed with angle range: red = 0º -15º and 345º -360º, orange = 15º -45º, yellow = 45º -75º, green-yellow = 75º -105º, green = 105º -135º, greencyan = 135º -165º, cyan = 165º -195º, blue-cyan = 195º -225º, blue = 225º -255º, blue-purple = 255º -285º, purple = 285º -315º, red-purple = 315º -345º. The Hue of each pixel of an RGB image can be calculated using H (hue) component formula of HSL, HSV or HCL color spaces [31,32]. From the hue value of each pixel, a histogram hue can be calculated. A Histo-gram hue is a distribution of the number of pixels having the same hue value in an image. Furthermore, this histogram hue can be used to determine what chrominance are dominants in an image. figs. 3b -d respective-ly show histogram hues for Lena, Roses, and Textile images. The dominant hue in Lena image is red, orange, and red-purple. These three hues are more in line with YC o Cg that is why its compression ratio performance is 5.63 % better than YC b C r (see table 2). Roses image has dominant hues: red, red-purple, blue-cyan, blue-purple, yellow, white, and black. The majority of these hues are less compatible with YC o Cg, this has an impact on decreasing its compression ratio performance to 1.27% lower than YC b C r . For Textile images, its dominant hues are yellow, orange, cyan, blue, red-purple. For this Textile image, YC o Cg and YC b C r provide a relatively similar compression ratio.
The proposed algorithm for the dominant colors analysis process in an image is given in Algo-1. The syntax of Hue_histo(red), Hue_histo(cyan), and so on represent the number of pixels in an image that have the red hues (0º -15º and 345º -360º), cyan hues (165º -195º), and so forth. The dominant colors are grouped into three parts according to the three color space models described in section 3.2. The first part (Model-1) is a combination of red, cyan, green-yellow, blue-purple, and gray-level colors. The second part (Model-2) is a combination of green, purple, orange, blue-cyan, and gray-level colors. The last part (Model-3) is a combination of blue, yellow, green-cyan, red-purple, and gray-level colors.

Proposed adaptive color space model
In this part, we adopted the permutation model proposed in [13] to produce two additional color spaces which are variants of YC o Cg and to be a solution to its weaknesses. The two color spaces are shown in Eqs. (16,17), and Eqs. (20,21). Thus, the proposed adaptive color space model (ACSM) consists of three color spaces and can be said as the YC o Cg color space family (ACSM-YC o Cg). As a side note, if each of their all chrominance components is multiplied by 2, they will be identical to the YCgCo-R color space family (YUV 1 , YUV 2 and YUV 3 ).
Model-1: YC c C r (luminance Y, chrominance cyan C c and chrominance red C r ) with dominant hues (colors) coverage: red, cyan, green-yellow, blue-purple, and grey-level.
Model-3: YCyC b (luminance Y, chrominance yellow Cy and chrominance blue C b ) with dominant hues coverage: blue, yellow, green-cyan, red-purple, and grey-level.
The color image conversion algorithm from RGB space to ACSM and its inverse conversion from ACSM to RGB are presented by Algo-2 and Algo-3. The choice of color s pace (Color_Space), for the color image conversion process, depends on the dominant colors calculated using Algo-1. If the Color_Space = "Model_1", then the color space YC c C r will be used, or the color space YCpCg that will be selected if Color_Space = "Model_2", whereas if Color_Space = "Model_3", then the color space YCyC b will be used. The color image conversion process from RGB_Image to ACSM_ Image is done in the last line of the algorithm. This conversion process is carried out pixel by pixel. The Algo-3 is used for the inverse color conversion process from ACSM_ Image to RGB_Image.

Results and discussions
The proposed dominant color analysis and ACSM algorithms have been integrated into the JPEG lossy image compression algorithm. Besides that, we also implemented the ACSM algorithm using YCgCo-R color space family (YUV 1 , YUV 2 and YUV 3 ). Both were tested using sixty images from [17,18,19,20,21]. The experimental results of the first twelve images are given in table 3 and the next forty-eight images are shown in tables 4 -5. In table 3, there are eleven of the twelve images with the highest CR value offered by the ACSM-YC o Cg color space family (last column), while the ACSM-YCgCo-R family only produces one image with the highest CR value (fourth column).
For Lena image, the adaptive color algorithm automatically selects and uses model-1 YC c C r color space, and obtained CR of 10.34 % better than YC b C r . Color space YUV 3 is also automatically selected when ACSM-YCgCo-R family is implemented. This color space is 4.362 % lower than YC b C r . Color space Model-1 is also selected for Roses and Ucid00295 images and their CR are better than YC b C r and ACSM-YCgCo-R family. An increase in CR is also seen in Peppers, Frymire, Tulips, and Textile images, where the color space chosen is model-3 YCyC b . Furthermore, for Baboon, Sails, Monarch, and Woman images, model-2 YCpCg color space is used, where the compression ratios results are also better than YC b C r and ACSM-YCgCo-R family. The higher CR for Yel-low_Orchid image is given by YUV 3 of ACSM-YCgCo-R family. Its compression ratio is 12.77 % better than YC b C r and 2.607 % better than the ACSM-YC o Cg family.
Furthermore, in tables 4 -5, it can be observed that there are forty-one out of forty-eight images having the highest quality and compression ratio produced by the proposed ACSM-YC o Cg family. YC b C r and ACSM-YCgCo-R family provide the highest quality and compression ratio of the other four and three images, respectively. On average, ACSM-YC o Cg family can enhance the CR by 5,046 % for the first 12 images, while for the next 48 images the CR and compression quality increase 4,555 % and 0.303 dB comparatively to YC b C r . Conversely, there was a decrease in CR of 0.420 % for the first 12 images and a decrease in CR of 2.087 % and compression quantity of 0.439 dB for the next 48 images, when ACSM-YCgCo-R family was used. Furthermore, the adaptive color space performance test is performed using seven Photoshop's quantization matrices from the high Q 12 to the low Q 6 compression quality (i.e. Q 12 , Q 11 , Q 10 , Q 9 , Q 8 , Q 7 , Q 6 ) [22,30]. Tables 6 -8 show the quality (PSNR) and compression ratio for Lena, Baboon and Textile images and graphically show by the curves in Figs. 4 -6. The adaptive color space curves for the three images are above the YC b C r color space curve. This shows that the proposed adaptive model has a better performance.   Besides its better performance, the implementation of the proposed adaptive color space in the form of an electronic circuit is very simple and only requires adders and subtractors. It is as simple as the electronic implementation of YCgCo-R color space family [13]. Suppose each color data R, G and B of an image is encoded with eight bits, that is: R = r 7 , ..., r 0 ; G = g 7 , ..., g 0 ; and B = b 7 , ..., b 0 . Arithmetic operations for these data are carried out in binary operations: addition, subtraction, and logic. The symbol "»1", is a onebit shift-right operation, used instead of multiply by 0.5. For example, the Eq. (10) can be simplified as: Y = 0.5R + 0.25G + 0.25B → Y = (R + ((G + B) »1)) »1, C c = 0.5G -0.5B → C c = (G-B)»1, C r = 0.5R -0.25G -0.25B → C r = (R -((G + B) »1)) »1.
Referring to this simplification model, the electronic schematic of RGB to YC c C r conversion can be created as shown in Fig. 7. This scheme only requires 2 adders and 2 subtractors. The luminance Y component requires 10 bits, consisting of 8 bits for integer (s 7 , ..., s 0 ) value and 2 bits for fractional (s -1 , s -2 ) value. While the cyan chrominance C c component needs 9 bits, that are 1 bit of borrow-out (s 7 as sign bit: "0" positive or "1" negative), 7 bits integer value (s 6 , ..., s 0 ), and 1 bit fractional (s -1 ). Furthermore, the red chrominance C r component takes 10 bits consisting of 1 bit borrow-out (s 7 as sign bit), 7 bits inte ger (s 6 , ..., s 0 ), and 2 bits (s -1 , s -2 ) fractional. The inverse conversion process from YC c C r to RGB in Eq. (11): R = Y + C c , G = (Y -C r ) + C c , B = (Y -C r ) -C c . The electronic schematic of this equation is given by fig. 8. It can be seen that this inverse conversion only needs 2 adders and 2 subtractors.
The electronic schematics diagrams of both RGB to YCyC b and YCyC b to RGB colors conversion are sequentially presented in figs. 11 -12.

Conclusion
In this paper, three color spaces YC c C r , YCpCg, and YCyC b (YCoCg color space family) have been developed that can be adapted according to the dominant colors contained in an image. The color analysis algorithm can calculate and determine the dominant colors in an image, and then automatically determines the appropriate color space to be applied in that image. The experimental results using sixty test images, which have varying colors, shapes, and textures, indicate that the proposed adaptive color spaces model provides improved performance of 3 % to 10 % better than other color spaces. Likewise, their electronic schematics require only two adders and two subtractors, both for forward and inverse conversions.
Our future research is to apply the dominant color analysis and adaptive color space model (ACSM) algorithm to the JPEG2000 image compression algorithm, and MPEG for video compression.