Enhanced Color Image Encryption Utilizing a Novel Vigenere Method with Pseudorandom Affine Functions

Ciphering images is the process of protecting them by altering their pixels in a way that makes them indecipherable. It protects image confidentiality and integrity, particularly when the images are sensitive and confidential, as in the military case [1] and medical images [2], [3]. Perturbations in ciphering steps complicate the statistical relationships between original and ciphered images and make predicting them difficult. Diffusion, on the other hand, distributes data in its initial form efficiently and uniformly throughout the entire ciphered image. Encryption methods achieve diffusion and confusion [4], [5] through dense substitution and permutation of each pixel. Substitution is performed by changing the image pixel values to other values. Permutation randomly arranges image pixels [6] to conceal the statistical relationships between the image pixels. Various methods can be used for replacement, such as the substitution box (S-box) operation [7], [8], [9]. As the patterns of replacement and permutations become more difficult, the situation becomes more unpredictable and complex. Therefore, a combination of replacement, which integrates dynamic affine functions, and permutations must be applied at the smallest unit of images, which is the pixel.

The most important component of the ciphering process is the key, which plays a vital role in protection and ciphering data confidentiality. Hence, the key must satisfy various criteria, encompassing considerations such as length, space, and complexity [10], [11]. This implies that the user-inputted key for image encryption needs preprocessing to generate a more intricate form, such as a pseudorandom sequence [12]. However, these keys can also be changed by simply performing standard operations, such as rearranging the image stream [13]. In the current context of modern image encryption, the use of keys can produce real sequences of chaotic behavior that are sensitive to initial conditions, thus providing a high level of security [14]. Chaotic methods have various variations, some of which are renowned for image ciphering, such as piecewise linear chaotic maps (PWLCMs), logistic, Henon, and skew tent maps [15], [16], [17], [18]. Focusing on the logistic and skew tent maps, they present several advantages, such as improving randomness and sensitivity, leading to more chaotic, unique, and secure sequences.

Most of the algorithms mentioned above use independent block encryption. Consequently, all these techniques remain vulnerable to statistical attacks. Moreover, the small size of their private keys exposes them to brute-force attacks. Additionally, in the absence of any diffusion or chaining function between the encrypted blocks and plaintext blocks, these techniques remain susceptible to differential attacks.

This study aims to develop a new image encryption cryptosystem using a large private key to guard against brute-force attacks. Additionally, diffusion and confusion functions are employed to ensure that the cryptosystem remains beyond the reach of differential attacks, placing the new technique out of the scope of any such attacks. Similarly, the integration of multiple pseudo-random vectors in the confusion process ensures strong protection against statistical attacks.

The rest of this study is divided into various sections, namely, a section on previous related work, detailing the assumptions and related research; a section describing the theoretical framework, explaining the basis of chaotic sequences, as well as classical Vigenere and affine techniques; a section detailing the proposed approach, revealing the nuances of the encryption and decryption processes; a section devoted to results and discussions, presenting the research findings, discussions, and comparisons with other similar techniques; and a section summarizing the findings and proposing research directions.

The prior sections explored the potential significance of confusion and diffusion features in image ciphering, applied across substitution and permutation procedures. Another aspect to highlight concerns the ciphering key quality. Research in the prior research on the design of methods that exploit permutation and substitution patterns, as well as on improving key qualities, has been a source of motivation for this study. Initially, substitution and permutation ciphering methods were applied exclusively to pixels. Recent research, such as the study of Sabir and Guleria [19], highlights the increasing popularity of pixel-level substitutions and permutations as image processing techniques. Zhang and Tian [20] presented a multiple image ciphering approach relying on 3D bit planes and a genetic central dogma to perform a permutation between bit planes. Similarly, Ramasamy et al. [21] proposed an improved logistic map that included diffusion, key stream generation, and permutation to resist assaults. Wu et al. [22] presented a method based on the deoxyribonucleic acid (DNA) genomic sequence permutation operation that coupled one-way mapping networks with an XOR operation. Butt et al. [23] introduced a new image ciphering method using binary bit plane scrambling and the SPD bit plane diffusion technique for an ordinary image, incorporating elements of the card game technique. Li [24] investigated the resilience of HCIE when faced with a known-plaintext-only attack and a known-plaintext/chosen-plaintext attack. Khan et al. [25] proposed a new DNA-based method capable of performing the simultaneous functions of bit plane searching and pixel diffusion in a single step. Zhang et al. [26] proposed an improved algorithm aimed at eliminating potential security issues in Liu's algorithm. However, Wang and Zhang [27] proposed a novel approach to color image encryption that utilizes a combination of heterogeneous bit permutation and correlated chaos techniques. A new bit-level image ciphering method proposed by Xu et al. [28] is based on PWLCMs. First, the ordinary image is transformed into two binary sequences of the same size. Second, a new diffusion strategy is introduced to mutually diffuse the two sequences. Niyat et al. [29] proposed an image ciphering method based on a self-organized structure with a set of units updated according to rules that depend on the number of limited neighboring units. Chen et al. [30] presented a solution for protected and efficient image ciphering using adaptive permutation-diffusion and random DNA coding, where the permutation and diffusion procedures are perturbed by the intrinsic characteristics of the plaintext. However, Ye and Huang [31] proposed an efficient symmetric image ciphering technique to resolve the low-sensitivity issue of an ordinary image. Recently, Chen et al. [32] developed an enhanced digital image ciphering method using a collage model and a one-dimensional quadratic chaotic system, thus expanding the key space that offers robust resilience against exhaustive attacks. Wang et al. [33] proposed an improved 3D chaotic system characterized by various dynamic behaviors, such as phase shifts and expansion and contraction phenomena that vary with parameter changes. The proposed model covers a wide chaotic range and demonstrates its applicability in image encryption processes. Hence, a color image ciphering method based on a hypercomplex chaotic system and a skew tent map was proposed by Kadir et al. [34]. Wu et al. [35] proposed a new color image ciphering method based on DNA, one-time keys, spatiotemporal chaos and sequence operations. Liu et al. [36] proposed a quantum image ciphering scheme based on permutation using an improved quantum representation model and realized consecutive intraputation by sorting an interpermutation and chaotic sequence operations involving the qubit XOR process between chosen bit planes. However, Winarno et al. [37] proposed a combination of several intertwined patterns, incorporating zigzag, Hilbert, and Morton patterns, to aggregate confusion-diffusion and improve complexity and randomness. Thus, Aung et al. [38] proposed a poly-alphabetic cipher that constitutes a new system for encrypting and decrypting data.

The mentioned systems generally use small-size substitution tables, namely 16×16, which constitute a simple permutation that does not influence the statistical distribution of pixels. This leads to vulnerability to statistical attacks. Additionally, the replacement functions are defined by simple analytical expressions. Similarly, techniques using genetic algorithms operating at the DNA level typically exhibit a static notation, such as the conversion from Z/4Z to DNA.

The new trends in the field of symmetric cryptography involve the use of genetic algorithms through the application of specific genetic operators adapted to the encryption of voluminous data, typically acting at the level of DNA and ribonucleic acid (RNA). On the other hand, researchers are attempting to utilize Euclidean networks for symmetric encryption of textual data.

The contribution of this study lies in overcoming all anomalies indicated in prior research and developing a new image encryption cryptosystem using two large substitution tables of sizes (256; 256), accompanied by dynamic affine functions for diffusion restoration. Furthermore, the size of the private key in the cryptosystem far exceeds 100 bits, placing the new technique beyond the reach of any differential attacks.

This section introduces the chaotic maps, classical Vigenere, and affine ciphering methods used in this work.

This map [28] is a one-dimensional map expressed by Eq. (1).

where, $h_0$ and $k$ represent the initial state and its control parameter, respectively.

The second chaotic sequence is generated by the logistic map [19]. It is a simple polynomial-degree recurrent sequence defined by Eq. (2).

The classical Vigenere cipher system is based on a matrix (Vt) of fixed dimensions (26, 26) reserved for text encryption only. It is defined by Algorithm 1.

Let $Pk$ be the plain message, $Ck$ be the cipher message, $Ke$ be the encryption key, $Vt$ be the Vigenere matrix, and $n$ be the length of the plain message. The encryption and decryption algorithms associated with the classical Vigenere method are given in Algorithm 2.

The classical Vigenere method uses a small 26×26 substitution table, whose static and public default makes the method easier to attack. Similarly, the simple analytical expression of the classical Vigenere substitution function is easily reconstructed.

Let $f$ be an affine function defined in the ring $Z/nZ$ by Eq. (3).

The function $f$ is bijective in ($Z/nZ$) if and only if ($a$) is invertible and ($b$) is any.

Indeed, $y=\bmod (a x+b ; n)$ is obtained.

Then, $a x=\bmod (y-b ; n)$ and $x=\bmod \left(a^{-1} \cdot(y-b) ; n\right)$, where, $a^{-1}$ is the inverse of $a$ in ring $Z / n Z$. Or, $a$ is invertible in $(Z / n Z)$ if and only if $\mathrm{a} \wedge \mathrm{n}=1$.

Particular case:

$n=2^k, k \in N$ Particular case, then $a$ is invertible in ring $Z / 2^k Z$ if and only if $a$ is odd.

Example in a ring Z/16Z: Table 1 represents an example of affine functions in ring Z/16Z.

The classical affine transformation is static and thus remains vulnerable to any brute-force attack. The shortcomings of traditional Vigenère and affine systems have been previously outlined in this study. To overcome all these flaws, substitution tables of large size are used, specifically 256×256, generated from pseudorandom vectors employing new replacement and diffusion functions involving more than one S-box. Similarly, the affine functions used are dynamic and pseudorandom.

This new technique uses the two most widely deployed chaotic maps in the field of cryptography [18], [19] by integrating large S-boxes and strong affine diffusion functions [20], [39]. This technique is structured around the axes described below.

Two chaotic sequences h and l that are extremely sensitive to the initial conditions and easy to implement in any cryptosystem are used in this approach, as described above.

Seven pseudorandom vectors $V c 1, V c 2, V c 3, V r, V e, V a$, and $V b$ with coefficients in the ring $Z / 256 Z$ are generated by Algorithm 3 below.

The two vectors $V a$ and $V b$ contain only the invertible elements in the ring Z/256Z. In addition, the system requires the generation of three binary vectors, $B a 1, B a 2$ and $B a 3$, to control the encryption process. These two vectors are generated by Algorithm 4.

The algorithm requires the development of two new replacement tables Tv1 and Tv2, each of size (256; 256) and with coefficients in the ring Z/256Z.

The main mission of this section is to construct the new Vigenere substitution matrix, called Tv1, with a size of (256; 256), following the instructions provided below.

• The first row of the table $T v 1$ is the permutation $P t 1$ of the first 256 values of the vector $V c 1$, obtained by sorting them in decreasing order.

• For ranks higher than 1, the rank line is a rank shift $V c 2(i)$ or $V c 3(i)$, depending on the value of the control vector $B a 1(i)$. This table was generated by Algorithm 5.

Example:

Creation of the first line

Table 2 represents an example of generation of s-box Tv1 of size (8; 8) controlled by Ba1.

The construction of the new substitution matrix Tv2 of size (256; 256) is described by the following steps:

• The first line is the rearrangement $\operatorname{Pr} 1$ obtained by a broad ascending sort on the first 256 values of the vector $V c 3$;

• The second line is the rearrangement $\operatorname{Pr} 2$ obtained by a broad ascending sort on the first 256 values of the vector $V c 2$;

• The third line is the rearrangement $\operatorname{Pr} 3$ obtained by a broad ascending sort on the first 256 values of the vector $V c 1$;

• The $i$-th line $(i>3)$ is the composition of the functions of line $(i-2)$ and $(i-3)$ or $(i-3)$ and $(i-1)$, depending on the value of the control vector $\mathrm{Ba2}(i)$.

These steps are illustrated in Algorithm 6 below.

Example: Tv2 creation of size (8 ; 8) controlled by Ba2 as shown in Table 3.

Let $f_i$ be the family of affine functions acting on the pixels. These functions are defined by Eq. (4).

Since the elements $V a(i)$ and $V b(i)$ are invertible in ring $Z/256Z$, the functions $f_i$ are reversible for all $i \in[ 1 ; 3 \mathrm{~nm}]$.

The new substitution function involving tables Tv1 and Tv2 is given by Algorithm 7.

The encryption phase unfolds through the subsequent stages:

This phase involves uploading the original image of dimensions (n,m) and then extracting the Red, Green and Blue (RGB) channel vectors Cr, Cg and Cb, which are concatenated under the control of the binary vector Ba1 into a single vector X of dimensions (1,3nm), as illustrated in Figure 1 below.

The mathematical formulation of this phase is described in Algorithm 8.

This improved Vigenere lap starts by calculating the initialization value In, which is closely linked to the plain image and is intended to change the value of the starting pixel and launch the encryption phase. This value is calculated by Algorithm 9 below.

This algorithm is illustrated in Figure 2 below.

To overcome any differential attack, a diffusion round is first performed using the chaotic confusion vectors and a chaining between the ciphered pixels and the following plain pixels using the bijective affine functions. The diffusion process is illustrated by Algorithm 10.

This algorithm can be interpreted in Figure 3.

To augment the attack complexity of the proposed system, the vector $Z$ is subjected to a global rearrangement $Pg$ process involving the first (3nm) values of the chaotic sequence $l$. This permutation phase is determined by the following Algorithm 11:

The image generated by this new Vigenere-affine hybrid method is represented by the vector Zs.

Histograms of cipher images represented by the vector Zs is given by the Figure 4.

The suggested encryption system is symmetric and uses two diffusion functions, which require the decryption process to start with the last intervention using the inverse functions. The encrypted image is transformed into a vector Zs with dimensions (1; 3nm) on which the following steps are executed:

• Application of the inverse permutation;

• Inverse of the affine function;

• Inverse of the improved Vigenere-affine function;

• Inverse of the diffusion function.

The decryption process is illustrated in Figure 5.

All simulations were executed using Python on the Windows 10 operating system, employing a hardware setup comprising an i7 processor laptop with a 1 TB hard drive and 32 GB of RAM. Figure 6 displays the main test image, “Lena,” along with its encrypted and decrypted versions, as well as all plain images utilized. These image samples were sourced from the SIPI database (https://sipi.usc.edu/database/). The keys and various experimental parameters are generated using the previously described chaotic maps. Before commencing the decryption process, it is imperative to securely transmit the secret key to the recipient through a protected channel.

The new algorithm was applied to a random selection of reference images, and the resulting simulations were documented as follows:

The system employs two chaotic maps generated using four real parameters, each represented by 32 bits, resulting in a key comprising 120 bits. This design guarantees resistance against brute-force attacks.

The system employs two extensively employed chaotic maps within the realm of cryptography. Their heightened sensitivity to initial conditions ensures a pronounced responsiveness to the encryption key, as illustrated in Figure 7.

As depicted in Figure 7, altering the encryption key results in the generation of two distinct encrypted images during the encryption process. Furthermore, the decryption stage produces two decrypted images with entirely distinct shapes, highlighting the proposed algorithm's heightened sensitivity to even minor changes in the encryption key.

The encrypted image, as depicted in Figure 8, exhibits a distinct visual dissimilarity from the original image. Furthermore, the algorithm ensures uniform distribution in the histograms of encrypted images, affirming robust protection against statistical attacks.

Histograms depict the distribution of pixel values in images. Unauthorized individuals could extract crucial details about an encrypted image by analyzing its irregular histogram. Therefore, it is crucial to ensure that encrypted image histograms exhibit no numerical similarity with the original image, and maintain a consistent pixel distribution. Figure 9, Figure 10, Figure 11, Figure 12 and Figure 13 showcase histograms of all test images, while Figure 14 displays histograms of images generated through encryption for each corresponding test image. Figure 9 specifically illustrates the histogram of the main test image “Lena.” Figure 9, Figure 10, Figure 11, and Figure 12 represent RGB channel histograms of clear images, whereas Subgraphs (a)-(j) of Figure 14 portray RGB channel histograms of encrypted images using the proposed method. Notably, the histograms of images produced during the encryption stage are characterized by a nearly uniform and flat distribution.

Likewise, by calculating the variances of histograms using Eq. (5), the consistency of encrypted images was examined. A reduced variance in an encrypted image signifies increased uniformity and a heightened level of protection for the suggested image encryption method [21], [22].

where, $X=x 1, x 2, \ldots, x 256$ represents the histogram value vectors, and $x_R$ and $x_c$ denote pixels with $p$ and $c$ representing gray levels, respectively.

Table 4 presents the observed variations in the chosen test images. Examination of the data in the table reveals markedly elevated variances in the unencrypted images, in contrast to significantly lower variances in the encrypted versions. Specifically, the mean variance for the encrypted “Lena” image was 223.667, in comparison to 81232.023 for its plaintext counterpart. A comparative analysis further indicates that, for most of the tested images, the histogram variances in encrypted images generated using the proposed method consistently surpassed those obtained by several researchers [23], [24], [25]. This comparative evidence supports the claim that the proposed algorithm has an improved capability to enhance the security of the encryption process.

The entropy of an image with dimensions (n, m) functions as a measure to evaluate the security of the image encryption algorithm. It signifies the degree of unpredictability and randomness inherent in the system, as expressed by the Eq. (6) provided.

where, $p(i)$ represents the probability of the occurrence of level $i$ in the plain image.

Therefore, as the entropy approaches Eq. (8), the random arrangement of pixels in the image becomes more refined. Increased entropy contributes to minimizing information leakage from the encrypted image.

Table 5 displays entropy values for the examined images, compared to those of various existing encryption methods. The results indicate that the entropy values are consistently equal to or greater than 7.996, similar to the study of Khan et al. [25], and outperforming values reported by several researchers [23], [33], [34], [35].

Eq. (7) provides the correlation of an image with dimensions (n, m).

Table 6 displays the pixel correlation values of images obtained from the SIPI database and additional test images. Examination of the table data indicates a remarkably high correlation in the original image, approaching a value of 1 for each channel. In contrast, images encrypted with the proposed algorithm exhibit a significantly lower correlation. This observation supports the robust security achieved by the algorithm. Moreover, these results highlight a considerable decrease in correlation within the encrypted image, indicating that attackers cannot extract information from the encrypted image using this method.

Table 7 displays the pixel correlations detected within the 'Lena' image. A juxtaposition with previous techniques reveals that the mutual correlation of adjacent pixels in the encrypted image is lower than that in several studies [25], [26], although it is similar to that in the study of Butt et al. [23]. The correlation metrics for all tested image correlations by the proposed encryption system are nearly zero, providing protection against statistical attacks.

To evaluate the effectiveness of the algorithm against differential attacks, metrics like the number of pixel change rate (NPCR), the unified average change intensity (UACI), and the avalanche effect are utilized.

These metrics can be given by Eqs. (8) and (9) below.

where, $D f(i, j)=\left\{\begin{array}{lll}1 & \text { if } \quad {Im}_1(i, j) \neq {Im}_2(i, j) \\ 0 & \text { if } \quad {Im}_1(i, j)={Im}_2(i, j)\end{array}, {Im}_1(i, j)\right.$ is the first image pixel of rank $(i, j)$ and ${Im}_2(i, j)$ is the second image pixel of rank $(i, j)$.

The information in Table 8 reveals UACI and NPCR values for two images, “Lena” and “Pepper”. It illustrates that the UACI is equal to or exceeds 33.4, and the NPCR is equal to or exceeds 99.6. These results unequivocally demonstrate that the NPCR performance of the suggested encryption algorithm is on par with that of the algorithm proposed by Butt et al. [23] and surpasses that of several other algorithms [25], [29], [30], [31], [32]. Meanwhile, the UACI value aligns closely with that of several algorithms [23] [25], [29], [30], [31], [32]. This indicates that the proposed approach showcases robust encryption performance, particularly in thwarting differential attacks.

In the field of image processing, the mean squared error (MSE) and PSNR are widely used to assess the effectiveness of encryption. These metrics serve as common criteria for evaluating the quality of two images in a cryptographic system. The PSNR gauges the similarity between images and complements the MSE. Eq. (10) can be applied to calculate the MSE for the original, decrypted, and encrypted images.

where, $Im_1$ and $Im_2$ represent the original and encrypted images, respectively.

The symbol $n$ indicates the number of rows in the original image, while m signifies the number of columns in the image. The evaluation of $PSNR$ is conducted in decibels and exhibits an inverse relationship with the mean squared error, as determined by Eq. (11).

where, $L$=8 denotes the bit depth of the particular image.

A higher PSNR indicates a smaller difference between the original and decrypted images. When the original and encrypted images are identical, the PSNR becomes infinite. The table below presents a comparison of PSNR values between the proposed approach and other references.

The lower PSNR values observed in Table 9 for the original-to-encrypted images suggest that the proposed algorithm offers superior encryption compared to the methods in several studies [23], [36], [37]. The metric's value is comparable to that in the study of Aung et al. [38], indicating that the proposed method can recover images without substantial information loss.

The method has several benefits, namely:

• Difficulty to recover encryption keys due to the extreme sensitivity of the used chaotic maps to the initial conditions.

• Robustness of the method due to non-commutative algebraic operations.

• The system can be applied to any image of arbitrary format and size.

• Use of pseudo-random and bijective affine functions.

• High attack complexity of the method due to the replacement S-box design under a chaotic decision control vector.

• Difficulty of S-box reconstruction due to their pseudorandom vector-based construction.

The effectiveness of the approach is predominantly influenced by the constraints imposed by the selection of chaotic maps, the design of S-boxes, and the pseudo-random characteristics of the generated S-boxes.

A hybrid circuit combining the affine and Vigenere techniques was applied, employing two novel substitution tables and a robust diffusion and confusion function, followed by a global permutation tailored for image encryption. Additionally, the incorporation of highly sensitive chaotic maps into the initial conditions allowed us to suggest an improved method for ciphering color images. Simulations conducted on a randomly selected sample of images from the SIPI database, featuring various formats and sizes, demonstrated the algorithm's ability to withstand any known attacks.

As a perspective, some algorithms will be integrated with the proposed method, such as wavelet transformations, reinforcement learning, supervised learning, and fuzzy methods.