Text Readability Evaluation in Higher Education Using CNNs

English, often regarded as the universal language, exerts a significant influence on higher education, particularly in countries where English is not the native language. Its role transcends national borders, providing students from diverse linguistic backgrounds with access to a plethora of global academic resources. The widespread use of English in scholarly journals, textbooks, and academic conferences necessitates its mastery by students and researchers for engaging with contemporary research and academic discourse. Moreover, numerous universities in non-native English-speaking countries offer English-taught programs, attracting international students and fostering intercultural collaboration. This not only enhances educational experiences and aids in developing a global outlook but also improves professional prospects in an increasingly interconnected world [1], [2].

In the realm of text readability, traditional methods such as the Flesch Reading Ease, Gunning FOG, and Automated Readability Index were employed. However, these formulae provided less precise assessments as they merely skimmed the surface of word features, lacking in-depth understanding of word meanings. The advent of DL models has revolutionized text readability assessment by automating the process and obviating the need for manual word feature configuration. The efficiency and accuracy of automatic word feature extraction via DL models represent a significant advancement, reducing the reliance on human intervention. Notably, the development of the Ranked Sentence Readability Score method, a deep neural network-based approach capable of processing multilingual texts, has been a notable contribution by international researchers [3], [4]. Despite these advancements, there remains a gap in addressing the challenges of English instruction in college settings. The dual pressures of employment and further study faced by students in higher education exacerbate the competition for job opportunities. Additionally, the ongoing reforms in the higher education system, still in their nascent stages, require a more robust alignment with secondary education. Consequently, English education in higher education contexts necessitates an efficient and effective approach.

This paper contributes to the field by evaluating English text readability through a DL model tailored for college English education. A text CNN evaluation model is developed, with parameters fine-tuned for optimal performance. The methodology includes the creation of a text readability dataset from college English textbooks. The results demonstrate that the model effectively predicts text readability with an accuracy rate of 82%, evaluating sentence complexity and text length through weight learning. This approach provides substantial support for English instruction in colleges, aiding in bridging the gap in students’ English proficiency.

Deep learning algorithms have revolutionized the assessment of text readability, offering more precise and adaptable methodologies compared to traditional measures. Utilizing large-scale text data, these algorithms, which include recurrent neural networks (RNNs), long short-term memory networks (LSTMs), and CNNs, are employed to predict textual complexity. RNNs and LSTMs, in particular, have been recognized for their ability to analyze the context and structure of text [5], significantly surpassing traditional readability formulas such as Flesch Reading Ease and Gunning FOG Index in accuracy. These models, through training on extensive datasets of human-rated text readability, can meticulously evaluate text complexity by considering factors such as sentence length, word difficulty, and syntactic complexity. The proficiency of RNNs and LSTMs lies in their capacity to capture sequential relationships in language, thus enabling a comprehensive assessment of text coherence and flow [6]. Furthermore, CNNs, traditionally used in image processing, have been adapted for text analysis, demonstrating their versatility. They can extract salient features from text data, such as n-grams, to determine readability. By training CNNs on large text corpora annotated with readability labels, these models can discern patterns indicative of text difficulty, rendering them effective for a broad spectrum of readability evaluation contexts.

Moreover, the introduction of Transformers, a groundbreaking deep learning architecture, has markedly enhanced the assessment of text readability. Renowned for their efficacy in natural language processing, models like Bidirectional Encoder Representations from Transformers (BERT) utilize a bidirectional approach to gather context, allowing for a more nuanced understanding of text complexity [7]. Transfer learning has been instrumental in adapting these pre-trained models to readability datasets across various domains, thereby broadening their application scope.

These deep learning methods, encompassing sophisticated architectures like Transformers and neural networks like RNNs, LSTMs, and CNNs, have been pivotal in evaluating text complexity. By training on large, labeled datasets, they consider aspects such as syntax, semantics, and context, offering more refined and nuanced readability assessments. This advancement has profound implications for content creation, educational applications, and other sectors where text readability analysis is crucial [8]. Additionally, self-training and domain adaptation techniques have been explored to enhance these systems' proficiency in processing input from second language learners.

In the context of text readability evaluation, Transformers have significantly outperformed conventional sequence models like RNNs or LSTMs. Their design, incorporating a self-attention mechanism, enables the assessment of the relative importance of each word in a sequence. Models like BERT, pre-trained on extensive text corpora, have been fine-tuned for readability evaluation, excelling in interpreting text complexity and adapting to domain-specific linguistic nuances [9]. Their ability to handle long-range dependencies in text positions Transformers as highly capable in considering context, semantics, and syntax in readability assessment.

RNNs and LSTMs have been applied to various natural language processing applications, including text readability evaluation. RNNs process data sequences sequentially while maintaining a hidden state, and LSTMs, an advanced version of RNNs, effectively manage the vanishing gradient problem, making them suitable for longer sequences. These models excel in analyzing sentence structures and word dependencies, thus accurately estimating text complexity. While Transformers have largely superseded RNNs and LSTMs in NLP tasks, these models still hold relevance in certain contexts.

CNNs, though primarily associated with image processing, have proven effective in text analysis tasks, including readability evaluation. Employed to extract textual features such as n-grams, CNNs undergo training on extensive datasets with readability labels, enabling them to identify patterns correlating with text complexity. While their popularity in NLP may not match that of Transformers, CNNs continue to offer valuable insights in text readability analysis and remain computationally efficient for specific tasks [10], [11].

The initial step in utilizing deep learning for text readability assessment is text encoding, a process in which textual words are converted into numerical representations. Commonly, this is achieved through one-hot encoding, where each word is denoted by a large vector. The length of this vector corresponds to the total number of words in the corpus, with the position representing the specific word marked as one, and all other values set to zero [12]. This method, however, presents two substantial limitations: its inability to capture relationships between words and the problem of high dimensionality in large vocabularies [13].

An alternative to one-hot encoding is distribution-based text representation, which addresses these drawbacks effectively. In this approach, words are represented as low-dimensional real variables, significantly reducing the dimensionality issue and allowing for the representation of word-to-word relationships. Nevertheless, this method encounters a challenge in differentiating words with similar meanings, potentially leading to an overlap in word properties [14].

Prominent models utilized in distribution-based text representation include the Continuous Bag-of-Words (CBOW) model and the Skip-Gram model. The CBOW model focuses on predicting a word based on its context, estimating the likelihood of a word's occurrence within a specific contextual framework. Conversely, the Skip-Gram model infers the contextual words from a given word, essentially reversing the process employed by the CBOW model. Figure 1 and Figure 2 provide schematic illustrations of the CBOW and Skip-Gram models, respectively.

In enhancing the efficacy of word training, both the CBOW and Skip-Gram models employ hierarchical SoftMax in the fully connected layer, coupled with a negative sampling strategy [15], [16]. However, these models share a common limitation in their design: they are trained to recognize the contextual continuity of words but do not distinguish between words that may appear frequently throughout the text. The models do not differentiate between global and local word associations, which could be a drawback. Despite this, both the CBOW and Skip-Gram models have demonstrated competent performance in readability assessments, largely unaffected by these limitations.

DL networks, such as CNNs, RNNs, and LSTMs, play a pivotal role in the process of text readability assessment. Each network type possesses distinct characteristics and functions, as elaborated below.

•CNNs, comprising multiple deep layers, primarily utilize convolution computations. A standard CNN architecture includes an input layer, convolutional layers, and pooling layers. Renowned for their efficacy in image processing, CNNs have also demonstrated considerable capability in text categorization and semantic analysis, especially in the realm of text readability assessment, a form of text classification (Figure 3).

•As depicted in Figure 3, CNNs use various convolution kernels in their convolutional layers to extract word properties essential for text classification [17]. The activation function layer often employs the ReLU function for efficiency, and the output from the fully connected layer is flattened, enhancing its suitability for subsequent processing stages.

•RNNs, designed to handle sequential input, connect nodes in accordance with the sequence order, making them particularly effective in sequence processing. They exhibit a unique capability to retain accurate memory of previous datasets due to their complete and progressive parameters [18]. Figure 4 illustrates an RNN's structure, comprising an input layer, hidden layer, and output layer, underscoring its utility in both text processing and image recognition.

•LSTMs, a specialized form of sequence model, consist of three primary components: the forget gate, input gate, and output gate. These networks overcome the gradient vanishing problem commonly encountered in traditional RNN models. LSTMs' innovative cellular structure allows for intelligent decision-making regarding data retention or disposal. The model sequentially analyzes data, making crucial decisions about which information to preserve and which to discard [19], [20]. Eq. (1) outlines the control function governing the process.

where, $w_f$ and $b_f$ represent the weight and bias of the forget gate, respectively. The determination of the necessity and extent of alterations to historical data is a crucial step. Such data is selectively fed into the input gate, as delineated in Eqs. (2) to (4).

where, $w_i$ and $w_c$, $b_i$ and $b_c$ denote the respective weights and biases integral to the process. The value of the current cell state is represented by $C^t$. The role of the output gate is then to adjudicate whether the information, post-initial filtration by the preceding gates, is suitable for output. This gate encompasses internal switches that regulate the output mechanism, as shown in Eq. (5).

where, $w_o$ and $b_o$ are the parameters representing the weights and biases of the output gates, respectively.

In natural language processing applications, the Bidirectional LSTMs (BiLSTMs), which contain both forward and backward LSTM units, demonstrates the importance of contextual content in the representation of words [21]. Unlike standard LSTMs that solely utilize forward processing, BiLSTMs account for word context, highlighting the significance of bidirectional analysis.

In the construction of the text readability evaluation model, datasets comprising textual information are employed. These datasets serve as the foundation for training the model, after which new textual data is introduced for the purpose of evaluating the model's efficacy [22], [23]. The model is initially trained with a portion of the text, which has been pre-labeled for this purpose. The text data utilized for labeling is denoted as $D=\{(d 1, l 1),(d 2, l 2) \ldots,(d n, l n)\}$, representing the text dataset, with $n$ indicating the number of labeled texts. $l i \in\{G 1, G 2, \ldots, G m\}$ signifies the text readability grade, and $m$ represents the corresponding readability grade. The model's structure is bifurcated, aligning with its theoretical underpinnings. One segment encompasses the dataset used for model training, while the other pertains to the dataset under evaluation. The labeled data facilitates the extraction of text features, which are subsequently instrumental in assessing the text. Figure 5 offers a schematic depiction of the model's structure.

As illustrated in Figure 5, the model comprises three primary components: the text input layer, the text feature extraction layer, and the text classifier. These components collectively facilitate the transition from text dataset to readability evaluation. Additionally, references [24], [25], [26], [27], [28], [29], [30], [31], [32], [33] present further deep learning models pertinent to image detection, providing a broader context to the methodologies employed.

The process of text representation, crucial for text readability assessment, involves the generation of word vectors. These vectors, representing words numerically, are embedded within a vector space. The subsequent stages involve the extraction and training of features, with deep learning algorithms facilitating the training of words across multiple dimensions. This method allows for the representation of textual input, where the proximity of word vectors signifies similarity in meaning [34]. The model for text feature extraction, grounded in CNN, employs convolution kernels of varying sizes to train word data from the input layer, thus extracting word relationships.

In the text classifier component, words are categorized using a SoftMax layer, which applies logistic regression for classification. Logistic regression, under the assumption of a Bernoulli distribution for the data, utilizes gradient descent for optimizing parameters through maximum likelihood estimation [35], [36]. Post-classification, the categorized word vectors are then fed into the fully connected layer. Regarding dataset preparation, the required text readability dataset is derived from undergraduate English textbooks, following the procedure illustrated in Figure 6.

The process begins with the segmentation of college textbooks into sections using a book cutter. High-speed scanning is then applied to each page. Subsequently, text recognition interfaces identify the textual content within the instructional materials. Under expert supervision, the scanned content undergoes rigorous examination. Post-evaluation, any repetitive scanned words are removed using Excel, followed by updating the text database [37].

For sample balance, three textbooks are selected, categorized into three levels based on difficulty: Level 1 (easiest), Level 2 (moderately difficult), and Level 3 (most difficult) [38], [39], [40]. To maintain consistency, the extraction of sentences is limited to 1,000 per level. Table 1 provides a breakdown of the sentence count across these levels.

In the hyperparameter configuration for model training, 25% of the sample count is allocated for labeling, with the remaining 75% forming the training set. Table 2 outlines the hyperparameter settings for the text CNN model. A batch size of 64 is selected for each training iteration, incorporating a dropout rate of 0.5. Convolution kernel sizes are set at three, four, and five, with an established learning rate of 0.001.

The implemented deep learning-based model for text readability assessment has demonstrated an impressive accuracy rate of 82%. Table 3 illustrates the model's ability to classify sentences into distinct complexity levels following training.

These results indicate that the model can accurately categorize textbook phrases into Level 1, Level 2, and Level 3. Previously, the focus was on identifying the difficulty level of sentences. In the evaluation of a complete text, Level 1, Level 2, and Level 3 sentences are assigned different weights, leading to Scores 1, 2, and 3, respectively. The lower the score, the simpler the text is to comprehend, with Score 1 indicating the easiest level. Conversely, a higher score, such as Score 3, suggests greater complexity. Figure 7 displays the outcomes of the model's evaluation of text readability.

The model categorizes Texts 1, 2, and 3 into Level 1, Level 2, and Level 3 complexities, respectively (Figure 7). This classification underscores the feasibility and accuracy of the proposed approach. Such findings can be effectively employed in educational settings, allowing for tailored instruction that accounts for individual student capabilities. This approach is beneficial for student development. Moreover, as illustrated in Figure 8, the model can classify students' English proficiency levels, with 12% fitting into Level 1, 31% into Level 2, and 57% into Level 3.

Additionally, a control group experiment involving homework assignments was conducted to validate the model's impact on university teaching. The experimental group utilized the DL-based text readability evaluation method, while the control group did not. It was observed that texts classified as Level 1 and Level 2 received more correct responses compared to Level 3, which had the least. This correlation between reading test scores and text readability levels confirms the validity of the DL-based text readability evaluation method from a scientific perspective. The empirical evidence supports the effectiveness of this approach in enhancing college English instruction.

This study has explored a deep learning-based methodology for evaluating text readability within the context of collegiate English education. A text CNN evaluation model was developed and demonstrated an accuracy rate of 72%. The findings suggest that further exploration into more sophisticated models, such as LSTM, BiLSTM, Transformer, and BERT, is warranted to enhance the accuracy of text readability evaluations.

A limitation of this study is identified in the small dataset size, which may not comprehensively represent the diversity of texts encountered in college English courses. This limitation potentially affects the model's ability to accurately assess a broader spectrum of educational materials. Additionally, the study's small sample size could impact the validity of the findings and may not reflect the full range of texts utilized in higher education settings. Moreover, the model's parameter settings are based on pre-existing knowledge, which introduces the risk of bias or suboptimal performance.

The research underscores the need for future investigations into advanced models like LSTM, BiLSTM, Transformer, and BERT. Such research endeavors would aim to address the current study's limitations and further refine the assessment of text readability, particularly in the context of college English instruction. The enhancement of text readability assessment methods could significantly contribute to the development of more effective teaching strategies, benefiting both educators and students in higher education.