Page 95 - IJEEE-2023-Vol19-ISSUE-1

Alkabool, Abdullah, Zadeh, & Mahfooz
that giving a model full paper allows the model to perform
well in uncommon categories. However, some categories,
such as rebuttals and counterclaims, may require further
training examples.
Fig. 4: Average F-1 scores across all models (except
Baseline) for each discourse element
4) Increasing validation loss
All models started to show an increase in validation loss
after epoch 3, for example, the top-performing longformer
model increased its validation loss from 0.54 to 0.79 over
epochs 3 to 7 (Fig. 5). According to the Javatpoint article
"Overfitting in Machine Learning" [22], a telltale sign of
overfitting a model is increased validation error during
training, and one way to prevent this is to stop early. Figure
6 show the F-1 scores during model training. As defined by
the Elite Data Science website, early stopping is the process
of "...stopping the training process before the learner passes
that point [point where variance starts to increase] ..."[23]. I article guides the model during fine-tuning to learn positional
believe I should implement early stopping for my model
around the 2nd or 3rd epoch because that's when the variance
starts to increase. Another approach I could try is to augment
the examples during the training phase. According to
Xiaoshuang Shi in his article The Problem of Overfitting and
How to Resolve It [24], sharing more training examples is a
good way to solve the overfitting problem. In particular, I
should provide articles with many examples of
counterclaims and rebuttals, because that's where my model's
performance is weakest. My work here shows that when fine-
tuning a model for classifying discourse elements, more
emphasis needs to be placed on getting more examples,
rather than applying the model to a large number of epochs.
Fig. 5 Validation loss during model training
| 91
Fig. 6: F-1 scores during model training.
IV. CONCLUSION
In conclusion, writing is an important skill and it is vital
for young people to develop their writing skills. By using an
automated writing feedback system, we can help students
develop their writing talents by providing a detailed analysis
of their writing. One way to improve current automated
writing feedback systems is to combine them with machine
learning models to differentiate between different writing
elements in student essays. In this experiment, I show that
the longformer model outperforms the BERT or GPT2
models in discourse classification. I also show how the entire
relationships between utterance elements, especially for the
Lead and Final Statement classes. However, positional
encoding alone does not solve the discourse classification
problem, and more attention needs to be paid to acquiring
more categories of data, such as rebuttals or counterclaims,
to improve the overall results.
CONFLICT OF INTEREST
The authors have no conflict of relevant interest to this
article.
REFERENCES
[1] I. Yulianawati, “Self-Efficacy and Writing : A Case
Study at A Senior High School in Indonesian EFL Setting,”
Vis. J. Lang. Foreign Lang. Learn., vol. 8, no. 1, pp. 79–
101, 2019,
[2] L. Darling-Hammond, “Teacher quality and student
achievement: A review of state policy evidence,” Educ.
Policy Anal. Arch., vol. 8, no. November 1999, 2000.
[3] Trey, “5 reasons to use grammarly,” Oct 2019. [Online].
Available https://www.apoven.com/grammarly-benefits/
[4] T. N. Fitria, “Grammarly as AI-powered English Writing
Assistant: Students’ Alternative for Writing English,”
Metathesis J. English Lang. Lit. Teach., vol. 5, no. 1, p. 65,
2021.
[5] J. Devlin, M. W. Chang, K. Lee, and K. Toutanova,
“BERT: Pre-training of deep bidirectional transformers for
language understanding,” NAACL HLT 2019 - 2019 Conf.
North Am. Chapter Assoc. Comput. Linguist. Hum. Lang.
Technol. - Proc. Conf., vol. 1, no. Mlm, pp. 4171–4186,
2019.
[6] I. Beltagy, M. E. Peters, and A. Cohan, “Longformer: The