Law Students’ Essay Writing Performance and Their Writing Difficulties

Introduction

Many researchers (Ferretti, Andrews-Weckerly & Lewis, 2007; Neff-van Aertselaer & Dafouz-Milne, 2008) have confirmed that writing is the most difficult language skill. Unfortunately, both ESL and EFL students at the tertiary level often experience difficulties in the use of both generic structure and detailed subject knowledge required in writing. Consequently, due to the pitfalls in past language teaching methods, particularly in writing skills, a number of research studies (Lertpreedakorm, 2009; Promwinai, 2010) have sought to explore and illustrate how writing can be developed by using effective methods to assess the weaknesses of Thai EFL students’ writing ability. There is a need to know more about the specific problems students experience when writing essays. Unfortunately, research on writing difficulties has predominantly focused on holistic problems in writing essays. Hence, there is a critical need to investigate effective practices in order to develop students’ writing competence. However, awareness of teaching essay writing has not translated into effective practice for the assessment and instruction of university students. Moreover, research studies on writing difficulties in Thailand are meagre, resulting in a shortfall of insights into the problems that Thai EFL students encounter in writing tasks. Furthermore, the results of this study put forward recommendations and suggestions for Thai EFL lecturers in selected public universities in Thailand to gain further insights into the weaknesses of their learners’ language variants, as these insights can be used to develop their teaching programmes and instruction to more effectively support students’ writing development.

Review of Literature

Teaching Writing in ESL/ EFL Contexts

In the area of Second and Foreign Language Instruction, teaching writing has not been changed into an effective way in order to develop students’ writing competence. Although, there are a number of teaching strategies for writing in English as a Second Language (ESL hereafter) or English as a Foreign Language (EFL hereafter) contexts, not many ESL/ EFL writing teachers have a clear understanding on writing approaches. Therefore, much of teaching writing still focuses on a traditional approach that is mainly concentrated on the knowledge about the structure of language and writing improvement as the result of the imitation of input, in the form of texts provided by the instructors. ESL/EFL writing is a hard, complicated and demanding procedure. This difficulty and complexity in ESL/EFL writing arises from the reality that writing accounts for searching out a thesis, fostering support for the claim, formulating, modifying, and finally editing the thesis to ensure an effective, error free writing product. Additionally, ESL/EFL writing is one of the most pivotal genres of language teaching. As claimed by Coffin. (2004, p.3), “students’ academic writing continue to be at the centre of teaching and learning in higher education, but it is often an invisible dimension of the curriculum; that is, the rules or conventions governing what counts as academic writing are often assumed to be part of ‘common sense’ knowledge students have, and are thus not explicitly taught within disciplinary course.’’ To provide an effective ESL/EFL writing instruction is the main responsibility for instructors, researchers, textbook writers and programme coordinators in the area of foreign language teaching, but producing a textbook for most ESL/EFL students is a laborious task because the writing process needs an extensive range of cognitive and linguistic methods of which ESL/EFL students are largely limited. Moreover, research about ESL/EFL writing has developed dramatically over the last 40 years, specifically between the late 1980s and the early 1990s. Generally, there are three main types of ESL/EFL writing strategies including product approach (Silva, 1990; Brown, 2001), process approach (Silva, 1983) and genre-based approach (Hyland, 2003a; Hyland, 2003b).

The Difficulties of Writing Essays

Writing an essay is one of the most prevalent types of assignments set in Thai universities. This genre of writing requires students to generate a well-organised essay. Most students at the tertiary level have difficulty generating ideas for writing, planning what to write, organising thoughts, setting goals for effective writing, self-monitoring performance, and revising for content and mechanics. Producing a good piece of writing is often difficult due to the planning, composing, and revising skills required for effective writing.

Written expression is more effective in social-cognitive instances of communication than oral expression. Unlike speaking, writing enables students to convey a message independent of time and space (Hughes, 1996). It is considered man’s best academic achievement based on skills or components like mechanics, production, conventions, linguistics, and cognition. The act of writing is independent of time and place; the writer has to depend upon formal features to convey the intended meanings. Failure to take advantage of these features correctly causes frustration for the writer (Leisak, 1989). English is the foreign and official language in Thailand as well as an easy language to work with and to learn science and technology at higher levels. It is taught as a compulsory subject in schools; however, the majority of Thai EFL students cannot communicate properly in English and perceive it as a very difficult subject. Many of the students from the Arts and Humanities areas cannot meet the requirements of the examination in English. The purpose of the study was to explore difficulties in writing in the English language. It specifically aimed to identify difficulties in English language writing related to grammar, punctuation, L1 interference, vocabulary, and spelling for students with English as a second language. Based on many research studies, Thai EFL students spend little time on critical writing processes and tend to focus on low-level transcription skills such as handwriting, spelling, capitalization, and punctuation (Graham, 2006; Ka-kan-dee and Kaur, 2015). They are unable to express ideas or demonstrate knowledge in their writing tasks. Additionally, they often struggle with the planning, composing, and revising skills needed for effective writing.

Assessment of Writing Performance Issues of Reliability and Validity in the Assessment of Writing Performance

According to Weigle (2002), any writing test associated with actual writing, as opposed to completing selected-response or limited-production items, can be regarded as performance evaluation. McNamara (1996) suggested that performance evaluation can be differentiated into a strong and a weak sense. In English as a foreign language (EFL) and English as a second language (ESL) writing tests in the strong sense, the test task will represent a real-world task, such as making an official request, and performance will mainly be judged on real-world criteria as well. The focus of performance evaluation is on the successful completion of the task, and not on language use in the writing task performance. Messick (1994) stated that EFL writing is only a medium of performance and an insufficient condition for success. In fact, if aspects of EFL writing ability are highlighted, a larger set of criteria is used to reflect EFL writing ability. Performance of the task itself is the target of the evaluation. On the other hand, in the weak sense of performance evaluation, the focus of the assessment is on the language used. Although the task used to elicit writing may be similar to real-world tasks, the aim is to elicit a display of writing ability. McNamara (1996) suggested that most language performance tests are weak in this sense. The difference between the strong and weak forms of performance assessment is a crucial conceptual distinction. For example, when an EFL writing test in the weak sense is designed to elicit a performance of test-takers’ EFL writing ability, the scoring criteria must clearly reflect the definition of the construct of writing competence. Most importantly, raters need to be trained to interpret these criteria in language-related terms. Otherwise, test scores can reflect construct-irrelevant variance, such as creativity and neatness. In other words, it is not possible to draw inferences from task performance to make interpretations about any non-test situation. The relevance of test tasks to the expected score representation is considered the heart of the performance test (Kenyon, 1998).

Messick (1989, p.13) described validity as “an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores”. To ensure the relevance and validity of a test, test designers are required to consider all features of the testing setting that considerably influence test performance, including the specification of the domain in terms of topic knowledge, test requirements, administration conditions, and criteria for scoring. Such requirements illustrate what features of the test process are likely to affect test scores and need to be controlled (Messick, 1994). Weigle (2002) introduced a valuable way to conceptualise what is required for test construction. This concept can be connected to the three stages in Bachman and Palmer’s (1996) framework of test development: the design stage, the operationalisation stage, and the administration stage. To ensure the validity of the test at the design stage, McNamara (1996) proposed sampling test content from the communicative tasks experienced by test-takers in the target language use setting to ensure content validity. The recommended steps comprise consulting with expert informants, investigating available sources on the communicative requirements of the target language situation, exploring and classifying communicative tasks in the target language setting, gathering and analysing texts from the target language setting, and deciding on an extensive test procedure.

In the operationalisation stage, data from the design stage are employed to generate test requirements or detailed specifications for test writers to follow. According to Douglas (2000), test requirements consist of a description of the test tasks, the criteria for accuracy, and sample tasks. To ensure validity at this stage, it is imperative to elicit the components of the writing task that are contained in the definition of the construct to be tested. For an EFL writing task to elicit the constituents that accurately define the construct, it is recommended that the writing prompt be sufficiently clear and specific in order to restrict possible interpretations (Horowitz, 1991). In addition, a scoring rubric that does not precisely reflect the construct being assessed may influence the validity of interpretations made on the basis of test results (McNamara, 1996). The rubrics should thus comprise a precise and unambiguous statement of the constituents of the construct that are being measured. Brown and Bailey (1984) and Hamp-Lyons (1991) advised that it is crucial to consider rubrics with a greater number of subscales, similar to analytic rubrics. They are commonly seen as preferable for providing greater overall consistency in scoring.

Construct validation at the design and operationalisation stages roughly concurs with what Weir (1988, as cited in McNamara, 1990) categorised as construct validity a priori and a posteriori. A priori construct validation refers to validation in which test content relies on a precise theory of language and language use. On the other hand, a posteriori construct validation is associated with the empirical and statistical validation of the constructs considered at earlier stages using test performance information. This second type of construct validation correlates with the third and final stage in Bachman & Palmer’s (1996) test development process, the administration stage. At this stage, we are concerned with gaining evidence that the test scores actually reflect the elements of the construct represented in the design of the test by distinguishing and elucidating construct-irrelevant aspects in the test setting. Therefore, the evaluation of performance poses a unique set of challenges. However, with reference to the EFL writing evaluation context, indirect procedures of evaluation are more favoured. Following this reasoning, performance tests require a test-taker to demonstrate both language knowledge and the use of that knowledge and skill in real performance in communicative settings. They thus provide outstanding predictive validity. Moreover, interpretations can be made about test-takers’ future performance in real-world situations based on test performance. This leads to an effective form of evaluation that demonstrates sample facets of language knowledge differently. It does not yield information about real performance in a particular test context or in contexts in the real world. It is unfortunate that most lecturers experience the problem of how to define the real world and attempt to capture it in the test. This leads to the problem of how to score the limited sample that can be collected in the test so that the score has meaning for the target domain. It is suitable for any instructional situations that equip learners to become proficient communicators. Currently, there is an urgent requirement for a distinctive sort of scoring in ESL and EFL writing evaluation contexts to help ensure that scores reflect salient aspects of writing in a balanced manner. In order to achieve a rational balance among all those components of good writing, readers are required to pay close attention to all of them. It is believed that readers need to pay attention to the multidimensionality of ESL and EFL writing, in particular for a detailed scoring process. A carefully constructed writing proficiency test or diagnostic test with detailed score reporting is not only of great use for large-scale proficiency evaluation of writing but is also useful for the program evaluation purpose of evaluating change in writing. It also offers the possibility of providing useful data that can be employed in language teaching programs for making accurate initial placements.

It can be concluded that to validate a test score inference is to support the plausibility of the underlying interpretive argument with proper evidence. Consequently, the argument-based approach to validation is employed in order to adopt the interpretive argument as the framework for gathering and presenting validity evidence and to provide convincing evidence for its interpretations and assumptions, in particular its most questionable inferences. Thus, Kane (1992, p. 527-535) suggested that “to validate a test score one (a) decides on the statements and decisions to be based on test scores, (b) specifies the inferences and assumptions leading from the test scores to these statements and decisions, (c) identifies potential competing interpretations, and (d) seeks evidence supporting the inferences and assumptions in the proposed interpretive argument and refuting potential counterarguments”.

A Holistic Scoring Rubric for English Essays

With reference to evaluating the overall quality of the participants’ written argumentative essays, a 5-point scoring rubric employed by McCann (1989) and Nussbaum and Kardash (2005) was adopted to rate overall quality in this research study. This scoring rubric drew on the clear-cut statement of organization and language use in McCann’s (1989) rubric and the detailed description of the effectiveness of argument in Nussbaum and Kardash’s (2005) rubric. Specifically, this holistic rubric consisted of the following three aspects: the overall effectiveness of the argument, including the presence or absence of potential opposing views; overall organization; and language use. The procedure of validating the holistic rubric required several rounds. In the first round, two raters who were experienced English writing instructors teaching English for Academic Purposes 2 individually scored 16 writing essays randomly chosen from the sample, employing the holistic scoring rubric. With reference to inter-rater reliability, the value of coefficient alpha was .82 before negotiation. While rating the essays, the two raters were instructed to pay heed to the explicit criteria described in the rubric. Then the researcher discussed what dimensions of the rubric figured in the scores the raters provided. In the second round, based on the feedback from the two raters, the rubric was revised to ensure that the descriptions were specific and valid. It was imperative that each aspect of the rubric was appropriate and clearly delineated in relation to the participants’ written argumentative essays. In the third round, utilising the improved rubric, the two raters were asked to independently rate another 20 randomly chosen participants’ written essays. They were required to note down the criteria for yielding true scores. With reference to inter-rater reliability, the coefficient alpha value was .88 before negotiation. Then the raters’ feedback on the improved rubric was elicited, and some minor alterations were made. Finally, the researcher and one of the two raters rated all the participants’ written essays employing the final version of the rubric. The score for each participant’s essay was the average of the two scores from the raters. Therefore, the two raters were required to discuss and negotiate their scoring until they reached an agreement.

It should be mentioned that the rubric does not focus on the use of the Toulmin model. The three aspects of the quality of participants’ argumentative essays reflected in the rubric hinged on common criteria for effective essays, as represented in previous research (e.g., McCann, 1989; Nussbaum and Kardash, 2005). In regard to the raters’ training, attention was mainly focused on not displaying bias toward any of the three aspects. In other words, the raters were not guided toward any of the three aspects when rating the participants’ essays.

Methodology Research Design

This research employs a mixed-methods approach to investigate the level of writing anxiety of 60 law students by using a questionnaire and a semi-structured interview to identify the factors that affect their performance.

The researcher used a 5-point scoring rubric employed by McCann (1989) and Nussbaum and Kardash (2005) to assess the overall quality of students’ writing performance and identify their weaknesses in writing essays.

Participants

Data were gathered from 60 first-year law students who were enrolled in English for Academic Purposes 2 at a selected public university in Thailand for the academic year 2016. Ethical considerations were adhered to in this study, as students’ names were not kept anonymous for all data collection procedures and student consent letters were provided to the students before they decided to participate in the study.

Research Questions

1. What are the writing difficulties encountered by Thai EFL law students?

2. What is the overall quality of law students’ writing performance?

3. What is the level of law students’ writing anxiety?

Research Instruments

Qualitative data were also collected by employing a semi-structured interview to assess the difficulties encountered by the 60 law students.

Quantitative data were collected by administering two sets of writing assignments to the students. These writing prompts were taken from the assignments in English for Academic Purposes 2. These two essay topics were considered suitable for the students to express their ideas. The students were required to choose one topic and write an essay of between 250 and 265 words. The students were given one hour to complete this writing task. Additionally, the researcher employed a questionnaire to gauge the level of law students’ writing anxiety.

Data Analysis

The quantitative data from the questionnaire were analysed using SPSS to determine the students’ level of oral presentation anxiety. A Likert scale was employed to gauge the level of law students’ writing anxiety when they were writing their essays.

The 60 written argumentative essays were gathered from a group of English-major students at two selected public universities in Thailand. The written essays were assessed using a 5-point scoring rubric to evaluate the students’ writing assignments. The researcher employed the holistic rubric developed by McCann (1989) and Nussbaum and Kardash (2005) because it provided clarity in assessing the writing submissions and had a high degree of reliability and accuracy. This holistic rubric comprised three dimensions: the overall effectiveness of argument, including the presence or absence of possible opposing views; overall organisation; and language in general. These three dimensions were chosen to illustrate the quality of an essay relying on general criteria for an effective essay.

The two essay topics were distributed to the participants, and they were asked to select only one topic to answer the writing prompt. The 60 participants were given one hour to work on this writing task. The two essay topics were given to the participants after the researcher had briefed them on the objectives of the study. This research study aimed to assess law students’ writing performance and their weaknesses in writing. Therefore, a total of 60 essays were gathered from law students. The written essays were then systematically rearranged to allow the researcher to trace the essays back to the participants and to the raters, who included two experienced EFL teachers who would be rating all the written essays.

The two raters were EFL lecturers from Khon Kaen University, Nong Khai Campus, who were proficient in teaching writing.

A semi-structured interview was employed to investigate writing difficulties when students compose argumentative essays. The purpose of this study was to assess their writing and investigate the weaknesses in their essays. The researcher interviewed 60 students and analysed the data using NVivo 10, Computer-Assisted Qualitative Data Analysis Software (hereafter CAQDAS), to identify the main barriers in composing essays. The researchers used NVivo 10 to analyse the data after completing the data transcription. Initially, the researchers constructed the project and the entire transcription file was imported into the project. After that, the analysis process began:

1. the researcher read the content multiple times in order to identify the main themes,

2. the selected themes were coded into NVivo 10,

3. another researcher was asked to code the same project,

4. a coding comparison was conducted in order to check inter-coder reliability. At this stage, the researchers determined the level of agreement, namely whether there was agreement or disagreement on each theme. If there was no agreement on a theme, the researcher excluded it to retain only accepted Kappa results above 0.70. The Kappa coefficient is employed to measure the proportion of corresponding codes to indicate inter-coder reliability.

K = ( TA -∑EF ) ÷ ( TU - ∑EF )

TA = total units of agreement between two users

∑EF = the expected frequency of agreement occurring by chance

TU = total units within the sources

Findings Results the Level of Law Students’ Writing Anxiety

Level No. Percentage High ( 93 -110 )

45 75.00 Moderate ( 85-92 ) 12 20.00 Low ( 34 -84 ) 3 5.00 Total 60 100

Table 1 shows that law students experienced anxiety at a high level (75%) while they were writing essays. Law students with low levels of proficiency are likely to avoid tough and challenging tasks, which they recognise as a personal constraint. These law students have low aspirations and a low sense of obligation, make less effort, and may not have, or may quickly lose, their confidence to write their essays. In contrast, students with a strong sense of capability can build up personal achievement in various ways. They undertake difficult tasks as challenges to be proficient and to provide opportunities for them to demonstrate their strong sense of obligation in an endeavour to control the circumstances. Only 5% of the respondents reported feeling a low level of writing anxiety, but they believed that their anxiety or worry negatively affected their writing.

Results of Students’ Writing Performance

Sekaran and Bougie (2010) asserted that the Cronbach’s alpha coefficient reliability index is acceptable for a value above 0.80, and Hair et al. (2010) and Pallant (2010) confirmed that a value above 0.70 is satisfactory. Table 2 below illustrates the Alpha reliability estimate for inter-rater reliability in rating students’ writing performance. At 0.90, it confirms that the inter-rater reliability score scale given by the two raters possesses a high internal consistency. The scale used in this study was observed to be reliable and acceptable for assessing students’ writing essays.

Table 2

Interrater’s reliability analysis

Number of students Number of Scales Cronbach’s coefficient

Alpha 60 5 .90

Table 3 below illustrates the frequency and percentage of students’ writing tasks:

Scale Score Frequency Percentage

Scale 1 13 21.67% Scale 2 15 25% Scale 3 20 33.33% Scale 4 11 18.33% Scale 5 1 1.67%

Table 3: Frequency and Percentage of Students’ Writing Tasks

As shown in Table 3, 33.33% of students were rated in scale 3. 25% of students were evaluated in scale 2. Very few students were rated in scales 4 and 5. Only 1.67% were rated in scale 5 and 18.33% in scale 4.

According to the results, it can be concluded that students’ writing tasks have 8 main weaknesses as follows:

1. unfamiliarity with the rhetorical features of writing 2. insufficient grammatical knowledge 3. insufficient academic vocabulary 4. inability to generate well-organised ideas

5. inability to write effective conclusions; 6. lack of awareness of the audience’s expectations and motivational elements; 7. lack of awareness of the planning process of writing; 8. inability to think creatively.

The findings indicated that Thai EFL students’ writing tasks had weaknesses in word choice, grammar, mechanics, and structure. These weaknesses were the main barriers that led to ineffective writing. It was observed that Thai EFL students could not create persuasive arguments because they used only personal opinions to back up the thesis statement.

In the semi-structured interviews, students were asked whether they liked writing or not. Most students said they did not like writing, as evidenced in the excerpts below.

“I do not like writing at all” Cherry

“I hate writing” Metha

“I am unskilled for writing because I have no ideas to write” Nancy

In answering the second question as to whether they liked writing, most students said they did not like to give oral presentations as a tourist guide because they lacked confidence in giving oral presentations due to their lack of oral presentation skills, which required memorization of the content. The students made the following comments:

“I do not like writing. I am not confident to express my ideas.” Cherry “I am unskilled in writing therefore I hate writing.” Nancy “I hate writing because I cannot write a good essay” Metha Some students said they liked writing, but they needed to practice writing in order to build up confidence in writing. One of the students stated:

“Personally, I like writing but I need time to plan for my writing” Bomb

Students were asked to describe the experience of their previous writing assignment. Most students had anxiety about their writing. Students were very nervous, which caused them to forget the content. They were not able to write their essays effectively because they lacked training in writing preparation, as evidenced in the excerpts below.

“I had an anxiety about writing an essay because we did not have enough practice before writing.” Sai

“I worried a lot of while writing an essay. I felt pressured because I could not finish my writing on time.” Noah

“I am very excited. I have to lessen my own emotional excitement but I cannot control it. Then I forget what I prepared to write” Oil Some students can only control their excitement by practising writing every day. One male respondent said that he had to practise writing at home every day to build up confidence in writing. He stated that:

“I have some excitement but I can control while I am writing an essay but I have to practice writing every day to build up my confidence.” Pat

Implication for the Development of Writing Essays

The aim of this study was to assess the overall quality of law students’ writing performance and identify the writing difficulties that students encountered when they composed essays. It was noted that researchers, educators and policymakers have been aware that there is a critical need to pay greater attention to the development of writing instruction for university students. The way to improve writing instruction is to develop insights into the diagnostic evaluation of writing. This type of evaluation provides valuable data about students’ weaknesses in writing. Moreover, it is helpful for EFL lecturers in selected public universities in Thailand to gain further insights into the weaknesses of their students’ language variants that can be used to improve their teaching programmes and instruction to more effectively serve students’ writing development.

Additionally, it can also be employed as guidance for students to develop their writing. The findings of this study can also provide useful recommendations for curriculum planners and materials writers and designers to incorporate components that focus on writing, in order to address the range of difficulties students experience when they compose essays. Such stakeholders can then provide appropriate teaching strategies to develop students’ writing competence in the Thai EFL context.

Writing Difficulties

The goal of this study was to examine the difficulties that law students experienced when writing essays. The findings of the study revealed that the main difficulties faced by law students were vocabulary, grammar structure, interpreting the question, fulfilling task demands, evaluation, rhetorical features, awareness of audience expectations and motivational elements, topic choice, understanding the question, organised ideas, L1 transfer and translating, generating ideas, the planning process, specific supporting details, effective conclusion and creative thinking ability. It has to be pointed out that law students are incompetent in their knowledge of vocabulary and grammar structure. In addition, the students encountered various difficulties ranging from planning the process to generating ideas. These difficulties were crucial barriers in writing essays in the Thai context. The results of this study were similar to the studies carried out by Lertpreedakorn (2009) and Ka-kan-dee and Kaur (2015), in which they stated that Thai EFL students are incompetent in areas ranging from grammar structure to putting together organised ideas. Similarly, Bennui (2008) reported that most Thai EFL university students are incompetent in their writing skills because of their lack of preparation in English writing and this problem arose due to inadequate performance in their classroom practice. Future research should examine the relationship between different types of writing performance among students.

The description and perspective of the difficulties drawn out by the writing performance promotes the idea that this research approach can be used feasibly as a diagnostic tool to gain valuable comprehension into students’ weaknesses. The overall quality of students’ writing performance supported understanding about the difficulties that Law students encountered with writing. In addition, the findings highlighted weaknesses in students’ ability about writing essays. With the reference to what was revealed by the students’ writing performance, this diagnostic tool has the potential to provide lecturers with useful information on how to make interpretations about their students’ ability writing.

Conclusion

The aim of this research study was to gauge the overall quality of students’ writing performance and identify the difficulties that Thai EFL English-major students faced when writing argumentative essays. The findings showed that providing helpful information about learners’ weaknesses in their writing tasks can raise awareness among EFL instructors and learners. This assessment can also be combined with other appropriate teaching approaches toward making a significant contribution to creating a detailed profile of students’ weaknesses in writing argumentative essays. Such valuable descriptions would be beneficial to EFL instructors in designing and developing their writing programmes and teaching effectively to further support learners’ writing development.

References