Intelligence in Question: A Grammatical and Textual

Analysis on AI-Generated English

and Filipino Texts

A Thesis

Presented to the Faculty of the College of Arts and Letters

Polytechnic University of the Philippines

Sta. Mesa, Manila

In Partial Fulfillment of the Requirements for the Degree

Bachelor of Arts in English Language Studies

Proponents:

Delos Santos, Danielle G.

Estabaya, Jamie Marie A.

Gatawa, Blythe G.

Gines, Jhuliana A.

Laran, Lorraine P.

Maranan, Jhay Ann D.

ABELS 3-3

2024
 CHAPTER 1

THE PROBLEM AND ITS SETTING

Introduction

In recent years, Artificial Intelligence (AI) has gained immense popularity

due to its versatility in different fields, particularly in writing where it has made

incredible strides in its ability to generate human-like text (Hurst, 2022).

Reflecting one of the globe’s significant developments, AI has the power to

automatically produce content in different formats and structures such as essays,

emails, marketing materials, and many more in just one click, making its name in

the top technological advances used by the current generation. In order to

operate that way, AI tools use algorithms that combine several methods including

language models, neural networks, and deep learning. A vast quantity of text

data analysis is what these algorithms use to analyze linguistic patterns and

structures, which permits them to produce responses that are contextually

relevant to the given topic or prompt. The patterns and structures found in the

text data that AI writing tools have been programmed on are what generate their

answers. Different sources such as books, papers, websites, and other written

materials can also be used to satisfy and fulfill this training data. By identifying

frequently used phrases, sentence structures, common idioms, popular sayings,

and vocabulary usage, the algorithms became knowledgeable enough to imitate

human writing, allowing them to generate and produce text that sounds natural.
However, beyond the huge potential of the said system is its possibility to make

flawed outputs, leading to issues concerning its function. These pave the way for

sprouting arguments regarding AI’s infallibility and the integrity of people using it.

Several concerns that arise with AI writing tools exhibit upright and

practical questions. Despite it being easy to use with real-time feedback and

readily accessible for consultations, one major problem is their precision; even

though these programs have the ability to produce writing with proper syntax,

most of the time, they have trouble comprehending context as according to

Noam Chomsky, their deepest flaw is the absence of the most critical capacity of

any intelligence, which then results in inaccurate matches with the topic. In a

guest essay published under The New York Times, the American linguist also

added that the use of such is “basically a high-tech plagiarism” for it does not

really create anything but rather copies existing works of existing artists and

modifies them enough to escape copyright laws — an ethical issue at birth

especially for those relying heavily on AI engines thinking it is mere intelligence

working to generate texts. The use of AI to fulfill writing assignments or any

academic paper falls under this concern as it may limit human creativity and

decrease originality. These points bring out a talk on academic dishonesty and

compromised learning which may hinder the development of critical thinking skills

among students if it totally replaces instead of enhancing their analytical

adroitness.
Producing biased text is another issue since AI writing tools can likely create one

by employing prejudices found in their training data. Information security and

user privacy are also flagged by the function of these tools, which makes privacy

a subject matter. Moreover, the growing use of AI writing tools endangers the

probability of employment displacement in some sectors such as writing and

content creation. In connection to this, Professor Franz Guiseppe Cortez, a

faculty member of the Department of Philosophy at the University of Santo

Tomas stated in an interview held by 24 Oras on the 29th of January 2024 that

“we can speak of the demise of thinking itself, the death of thinking, the death of

interpretation” if AI would hastily progress given the speed at which technology is

developing at present times. These issues highlight how important it is to

thoroughly analyze the moral and sociological ramifications of AI writing tools.

This research aims to provide information regarding the reliability of

Artificial Intelligence in generating texts, particularly essays in both English and

Filipino languages. The researchers will test three (3) AI tools namely ChatGPT,

Google Gemini, and You.com which ranked as the top three most visited AI

text-generating tools in a study by Sarkar (2024). Through the process of

text-generation experiment, the researchers will give writing prompts to the

aforementioned sites and their respective answers will be obtained as the data of

this study, which will then be checked for grammatical errors, as well as rating

their overall quality in terms of unity, coherence, and completeness.

 By the end of this research, a brief analysis of the experiment results will

be presented including a report on the performance of each AI tool used in the

study. Above anything else, the researchers’ primary objective is to raise

awareness and deliver a focal understanding of AI systems, their limits, and how

they should be utilized properly, especially for those using them in the writing

industry and the academe; to expand the view holistically in such a way that it

would be beneficial in the next generations, and would extend new avenues for

exploration to future researchers.

Statement of the Problem

This research seeks to examine the quality of the English and Filipino

essays produced by the world’s top three (3) most visited Artificial Intelligence

text-generating tools in terms of grammar and paragraph composition in

accordance with unity, coherence, and completeness. Furthermore, this study

aims to answer the following questions:

1. What are the responses of the AI text-generating tools ChatGPT, Google

Gemini, and You.com using the following prompts and requirements:

 Topic Sentence:

● How will technology change education?

● Paano babaguhin ng teknolohiya ang edukasyon?

Settings:

● Types of Paragraph:

○ Narrative

○ Cause and Effect

○ Descriptive

○ Comparison and Contrast

○ Argumentative

● Length: 1500 characters (excluding spaces)

● Number of Attempts: 1 (first response)

● Words to include: education, technology, research, students;

edukasyon, teknolohiya, pananaliksik, mga mag-aaral

2. What existing English and Filipino grammar rules were violated by the

AI-generated texts?

3. How effective are the AI-generated texts in terms of complying with the

following characteristics of a quality paragraph:

a.) Unity

b.) Coherence

c.) Completeness
 4. Based on the findings, what is the general impression on the quality of the

AI-generated texts in English and Filipino?

Theoretical Framework

The study will use Edward Owusu's Essay Analytical Framework (EAF) to

analyze the quality of the texts generated by the AI writing tools ChatGPT,

Google Gemini, and You.com. This framework (see Figure 1) consists of three (3)

main variables: the structure of the essay, the paragraph theory, and the methods

of development.

In the structure of the essay, the traditional types of paragraph introduction,

body, and conclusion, along with its basic components i.e. the topic sentence,

supporting sentences, and concluding sentence were presented and set as an

outline of the text.

For the second variable, the paragraph theory concerns the quality of the text

taking into account that there should be unity, coherence, and completeness for it

to be considered a good and effective paragraph. Asfah (2019) defined unity as a

characteristic where all sentences in a paragraph must be related, centered, and

harmonized on one main topic; while coherence pertains to the continuity of the

concrete and logical structure of the paragraphs.

 Owusu (2020) also added that coherence is a kind of relation that exists

between one entity in discourse and other elements that are vital to its

elucidation, helping to create a symbiotic relationship between grammatical

structures as the ideas were arranged in a manner that is clear, orderly, and

connected to one another. Another essential characteristic is completeness or

adequate development which refers to a satisfactory covering and substantiating

of the issues or ideas that a paragraph aims to address by including an adequate

number of evidence and details (Owusu, 2020).

Lastly, for the third variable, there exist five methods of development which

refer to the ways on how a paragraph can be written. In a study written by Owusu

and Yeboah in 2014, they featured the five existing methods of paragraph

development namely the narrative, cause and effect, descriptive, comparison and

contrast, and argumentative which this research is also planning to adopt.

 Figure 1. Essay Analytical Framework (EAF)

In addition, the researchers will consult the existing grammatical rules

conforming to the individual aspects of universal grammar in looking for possible

grammatical errors within the AI-generated texts, which is one reflection of their

credibility. This theoretical framework would serve as a guide on how the study

should be approached, particularly on the actions that must be done for the

purpose of obtaining both data from AI tools and the results of their analysis per

se.
Conceptual Framework

Figure 2. Research Paradigm

The Input-Process-Output (IPO) framework will be used in this research as

it provides a structured approach for examining the quality of AI-generated texts

in English and Filipino. The input includes the prompts for the top three (3) most

visited AI text-generating tools globally to write about. The researchers will gather

the texts, specifically the produced essays in both English and Filipino

languages, and a theoretical framework will then be integrated which would serve

as the guiding principle for the analysis of the aforementioned. These inputs will

provide a clear starting point for evaluating the performance of artificial

intelligence in the field of writing.

In the processing phase, a thorough examination of the data collected

from the top three AI text-generating tools will be conducted.

 This involves presentation, analysis, and interpretation of the generated

texts, focusing on their grammatical qualities and paragraph development. The

evaluation systematically assesses the credibility and identifies possible

deficiencies of artificial intelligence in writing mistake-free and high-quality texts.

Finally, at the output stage, the study culminates in the production of a

comprehensive report entitled "Intelligence in Question: A Textual and

Grammatical Analysis on AI-Generated English and Filipino Texts." This output

encapsulates our findings, shedding light on the limitations and potential

improvements of AI in generating texts in consideration of the language of

writing, thereby contributing to the ongoing discourse on its credibility and

general use in the industry. This framework ensures that the conclusions,

particularly on the implications of the results, were drawn in a methodical manner

and therefore critically deliberated.

Scope and Limitation of the Study

This study aims to analyze the textual and grammatical qualities of

AI-generated English and Filipino texts. The researchers will examine the quality

and reliability of the World’s Top 3 AI text-generating tools ChatGPT, Google

Gemini, and You.com, focusing on their whole performance in producing texts.

 A total of thirty (30) essays will be generated; fifteen (15) in English and

fifteen (15) in Filipino, with each of the three (3) tools having five (5) types of

paragraph to generate using the given prompt and requirements. One one (1)

topic sentence will be used for every of these paragraph types per language. All

the generated essays will be assessed by the researchers, to be checked by the

research validators.

The evaluation includes checking of possible grammatical error

occurrences, while the quality of the produced essays will be assessed and rated

in terms of unity, coherence, and completeness.

In analyzing the textual aspects of the input, the researchers will focus on

the text generation capabilities and respective quality of the selected AI tools and

will not extensively delve into other functionalities such as image recognition and

speech synthesis. The assessment will be limited to the Top 3 AI text-generating

tools based on current industry rankings and popularity. Other AI platforms not

included in this selection will not be evaluated in this study.

Significance of the Study

The result of this study will benefit the following groups:

ABELS Students. This study will help the English major students to know how

far they should ask for assistance from AI text-generating tools. The results of

this study provide insights on the limitations of AI-generated texts, helping them

discern why they should not depend on these tools alone and when to prioritize

human intervention in writing academic papers. Moreover, they can help in

improving the application or create a new application.

CAL Faculty. This would be beneficial to faculty staff, especially professors, as

they can utilize the findings of this study to educate students about the

importance of maintaining linguistic standards and creativity in their writing,

emphasizing the role of human intelligence alongside AI tools.

Writing Industry Professionals. This study will aid workers, particularly those in

the field of writing such as content creation and writing tasks, by raising

awareness about the potential pitfalls of relying solely on AI for producing texts

and the necessity of human oversight to ensure its quality. The study also does

not support the declining labor demands on human workforce and would promote

reshaping and redefining job roles rather than full employment displacement due

to lower operational costs brought by AI.

AI Users. Anyone capable of using AI text-generating tools, regardless of the

profession, is the general target of this research. As they gain a better

understanding of the capabilities and limitations of these tools, this study enables

them to make informed decisions about when and how to utilize AI assistance in

their writing endeavors.

Future Researchers. Future researchers will find value in this study as it

provides a foundational analysis of the grammatical and textual qualities of

AI-generated texts, guiding their investigations into improving the reliability and

accuracy of AI tools for text generation across different languages and contexts.

This study recognizes the convenience that Artificial Intelligence offers.

Nevertheless, the growing dependency on AI tools for generating textual content

is a point of contention as loopholes are within the bounds of possibility. Hence,

this study encourages everyone to understand how technology and humans

should work together, and that these tools offer only assistance; not a substitute

nor could outsmart real intelligence.

Definition of Terms

The following terms are conceptually and operationally defined to fit the

requirements of this study.

Artificial Intelligence (AI). This refers to the study that deals with all aspects of

mimicking cognitive functions for real-world problem solving and building systems

that learn and think like people (Holzinger et al., 2019).

AI Tools. This pertains to software applications or platforms that utilize artificial

intelligence and machine learning techniques to perform specific tasks or solve

particular problems (Shersiya, 2023). However, as this research will focus on AI

text-generating tools, the mention of the said term therefore pertains to those

platforms that produce texts, particularly the top three (3) most visited ones

ChatGPT, Google Gemini, and You.com.

AI-generated Texts. This refers to the texts produced by AI tools in reference to

the given prompts and requirements.

Grammatical Qualities. This refers to how the texts adhered to or violated

predefined grammatical rules.

Human Intelligence. This refers to the mental quality of real people that consists

of the abilities to learn from experience, adapt to new situations, understand and

handle abstract concepts, and use knowledge to control an environment

(Sternberg, 2024).

Prompt. This refers to something that assists one to move to action by

suggesting or saying the next words; this also serves as the inciting cause of

something as Merriam-Webster Dictionary defines it. This research integrates

similar meaning, but mainly focusing on the settings that will be given or posted

to the AI tools to generate texts.

Textual Qualities. This refers to how the texts complied with the qualities of a

good paragraph stated in the theoretical framework, covering the structure of the

essay, the paragraph theory, and the methods of development.

 CHAPTER 2

REVIEW OF RELATED LITERATURE

This chapter presents research studies that contain significant and relevant

information to the topic, gathered from different scholarly sources available on

the internet. The following are arranged thematically and are based on the key

concepts salient to this study.

Settings

The use of AI tools has sparked remarkably around the globe, with the

market even projected to reach a staggering $407 billion by 2027 (Forbes, 2024).

An annual growth rate of 37.3 percent between 2023 and 2030 is expected,

emphasizing the increasing impact of AI technologies in the coming years (ibid.).

This impact, along with other industries, can be seen in the field of writing as

there exists text-generating AI tools that work with any given prompt, whether in

the form of questions or commands. As the study will make use of AI tools to

acquire data, the researchers then employed certain prompts and requirements

for ChatGPT, Google Gemini, and You.com to write about, adopted from the

study conducted by Kumar and Mindzak in 2024, with some modifications. The

settings in their research are as follows:

 In this study, the researchers used the same prompt as well as the length

and words to include. However, the number of attempts was reduced to one and

the platforms used were chosen from the statistics of most visited AI tools

provided by Sarkar (2024). Since the research includes both English and Filipino

texts, the prompt and words to include were translated in Filipino in order for the

AI tools to generate essays in the aforementioned language. All generated texts

will undergo rule-based grammar checking and quality assessment whose results

would draw this study's conclusion.

Additionally, the researchers consulted Edward Owusu and Asuamah

Adade-Yeboah's “Thesis statement: A Vital Element in Expository Essays” (2014)

for the types of paragraphs that they would ask the AI tools to generate besides

the prompts and requirements.

The said study may not be directly related to this research as it mainly revolves

around the essence of using a thesis statement in expository essays, but a few of

its components contributed to the major details that the researchers could use as

bases to rate the quality of the texts that the AI tools will generate such as the

structure of an essay having the main parts introduction, body, and conclusion,

the elements of the paragraph theory namely unity, coherence, and

completeness, and the methods of development narrative, cause and effect,

descriptive, comparison and contrast, and argumentative. The generated texts

must adhere to these as part of the variables of the Essay Analytical Framework

that this study will employ, which was also cited from another research published

by Edward Owusu entitled “Are Paragraph Theories Array Of Impressions?” back

in 2019. Elaborating on the five methods of development, they refer to different

approaches or strategies of paragraph construction that have to be considered

upon the text generation experiment and textual analysis. Each are described as

follows:

● Cause and Effect: This method should explore the relationship between

events, actions, and/or phenomena, considering the result. Examining

AI-generated texts in this context allows the researchers and users to

assess how well these tools convey causal relationships grammatically.

 For instance, are the cause-and-effect relationships evident and rationally

organized? Do these relationships’ syntactic constructions include any

errors?

● Narrative: This involves telling a series of events in order to deliver a

message or make a point. By using this method to assess AI-generated

texts, the study can determine how well the AI text-generating tools

construct narratives and keep grammar and its quality consistent

throughout. For instance, are the events relevant to one another? Are the

events organized in chronological order? Are there no syntactic errors in

the narrative? Are there grammatical errors and typos in the narrative that

make it hard to understand and grammatically incorrect?

● Descriptive: This method of development will also be utilized in the study

to assess how effectively AI text-generating tools manage the complexities

of languages, specifically English and Filipino, used in the description. In

assessing the AI-generated texts with this method, it allows the study to

conclude if the tools’ proficiency in expressing descriptive language

grammatically is appropriate. This also includes questions such as: are the

sensory details effectively articulated? Is the descriptive language

syntactically correct and coherent? Are there any orthographic errors in

the description that take away from its vividness?

 ● Comparison and Contrast: This method works on analyzing similarities

and differences between two or more subjects. By evaluating

AI-generated texts that used comparison and contrast, the researchers or

the users may assess the tools’ accuracy in handling comparative

language and syntax. For instance, are comparative structures being used

correctly? Are there any grammatical flaws in the texts and do they

represent similarities and contrasts effectively?

● Argumentative: This method emphasizes on persuading the audience by

delivering a claim or thesis statement backed up by logic and evidence.

Analyzing AI-generated writings using this allows the study to evaluate the

AI text-generating tools’ grammatical argument construction skills. Do the

assertions make sense? Does the evidence follow grammar rules? Are

there any grammatical mistakes that make the argument less compelling?

All these will be checked in every generated text as they hold a significant

relevance in analyzing how ChatGPT, Google Gemini, and You.com respond to

different writing styles and structures which would reflect not just the practical

applications of AI text-generating tools, but also the general quality of

AI-generated texts in the context of effective automated content writing and

language processing.
AI Tools

Generative Artificial Intelligence (GAI) is a form of AI that can autonomously

generate new content, such as text, images, audio, and video (Lyu, 2023).

Examples of this are the chosen AI tools for this study, which are programs

trained on a set of data and learn the underlying patterns to generate new data

that mirrors the training set (Forbes, 2023). The selection process of these AI

tools involves the article “AI Industry Analysis: 50 Most Visited AI Tools and their

24B+ Traffic Behavior,” which provides a comprehensive analysis of AI

text-generating tools, detailing key trends, challenges, and ramifications for the

near future. It shows the significance of the extensive use of AI technologies in

various sectors, including healthcare, finance, and manufacturing, driven by the

necessity for automation, efficiency, and innovation. Moreover, the article

discusses the impact of AI on the job market. Despite AI's potential to create new

job opportunities and enhance productivity, it also raises concerns about job loss

as companies increasingly invest in AI initiatives, leading to a shortage of skilled

professionals with expertise in areas like machine learning, natural language

processing, and data science. The talent gap caused by this is expected to

continue unless addressed by seminars, proper education, and training

programs. Privacy concerns connected to the compilation and function of

personal data by AI systems were also highlighted, emphasizing the need for

ethical instructions and regulations.

Connected to this are moral considerations regarding the use of AI, particularly in

academia, which poses issues as demand gradually increases and the economy

slowly becomes AI-driven. This adoption is anticipated to hasten further, powered

by innovation and progress in AI research and development.

The researchers chose the mentioned AI tools based on the rankings

given in the aforementioned article. ChatGPT, a text-generating chatbot, leads

the list which automatically secures its place in this study. The second in ranking,

Quillbot, is an AI writing tool but its features do not include generating texts and

only serves as a writing assistant that paraphrases texts hence, the researchers

skipped the said tool and the others that do not fall under the category being

seeked upon until the proper one is found. The next selected tool then was

Google Gemini which placed top six on the list, and You.com on the twelfth spot

respectively.

Quality Analysis

The quality analysis of AI-generated texts is essential to understanding the

capabilities and limitations of current AI writing tools. This study utilizes Edward

Owusu's Essay Analytical Framework (EAF) to evaluate texts produced by

ChatGPT, Google Gemini, and You.com.

The EAF, comprising three main variables essay structure, paragraph theory, and

methods of development, provides a comprehensive approach to text analysis.

Owusu (2020) provided a brief description of these variables in which the first

one, the structure of the essay, examines the traditional components of an essay:

introduction, body, and conclusion. This structure is foundational in academic

writing, ensuring clarity and logical flow. Each section of the essay serves a

distinct purpose:

● Introduction: This section should provide a clear topic sentence that

introduces the main idea of the essay.

● Body Paragraphs: Each body paragraph should begin with a topic

sentence, followed by supporting sentences that provide evidence,

examples, or explanations, and conclude with a concluding sentence that

summarizes the paragraph’s main idea or transitions to the next one.

● Conclusion: The concluding paragraph should summarize the main points

of the essay and provide a final perspective on the topic.

For the assessment, the researchers will examine whether the

AI-generated essays adhere to this traditional structure. This involves ensuring

that each part is present and effectively constructed. The clarity and coherence

of the introductions will be evaluated to check if they exhibit a clear topic

sentence and thesis statement.

For the body, each paragraph is expected to begin with a topic sentence along

with well-organized supporting sentences and ends with a concluding sentence.

An effective summarization of the main points and a strong final perspective will

be looked upon the conclusions. This evaluation will help us determine the AI

tools' proficiency in generating structured and coherent essays. Adherence to the

traditional essay structure is not only a marker of good academic writing, but also

a key indicator of the AI's ability to understand and apply logical flow in writing.

By assessing these elements, strengths, weaknesses, and areas of improvement

of the AI-generated texts could be identified. This analysis will contribute to a

comprehensive understanding of the AI tools' capabilities and limitations, guiding

future developments in AI-assisted writing technologies and the use of AI in

general.

The second variable, paragraph theory, focuses on the internal quality of

the paragraphs. A good paragraph must exhibit unity, coherence, and

completeness in which for unity, all sentences within a paragraph must relate to a

single main idea, ensuring they are centered and harmonized. This characteristic

is crucial for maintaining focus and clarity. Coherence pertains to the logical flow

of ideas within a paragraph. This is achieved through a clear and orderly

arrangement of ideas, which helps create a symbiotic relationship between

grammatical structures and concepts, making the text easier to understand.

In completeness, a paragraph must satisfactorily cover the topic it addresses by

providing sufficient evidence and details. This means including an adequate

number of examples, explanations, and other supporting information. Using

these criteria, the researchers will evaluate the AI-generated texts to determine

their adherence to good writing practices and how well they conform with those.

Having a clear grasp of AI's ability to create unified, coherent, and complete

paragraphs is crucial for evaluating their overall writing proficiency and their

potential applications in various writing contexts.

The third variable involves the methods of paragraph development, which

refer to the different ways a paragraph can be written. According to Owusu and

Yeboah (2014), there exists five primary methods namely narrative which

involves telling a story or recounting events, cause and effect which explains the

reasons and results of a particular event or situation, descriptive which provides

vivid details to describe a person, place, thing, or idea, comparison and contrast

which highlights the similarities and differences between two or more items, and

argumentative which presents divergent views with reasoning. By specifying the

methods of development to use in the prompts,, the researchers would be able to

assess the appropriateness of the writing styles employed by each AI tool.

This analysis will reveal how well each AI tool adapts to different writing tasks

and genres, reflecting their flexibility and effectiveness in generating coherent

and contextually pertinent paragraphs, as well as producing texts that meet

various rhetorical and communicative purposes. For instance, an AI tool that

excels in narrative writing might be particularly useful for creative writing

applications, while one that is strong in argumentative writing might be better

suited for generating persuasive essays or reports. The diversity in paragraph

development methods also reflects the AI's capacity to handle different topics

and writing prompts. A tool that can effectively switch between descriptive and

analytical styles, for example, demonstrates a higher level of sophistication and

utility for a broader range of writing tasks. This versatility is crucial for

applications in education, content creation, and professional writing, where

different writing styles are often required. This, in turn, will help users select the

most appropriate AI tool for their specific needs and guide developers in refining

their models to better meet user demands. The results of the analyses would

also provide implications regarding the general use of AI text-generating tools,

which may vary depending on how users would accept the inferences.

In addition to the EAF, the researchers will consult a predefined set of

grammatical rules in English and Filipino in looking for possible grammatical

mistakes to encounter within each AI-generated essay, reflecting how they meet

the standards of proper writing.

 CHAPTER 3

METHODOLOGY

This section includes the processes done to fulfill the goals of the study, mainly

composed of the research design, sources of data, data gathering procedure,

and ethical considerations.

Research Design

Qualitative. This study employs a qualitative-descriptive research methodology,

as no substantial inferential statistics will be presented. The majority of the data,

processes, and results will require only textual explanation. This research is

qualitative because it addresses how and why we ought to evaluate any possible

gaps of AI rather than calculating numerical data points regarding the frequency

of the issue.

Descriptive. This research will employ a descriptive design that aims to describe

the text generation capabilities of the selected AI tools and the respective

qualities of the essays produced, which will be analyzed in terms of grammatical

aspects and paragraph elements using an established criteria for essay quality

assessment. This will reflect on the broader use of artificial intelligence in writing

by addressing the "what" research questions, classifying it as descriptive

research.
Sources of Data

The primary source of data for this research will be the essays, gathered

by asking the chosen AI tools to generate according to the given settings

consisting of the prompt or topic sentence, types of paragraph, length, number of

attempts, and specific words to include. The accumulated texts will then be

analyzed by first, listing down all the grammatical errors encountered in each

essay with the existing grammatical rules in English and Filipino serving as a

basis; and second, using a standardized rubric for evaluation which was adopted

from an existing study and modified to fit the specific needs of this study

particularly in terms of checking the essays’ levels of unity, coherence, and

completeness.

In addition, secondary data such as valuable literature and resources

concerning artificial intelligence discourses and effective text characteristics will

be sourced for better contextualization of the findings.

Data Gathering Procedure

For the data gathering procedure, the researchers plan to follow three

main processes namely the text generation experiment, rule-based grammar

checking method, and quality assessment.

By doing these, the researchers aim to systematically gather and analyze data to

understand the capabilities and limitations of various AI text-generating tools, as

well as the implications of the results in the general use of artificial intelligence.

Text Generation Experiment

To start with, the Text Generation Experiment includes the use of AI tools

to create essays based on a given topic sentence in both English and Filipino.

Parameters like paragraph type, length, number of attempts, and specific words

to include will be set. These prompts and requirements will be given to the AI

tools to generate essays, which will then be reviewed for grammatical errors and

overall quality. After that, the researchers will conduct a comparative analysis of

the generated texts to analyze the performance of different AI tools. Criteria such

as grammatical accuracy, vocabulary richness, and adherence to the settings will

be used to evaluate the texts. The researchers will then interpret and synthesize

the findings into a report. The interpretation will discuss the strengths and

weaknesses of each AI tool, provide insights into their effectiveness, and offer

recommendations for future research or practical applications.

Rule-based Grammar Checking Method

This method involves the application of predefined grammatical rules to

evaluate the texts produced during the Text Generation Experiment.

It allows an objective assessment on the performance of different AI

text-generating tools by using a consistent set aligned with both the universal

grammar of English and balarilang Filipino, which ensures that the interpretation

will be fair and unbiased. This process is crucial for the study as it helps to

determine the overall quality and reliability of the AI-generated texts. The insights

gained from this analysis will contribute to the understanding of the capabilities

and limitations of AI in text generation, guiding future improvements and

applications of these tools.

Screening is the initial step in looking for possible grammatical errors

within the essays. This covers checking for issues that include but are not limited

to subject-verb agreement, proper tense usage, punctuation, spelling, sentence

structure, and adherence to syntactic norms. Once the texts have been

screened, the next step is to tabulate the findings. This involves recording the

frequency and types of grammatical errors present in the generated texts. By

organizing the data in a structured format, we can perform a comparative

analysis of the AI tools. The tabulation will help in identifying patterns and trends

in the errors, providing insights into the strengths and weaknesses of each AI

tool's grammatical capabilities.

Quality Assessment

The generated essays will be evaluated using a rubric adopted from Agan

and Deniz (2019) in which each will be rated as poor, moderate, good, or very

good depending on how they adhered to the given characteristics. The said

rubric, which was developed for assessing paragraph level written texts, was

modified to align with the elements of the paragraph namely unity, coherence,

and completeness and fit with the goals of the study. A total of six (6) research

validators, three (3) from the English department and three (3) from the Filipino

department of the Polytechnic University of the Philippines - College of Arts and

Letters, will then be requested to check the assessment done by the researchers

as part of the research validation process. Furthermore, the findings will be

abridged by calculating the average percentage of the general quality of the

generated texts per language and the overall performance of the AI tools, as well

as the most abided type of paragraph. A clear interpretation of the results will be

presented in the succeeding chapters of this paper.

Ethical Considerations

For there are no human respondents needed in the study, consent forms

from the researchers are no longer necessary. Notwithstanding, the researchers

will maintain integrity, accuracy, fairness, and transparency over the details of the

gathered data, upholding the study's ethical worth.

 Accountability on the conclusions of this study is also guaranteed,

particularly on the implications of the results regarding the general use of AI in

the field of writing which enhances the validity and impact of the research in the

community. Effective communication will also be guaranteed between the

researchers and expert validators for feedback regarding the research findings.

Additionally, the researchers will try to reach out to the creators of the AI tools to

ask how they ensure the quality of its responses.

 REFERENCES

● Asfah, I. (2019). An analysis of paragraph writing abilities of English

Department students UNM - Neliti.

https://www.neliti.com/publications/345521/an-analysis-of-paragraph-writin

g-abilities-of-english-department-students-unm

● Chomsky, N., Roberts, I., & Watumull, J. (2023, March 8). Noam

Chomsky: The false promise of chatgpt. The New York Times.

https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.ht

ml

● Common english grammar mistakes: Oxford International english. Oxford

International English Schools. (2024, May 2).

https://www.oxfordinternationalenglish.com/common-english-grammar-mis

takes/

● Google. (2024, January 29). Teachers wary of possible AI use by some

students. Google.

https://www.google.com/amp/s/www.gmanetwork.com/news/scitech/techn

ology/895768/teachers-wary-of-possible-ai-use-by-some-students/story/%

3famp
● Haan, K. (2024, June 15). 24 top AI statistics and Trends in 2024. Forbes.

https://www.forbes.com/advisor/business/ai-statistics/

● Holzinger A;Langs G;Denk H;Zatloukal K;Müller H; (2019). Causability and

explainability of Artificial Intelligence in medicine. Wiley interdisciplinary

reviews. Data mining and knowledge discovery.

https://pubmed.ncbi.nlm.nih.gov/32089788/

● Hurst, L. (2022, November 8). AI writing is here, and it’s worryingly good.

Can writers and academia adapt? Euronews.

https://www.euronews.com/next/2022/11/08/ai-writing-is-here-and-its-worr

yingly-good-can-writers-and-academia-adapt

● Kumar, R., & Mindzak, M. (n.d.). Who wrote this? detecting artificial

intelligence–generated text from human-written text. Canadian

Perspectives on Academic Integrity.

https://journalhosting.ucalgary.ca/index.php/ai/article/view/77675

● Lyu, Z., & Generative artificial intelligence (AI) is a form of AI that can

autonomously generate new content. (2023, June 20). Generative Artificial

Intelligence in the metaverse era. Cognitive Robotics.

https://www.sciencedirect.com/science/article/pii/S2667241323000198
● Marr, B. (2024, February 20). The difference between Generative AI and

traditional AI: An easy explanation for anyone. Forbes.

https://www.forbes.com/sites/bernardmarr/2023/07/24/the-difference-betw

een-generative-ai-and-traditional-ai-an-easy-explanation-for-anyone/

● Nozima, B., & Nusratovich, K. U. (n.d.). Addressing common grammar

mistakes in EFL writing. PEDAGOG.

https://bestpublication.org/index.php/pedg/article/view/8715

● Owusu, E. (2019a, November 30). Are paragraph theories array of

impressions?. Language Teaching Research Quarterly.

https://eric.ed.gov/?id=EJ1270180

● Owusu, E., & Adade-Yeboah, A. (2014, January). (PDF) thesis statement:

A Vital Element in expository essays.

https://www.researchgate.net/publication/275202372_Thesis_Statement_

A_Vital_Element_in_Expository_Essays

● Sarkar, S. (n.d.). Ai Industry Analysis: 50 most visited AI tools and their

24B+ traffic behavior. WriterBuddy.

https://writerbuddy.ai/blog/ai-industry-analysis
● Shersiya, J. (2023, October 15). What is Artificial Intelligence (AI) tools?.

What is Artificial Intelligence (AI) Tools 2023?

https://www.linkedin.com/pulse/what-artificial-intelligence-ai-tools-jatin-uaq

kf

● Sternberg, R. (2024, April 5). Human intelligence. Encyclopædia

Britannica.

https://www.britannica.com/science/human-intelligence-psychology