Scribd
Upload
46 views

BERT

BERT stands for Bidirectional Encoder Representations from Transformers, an open-source machine learning framework for natural language processing. BERT is pre-trained on large datasets like Wikipedia to understand word relationships. While useful, BERT has disadvantages like being slow to train and not tailored for specific tasks or local datasets. Therefore, this document discusses examining ChatGPT's framework and proposing a modified pre-trained model through reverse engineering to address these issues for local use cases.

Uploaded by

ice queen
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
46 views

BERT

BERT stands for Bidirectional Encoder Representations from Transformers, an open-source machine learning framework for natural language processing. BERT is pre-trained on large datasets like Wikipedia to understand word relationships. While useful, BERT has disadvantages like being slow to train and not tailored for specific tasks or local datasets. Therefore, this document discusses examining ChatGPT's framework and proposing a modified pre-trained model through reverse engineering to address these issues for local use cases.

Uploaded by

ice queen
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 1

 BERT stands for Bidirectional Encoder Representations from Transformers, which is an open-

source machine learning framework for natural language processing (NLP). It is designed to help
computers understand the meaning of language by processing any given word in relation to all
other words in a sentence or phrase.
https://www.techtarget.com/searchenterpriseai/definition/BERT-language-model
 BERT is based on the transformer architecture, which includes two separate mechanisms - an
encoder that reads the text input and a decoder that produces a prediction for the task.
https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-
f8b21a9b6270
 BERT is pre-trained on large datasets such as English Wikipedia and can be fine-tuned with fewer
resources on smaller datasets to optimize its performance on specific NLP tasks.
https://en.wikipedia.org/wiki/BERT_(language_model)
 The disadvantage of BERT is its size, which can make it difficult to train on smaller datasets.
Another disadvantage is that it is slow to train because it is big and there are a lot of weights to
update. https://www.projectpro.io/article/bert-nlp-model-explained/558
 BERT is designed to be input into other systems, not a standalone program, and because of that,
it must be fine-tuned for downstream tasks, which can be fussy.
https://www.projectpro.io/article/bert-nlp-model-explained/558
 BERT is similar to what is used in ChatGPT.
 ChatGPT uses massive pretrained model. It is global, it has all the corpuses from google which is
very massive and adds ambiguity when used for local purposes.
 ChatGPT works on knowledge graphs and reasoning. It is very sophisticated and massive. Its
knowledge can be misleading.
 Since BERT is not trained on private local data i.e. local data of Khasam Sohar, there is still a need
for local transformation.
 Therefore, we aim to discuss the framework of ChatGPT and propose an altered pretrained
model by reverse engineering.

Global pre-trained models like BERT are becoming increasingly popular recently for natural language
processing (NLP). Applications of such models range from sentiment analysis to automated question-
answering chatbots like ChatGPT. BERT could appear to be practical in many contexts as it is trained on a
massive dataset and works on knowledge graphs and reasoning. But the sophisticated knowledge
generated by these models generates ambiguity which is particularly not beneficial for local data.
Therefore, this paper aims to focus on the need for a local transformation while examining the
framework of existing pre-trained models and discussing the recommendation system in this context and
at the same time we propose a solution architecture to tackle this issue.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy