Generative pre-trained transformers (GPT)

Generative pre-trained transformers (GPT) are a recent development in artificial intelligence (AI). They are specialized AI models that are pretrained using massive quantities of human-produced media, integrating it into complex neural networks. Once pretrained, GPTs are capable of quickly and easily researching and producing large quantities of content. They can also convincingly converse with people and gradually improve their own neural networks. The first commercially successful GPT, OpenAI’s ChatGPT, launched in 2018. The model was quickly adopted by countless businesses and consumers, becoming an immediate commercial success. However, some activists criticized it for its potential to plagiarize work and eliminate large numbers of jobs.

Background

Artificial intelligence (AI) refers to intelligent machines, most commonly intelligent computer programs. AI can carry out tasks with limited human input, often with apparent elements of creativity. Though they were once considered science fiction, AI computer programs are now used by businesses throughout the world. Many AI programs are better than people at analyzing large datasets and finding new patterns. In the business field, they might be used to recognize new customer trends or streamline supply chains. In the medical field, they may be used to help doctors diagnose patients or find new treatments for various diseases. Some AIs are now capable of generating visual art, music, and large quantities of text based on pre-recognized patterns and structures.

AI programs are commonly broken down into three categories: narrow AIs, general AIs, and super AIs. Narrow AIs refer to computer programs that perform a single task intelligently. They may be trained to accomplish new tasks but are not capable of truly learning on their own. General AIs are capable of learning new skills and applying them on their own, often in new and innovative ways. Super AIs are AIs that surpass humans in many categories. Scientists worry that super AIs might one day become much more intelligent than human scientists. However, the most advanced modern AI programs are complex narrow AIs. Though many research firms are working toward developing a true general AI, none has succeeded. As general AIs have yet to be developed, super AIs do not exist.

Overview

Generative pre-trained transformers, commonly referred to as GPT, are a subtype of AI pioneered by the company OpenAI, which released the first public GPT in 2018. GPTs are created using a large-scale neural network model, a type of machine-learning algorithm that utilizes a structure inspired by the human brain. Neural networks are composed of interconnected units known as nodes or artificial neurons that are further organized into layers. This allows the application to create categorical connections between pieces of information. As these connections are refined, the neural network is improved.

GPT models quickly became famous for producing large quantities of text, images, media, and music based on human prompts. These programs were able to converse in a human-like manner, leading some researchers to claim that GPT models had passed the landmark Turing Test for AI. Most notably, GPT models can process information much more quickly than a human mind. It may take a human student several hours to research and write a report on a new topic. However, a GPT model can scan its information databases for the necessary research, analyze the constraints set by the assignment, and produce a coherent report in just seconds.

GPT models distinguish themselves from other generative AI models by their ability to pre-train on specific languages or tasks. During this process, the model is allowed to scan massive quantities of data regarding a particular topic. This can include books, articles, websites, transcripts, and even visual media. This information is carefully integrated into the program’s neural network before it begins receiving prompts from users. Software engineers can then begin fine-tuning the model by teaching it which responses are correct and which are incorrect. Through this process, the GPT model can quickly be taught to complete new tasks.

Because GPT models are general-purpose language models, they can be adapted to a large variety of tasks. Many businesses use GPT models to quickly create a social media presence for their enterprise. GPT models can mimic the tone of specific types of human writing, allowing them to tailor the language of the social media posts to the business’s target audience. Similarly, GPT models may be used to quickly generate effective web copy.

The process for training a GPT model on traditional languages is like training a GPT model on programming languages. For this reason, many modern GPT programs can produce computer programming as quickly as they can produce copy. Human users provide prompts for the service, requesting code that performs a specific task within a language.

OpenAI released its flagship GPT model, ChatGPT, in 2018. The model has since been updated to GPT-3, an improved version of the original model. Upon its release, ChatGPT achieved significant media attention for its ability to easily imitate a human conversation. Additionally, the model was praised for its ability to quickly and accurately fulfill most tasks submitted by users. Critics and scientists noted that ChatGPT and similar GPT models might eventually have a negative effect on the economy. The AI model could quickly and accurately research, compose text, create code, and converse with humans, making it a genuine threat to countless jobs. Many online media outlets reduced the size of their staff, instead utilizing GPT models to generate the bulk of their copy. Additionally, both students and employees began using ChatGPT to fulfill their assignments against the wishes of their teachers and employers.

Some critics allege that the work produced by GPT models should be considered plagiarism. They argue that GPT models are incapable of truly creating work and instead closely imitate or copy information upon which they were pre-trained. Because of this, some courts ruled that AI-generated content could not be copyrighted.

Bibliography

“7 Types of Artificial Intelligence That You Should Know in 2023.” Simplilearn, 2023, www.simplilearn.com/tutorials/artificial-intelligence-tutorial/types-of-artificial-intelligence. Accessed 23 Aug. 2023.

“AI Cannot Hold Copyright, Federal Judge Rules.” Politico, 2023, www.politico.com/news/2023/08/21/ai-cannot-hold-copyright-federal-judge-rules-00111865#:~:text=Friday's%20ruling%20will%20be%20a%20critical%20component%20in%20future%20legal%20fights.&text=Artificial%20intelligence%20cannot%20hold%20a,a%20federal%20judge%20ruled%20Friday. Accessed 23 Aug. 2023.

“Generative Pre-Trained Transformer (GPT).” Encord, 2023, encord.com/glossary/gpt-definition/. Accessed 23 Aug. 2023.

“Neural Networks.” Encord, 2023, encord.com/glossary/neural-networks-definition/. Accessed 23 Aug. 2023.

“What Is GPT?” Amazon Web Services, 2023, aws.amazon.com/what-is/gpt/. Accessed 23 Aug. 2023.