A Brief Story of Generative AI

05. October 2023, Dmitry Orlov

Welcome to our blog, where we explain what Generative AI is and how it differs from AI, ML, and Deep Learning. You will learn about its brief history and the best tools for text, image, video, voice, and music generation. Get ready for an insightful journey into this transformative technology.

Generative Ai Whitepaper

Go to download >

Introduction
Terminology of Generative AI
A Brief Story of Generative AI
Summary
References

Introduction

Data — ChatGPT Shoots Past One Million User Mark at Lightning Speed ^[1]

With the launch of ChatGPT, the technological arena experienced an unprecedented shift. The acceptance of this generative AI tool was meteoric, outpacing even the likes of Facebook and Instagram. Such rapid adoption across various applications underscored ChatGPT's revolutionary potential. However, this was only the initial wave. Riding on its momentum, a plethora of generative AI solutions followed suit. While products like DALL·E and Midjourney are lauded for their image generation capabilities, others like Jasper (an AI content generator), Jukebox (a music-generating AI), Synthesia (AI-driven video creation), and StyleGAN (a facial image synthesiser) have also seized significant attention. As the boundaries of digital possibilities continue to expand, the suite of AI tools grows correspondingly diverse, with each solution carving a unique niche in this rapidly evolving ecosystem. Moreover, as reflected in the Gartner Hype Cycle, generative AI is on a trajectory that showcases its evolving maturity and growing potential, further emphasising the need for enterprises to recognize and leverage its capabilities.

However, in the intricate realm of enterprise, marked by its complex business processes and operations, the rapid adoption of emerging technologies is often met with hesitation. Despite traditionally being a stronghold of innovation, the sector exhibits caution when confronted with recent advances in machine learning and AI. For example, a pertinent observation from a survey by the European Commission^[3] underscores this sentiment: while 42% of enterprises utilise at least one AI technology, an equally significant 40% remain detached, with neither current usage nor future plans for AI integration. This reluctance often stems from barriers like skill gaps, prohibitive costs, and concerns over data quality. Within these complex enterprise ecosystems, the pressing queries persist:

How can Generative AI be seamlessly integrated into existing operations?
Where are the tangible business benefits, and how does one quantify the return on investment?

Terminology of Generative AI

At first let’s check what are the differences between Artificial Intelligence (AI), Machine Learning (ML), Deep Learning, Generative AI and how they are related to each other.

Artificial Intelligence (AI) is the broader concept of creating machines that mimic human intelligence to perform various tasks.
Machine Learning (ML) is a subset of AI focused on enabling machines to learn and make decisions from data without explicit programming.
Deep Learning is a specialised form of ML that uses deep neural networks to understand complex patterns, especially in tasks like image and speech recognition.
Generative AI is a specific application of ML where models create new content, such as text or images, based on patterns learned from data. It's used in creative tasks like art and content generation. Usually it is a subset of Deep Learning, but not always.

On the next figure, is shown how Generative AI is related to other terms of Artificial Intelligence:

Now let’s consider a more detailed definition of Generative AI.

Generative AI is a subfield of artificial intelligence capable of generating text, images, or other media in response to prompts. It is focused on developing algorithms and models that have the ability to generate original data and resembles human-created content. It is characterised by its capacity to produce novel output based on patterns and information learned from a given dataset, without duplicating that data.

One of the breakthroughs with generative AI models is the ability to leverage different learning approaches, including unsupervised or semi-supervised learning for training. This has given organisations the ability to more easily and quickly leverage a large amount of unlabeled data to create foundation models.^[5]

Key components and characteristics of generative AI include:

Learning from Data: Generative AI models learn from a large dataset, extracting patterns, relationships, and structures present within the data.
Diversity and Creativity: Instead of copying existing examples, it aims to create variations and novel instances that expand upon the input data.
Unimodal or multimodal: Unimodal systems take only one type of input, whereas multimodal systems can take more than one type of input (Text, Code, Images, Music, Video, etc.).
Conditional and Unconditional Generation: In conditional generation, it produces output based on specific input conditions or constraints, such as generating a caption for an image. In unconditional generation, it generates content without any specific input.
Training and Fine-Tuning: Generative AI models require extensive training on large datasets. The training process involves adjusting model parameters to minimise the difference between generated output and real data.

A Brief Story of Generative AI

Early days: What led to the emergence of Generative AI?

In the 1950s, AI moved from sci-fi to reality as we developed capable electronic computers. Researcher Alan Turing explored the maths behind AI, suggesting that machines, like humans, could use information and reasoning to solve problems. He introduced these ideas in his 1950 paper, "Computing Machinery and Intelligence," where he discussed intelligent machines and proposed the Turing Test. The test posits that if a machine can carry on a conversation (over a text interface) that is indistinguishable from a conversation with a human being, then it is reasonable to say that the machine is "thinking." Using this simplified test, it is easier to argue that a "thinking machine" is at least plausible.

One of the first examples of GenAI was the Markov Chain, a statistical model that could be used to generate new sequences of data based on input. However, computers in the 1950s were very limited and expensive, restricting AI research.

During the 1980s, neural networks gained popularity for data generation, with Geoffrey Hinton's "Boltzmann Machines" being a notable contribution. This decade also witnessed the onset of the AI Winter, a period of significant slowdown in AI research and development, following a decade of substantial progress in the field.

Generative AI evolved into its current form around 2006, marked by Geoffrey Hinton's influential paper "A Fast Learning Algorithm for Deep Belief Nets," reintroducing Restricted Boltzmann Machines initially introduced in 1983.

Subsequently, progress was limited until 2014 when Ian Goodfellow introduced Generative Adversarial Networks (GANs). In the ensuing years, significant strides were made, including the introduction of the transformer architecture for natural language processing in the paper "Attention is all you Need" by Vaswani and colleagues at Google in 2017.

The advancements, such as variational autoencoders and generative adversarial networks, paved the way for practical deep neural networks capable of learning generative models for complex data like images. These deep generative models were revolutionary as they could generate not only class labels but entire images.

While generative AI largely went unnoticed by the public until 2022, it gained widespread attention as technology became accessible to consumers. This shift was facilitated by the launch of various text-to-image model services, including MidJourney, DALL-E 2, Imagen, and the open-source release of Stability AI's Stable Diffusion. Soon after, OpenAI introduced ChatGPT, a version of GPT-3 re-trained on conversational dialogue, captivating users with its comprehensive responses delivered in a remarkably human-like manner.^[6]

Milestone breakthroughs: Key developments in recent years

Here are the main milestones of Generative AI development in recent years:

The rise of GPT, DALL-E, Midjourney and other models: how they have changed the landscape

The field of Generative AI has undergone a remarkable transformation in recent years with the advent of groundbreaking models such as GPT (Generative Pre-trained Transformer), DALL-E, Midjourney, and many others. These models have demonstrated unprecedented capabilities in generating text, images, and even bridging the gap between modalities, ushering in a new era of generative AI.

GPT

OpenAI's GPT series has become synonymous with natural language generation. Starting with GPT-1, these models are trained on massive datasets, enabling them to generate human-like text with remarkable coherence and context-awareness. The release of GPT-3 marked a turning point, showcasing the ability to understand and generate text across multiple languages and domains. GPT-3's wide-ranging applications include chatbots, content generation, and language translation, revolutionising the way we interact with AI-powered systems.

2018: GPT-1 (117 million parameters) showcased human-like text generation capabilities.
2019: GPT-2 (1.5 billion parameters) producing more coherent and extensive text. It was able to generate harmful content like fake news.
2020: GPT-3 (175 billion parameters) could produce text nearly identical to human writing.
2022: GPT-3.5 Turbo (ChatGPT) was a major advancement in natural language processing. It could participate in human-like conversations and effectively respond to natural language queries.
2023: GPT-4 (1.8 trillion parameters) can process both text and images as input.
2024: GPT-5 will aim to achieve AGI (Artificial General Intelligence) – an artificial intelligence capable of performing a variety of tasks as well or even better than a human.

The evolution of GPT language models has profoundly influenced natural language processing and various text-based applications. Nevertheless, there are ethical concerns associated with these models, notably their capacity to produce fake news and misleading content.^[8]

DALL-E

DALL-E, also developed by OpenAI, takes generative AI to the next level by bridging the gap between text and images. This model can create images from textual descriptions, turning words like "a two-story pink house shaped like a shoe" into stunning visual representations. DALL-E's potential applications span from art and design to content creation, where it can generate illustrations and visual content from textual inputs, unlocking a world of creative possibilities.

DALL-E excels in various image styles, creatively arranging objects without explicit instructions. It adapts to design trends, handles different descriptions, and can solve Raven's Matrices (visual tests often administered to humans to measure intelligence).^[10]

On July 20, 2022, DALL-E 2 entered beta, initially limited to select users for ethical concerns. On September 28, 2022, it became open to all. In Oct. 2023, OpenAI announced the availability of DALL-E within the ChatGPT Plus interface.

Midjourney

Midjourney stands out for its dream-like artistic style, contrasting with competitors like DALL-E 2. It creates both realistic and expressive images, making it ideal for Sci-Fi and Gothic themes, emphasising its painting-like approach.

Its strength lies in generating highly relevant images based on your preferences for lighting, style, orientation, and colours. Midjourney excels in pure image generation, potentially ranking as the best generative AI tool.

Moreover, you can upload and modify your own images, altering backgrounds, outfit colours, and creating caricatures. Midjourney offers higher resolution upscaling than other AI art generators and extends image boundaries through its outpainting feature.

Its versatile applications span personal artworks, commercial illustrations, educational content, marketing materials, and innovative art forms. Notably, artist Jason Allen made history by winning a prize in the Digital Arts and Photography category using Midjourney for his piece, "Theatre D’Opera Spatial."^[11]

Other Tools

Generative AI is reshaping industries by providing advanced solutions in various domains. Here's a breakdown of its applications and popular tools in video, voice, music, and text generation:

Video Generation
- Applications: Realistic content synthesis; language-to-video synthesis; video inpainting; style transfer; data augmentation; AI animated avatars.
- Concerns: Potential misuse, such as deepfakes.
- Popular Tools: Pictory, Synthesia, Deepbrain AI, Elai, Neural Frames.
Voice Generation
- Applications: Virtual assistants; text-to-speech; language translation; entertainment voiceovers; interactive storytelling.
- Concerns: Potential misuse, such as deepfake voice attacks.
- Popular Tools: Lovo, Synthesys, Voice Over by Speechify, Listnr.
Music Generation
- Application: Assists musicians in generating new music and offers personal compositions based on tastes.
- Popular Tools: Amper Music, AIVA, Ecrett Music, Boomy, WavTool.
Text Generation
- Applications: Content creation; automated reporting; chatbots; language translation; content personalization; idea generation; educational content; legal documentation; content moderation; journalism.
- Concerns: Ethical considerations; importance of refining AI-generated text.
- Popular Tools: Jasper, Copy.ai, Anyword, Writesonic, Sudowrite.

Summary

The rise of generative AI models and tools marks a revolution in the AI landscape. These models have expanded the boundaries of what AI can achieve in terms of natural language generation, image synthesis, and multimodal capabilities. Their impact across industries is profound, offering new opportunities for creativity, productivity, and accessibility. However, as we embrace these transformative technologies, it is essential to remain vigilant and address the ethical challenges they pose, ensuring that the benefits of generative AI are harnessed responsibly for the betterment of society.

A Brief Story of Generative AI

Introduction

Terminology of Generative AI

A Brief Story of Generative AI

Early days: What led to the emergence of Generative AI?

Milestone breakthroughs: Key developments in recent years

The rise of GPT, DALL-E, Midjourney and other models: how they have changed the landscape

GPT

DALL-E

Midjourney

Other Tools

Summary

"Enterprise AI in Action" Whitepaper

Thanks! Here is your download-link:

References

Recommended