Text Generation in Deep Learning with Keras: Fundamentals
Access all tutorials at https://www.muratkarakaya.net
Code: https://colab.research.google.com/dri...
Text Generation Play List: • Text Generation in Deep Learning with...
Text Generation in Deep Learning with Tensorflow & Keras: Fundamentals
This tutorial is the first part of the "Text Generation in Deep Learning" series.
We will cover all the topics related to Text Generation with sample implementations in Python Tensorflow Keras.
In this part, we will learn the Fundamentals of Text Generation in Deep Learning.
If you would like to learn more about Deep Learning with practical coding examples, please subscribe to Murat Karakaya Akademi YouTube Channel or follow my blog on Medium.
Do not forget to turn on Notifications so that you will be notified when new parts are uploaded.
You can access this Colab Notebook using the link given in the video description below.
Text Generation in Deep Learning with Tensorflow & Keras
Part A: Fundamentals
Part B: Tensorflow Data Pipeline for Character Level Text Generation
Part C: Tensorflow Data Pipeline for Word Level Text Generation
Part D: Recurrent Neural Notework (LSTM) Model for Character Level Text Generation
Part E: Encoder-Decoder Model for Character Level Text Generation
Part F: Recurrent Neural Netework (LSTM) Model for Word Level Text Generation
Part G: Encoder-Decoder Model for Word Level Text Generation
Character-level text generation with LSTM
Toward Controlled Generation of Text
Attention Is All You Need
Talk to Transformer
What is the difference between word-based and char-based text generation RNNs?
The survey: Text generation models in deep learning
Generative Adversarial Networks for Text Generation
FGGAN: Feature-Guiding Generative Adversarial Networks for Text Generation
How to sample from language models
How to generate text: using different decoding methods for language generation with Transformers
Hierarchical Neural Story Generation
What is Text Generation? Text generation is a subfield of natural language processing (NLP). It leverages knowledge in computational linguistics and artificial intelligence to automatically generate natural language texts, which can satisfy certain communicative requirements.
How it works?
In general:
We first train a Language Model (LM) with a corpus. LM learns the conditional probability distribution of the next token for a given token sequence from the corpus.
In text generation, LM works in a loop:
We provide initial text (seed) to LM.
LM calculates the conditional probabilities of the vocabulary items for the next token.
We sample the next token using this conditional probability distribution.
We concatenate this token to the seed and provide this sequence as the new seed to LM
Text Corpus
A corpus (plural corpora) or text corpus is a language resource consisting of a large and structured set of texts.
For example, the Amazon Review corpus consists of a few million Amazon customer reviews (input text) and star ratings (output labels) for learning how to train a Language Model for sentiment analysis.
Language Model
A statistical language model is a probability distribution over sequences of words.
Sampling
Sampling means randomly picking the next word according to its conditional probability distribution. After generating a probability distribution over vocabulary for the given input sequence, we need to carefully decide how to select the next token (sample) from this distribution.
There are several methods for sampling in text generation (see here and here):
Greedy Search: Greedy search simply selects the word with the highest probability as its next word. However, if we always sample the most likely word, the standard language model training objective causes us to get stuck in loops like “I don’t know. I don’t know. I don’t know.”
Beam Search: Beam search reduces the risk of missing hidden high probability word sequences by keeping the most likely num_beams of hypotheses at each time step and eventually choosing the hypothesis that has the overall highest probability. For details check this blog.
Beam search will always find an output sequence with a higher probability than greedy search but is not guaranteed to find the most likely output.
Смотрите видео Text Generation in Deep Learning with Keras: Fundamentals онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь Murat Karakaya Akademi 01 Январь 1970, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 2,806 раз и оно понравилось 46 людям.