Natural Language Processing (NLP)

Youtube search... | Quora search... ...Google search

Speech recognition, (speech) translation, understanding (semantic parsing) complete sentences, understanding synonyms of matching words, sentiment analysis, and writing/generating complete grammatically correct sentences and paragraphs.

• Outline of natural language processing | Wikipedia
• Natural Language Processing | Wikipedia
• Grammar Induction | Wikipedia
• Natural Language Tools & Services
• Problem-solving with ML: automatic document classification | Ahmed Kachkach
• Average-Stochastic Gradient Descent (SGD) Weight-Dropped LSTM (AWD-LSTM)
• The Stanford Natural Language Inference (SNLI) Corpus
• How do I learn Natural Language Processing? | Sanket Gupta
• Natural Language | Chris Umbel
• NLP News | Sebastian Ruder
• Language services | Cognitive Services | Microsoft Azure
• Extracting Information from Text | NLTK.org
• Text Transfer Learning

• Courses:
  • Natural Language Processing with Deep Learning | Stanford
  • Natural Language Processing | Higher School of Economics
  • Deep Learning for Natural Language Processing | Oxford - Phil Blunsom
  • Getting Started with Natural Language Processing with Python | Swetha Kolalapudi
  • Natural Language Processing (NLP) Microsoft | Lei Ma, Roland Fernandez, Xiaodong He
  • NVIDIA Deep Learning Institute

Workbench / Pipeline

Youtube search... ...Google search

• Natural Language Tools:
• Outline of natural language processing | Wikipedia
• CogComp NLP Pipeline | Cognitive Computation Group, led by Prof. Dan Roth
• Building Custom Text Analytics solutions using Azure Machine Learning (AML)
• H2O Driverless AI

Text Preprocessing

Cleaning and preparation the information for use, such as punctuation removal, spelling correction, lowercasing, stripping markup tags (HTML,XML)

Regular Expressions (Regex)

Youtube search... ...Google search

• Breaking the Ice with Regular Expressions | Code Schol

Search for text patterns, validate emails and URLs, capture information, and use patterns to save development time.

Tokenization / Sentence Splitting

Youtube search... ...Google search

• Bag-of-Words (scikit-learn: Count Vectorizer)
• Term Frequency, Inverse Document Frequency (tf-idf)
• Ngram Viewer | Google Books
  • Ngram Viewer Info

Tokenization is the process of demarcating (breaking text into individual words) and possibly classifying sections of a string of input characters. The resulting tokens are then passed on to some other form of processing. The process can be considered a sub-task of parsing input. A token (or n-gram) is a contiguous sequence of n items from a given sample of text or speech. The items can be phonemes, syllables, letters, words or base pairs according to the application.

Stop Words

Youtube search... ...Google search

• Removing stop words with NLTK in Python | GeeksforGeeks

One of the major forms of pre-processing is to filter out useless data. In natural language processing, useless words (data), are referred to as stop words. A stop word is a commonly used word (such as “the”, “a”, “an”, “in”) that a search engine has been programmed to ignore, both when indexing entries for searching and when retrieving them as the result of a search query.

Stemming (Morphological Similarity)

Youtube search... ...Google search

Refers to a crude heuristic process that chops off the ends of words in the hope of achieving this goal correctly most of the time, and often includes the removal of derivational affixes.

Lemmatization

Youtube search... ...Google search

Lemmatization usually refers to doing things properly with the use of a vocabulary and morphological analysis of words, normally aiming to remove inflectional endings only and to return the base or dictionary form of a word, which is known as the lemma . If confronted with the token saw, stemming might return just s, whereas lemmatization would attempt to return either see or saw depending on whether the use of the token was as a verb or a noun. The two may also differ in that stemming most commonly collapses derivationally related words, whereas lemmatization commonly only collapses the different inflectional forms of a lemma. Stemming and lemmatization | Stanford.edu NLTK's lemmatizer knows "am" and "are" are related to "be."

Parsing/Relating Text

Understanding how the words relate to each other and the underlying grammar by segmenting the sentences syntax

Part-of-Speech (POS) Tagging

Youtube search... ...Google search

(POST), also called grammatical tagging or word-category disambiguation, is the process of marking up a word in a text (corpus) as corresponding to a particular part of speech,[1] based on both its definition and its context—i.e., its relationship with adjacent and related words in a phrase, sentence, or paragraph. A simplified form of this is commonly taught to school-age children, in the identification of words as nouns, verbs, adjectives, adverbs, etc.

Chunking

Youtube search... ...Google search

The Hierarchy of Ideas (also known as chunking) is a linguistic tool used in NLP that allows the speaker to traverse the realms of abstract to specific easily and effortlessly. When we speak or think we use words that indicate how abstract, or how detailed we are in processing the information. In general, as human beings our brain is quite good at chunking information together in order to make it easier for us to process and simpler to understand. Thinking about the word “learning” for example is much simpler that thinking about all the different things that we could be learning about. When we memorise a telephone number or any other sequence of numbers we do not tend to memorise them as separate individual numbers, we group them together to make them easier to remember. Hierarchy of Ideas or Chunking in NLP | Excellence Assured

Chinking

Youtube search... ...Google search

• Chinking with NLTK | Discord

The process of removing a sequence of tokens from a chunk. If the matching sequence of tokens spans an entire chunk, then the whole chunk is removed; if the sequence of tokens appears in the middle of the chunk, these tokens are removed, leaving two chunks where there was only one before. If the sequence is at the periphery of the chunk, these tokens are removed, and a smaller chunk remains.

Named Entity Recognition (NER)

Youtube search... ...Google search

• NERClassifierCombiner | Stanford CoreNLP

(also known as entity identification, entity chunking and entity extraction) is a subtask of information extraction that seeks to locate and classify named entities in text into pre-defined categories such as the names of persons, organizations, locations, expressions of times, quantities, monetary values, percentages, etc. Most research on NER systems has been structured as taking an unannotated block of text, and producing an annotated block of text that highlights the names of entities.

Neural Coreference

Youtube search... ...Google search

• Neural Coreference

Coreference is the fact that two or more expressions in a text – like pronouns or nouns – link to the same person or thing. It is a classical Natural language processing task, that has seen a revival of interest in the past two years as several research groups applied cutting-edge deep-learning and reinforcement-learning techniques to it. It is also one of the key building blocks to building conversational Artificial intelligences.

Hierarchical Classifier

Youtube search... ...Google search

• A Survey of Hierarchical Classification Across Different Application Domains | Carlos N. Silla Jr. and Alex A. Freitas
• Hierarchical Classification – a useful approach for predicting thousands of possible categories | Pedro Chaves - KDnuggets

Classification approaches:

• Flat - there is no inherent hierarchy between the possible categories the data can belong to (or we chose to ignore it). Train either a single classifier to predict all of the available classes or one classifier per category (1 vs All)
• Hierarchically - organizing the classes, creating a tree or DAG (Directed Acyclic Graph) of categories, exploiting the information on relationships among them. Although there are different types of hierarchical classification approaches, the difference between both modes of reasoning and analysing are particularly easy to understand in these illustrations, taken from a great review on the subject by Silla and Freitas (2011). Taking a top-down approach, training a classifier per level (or node) of the tree (again, although this is not the only hierarchical approach, it is definitely the most widely used and the one we’ve selected for our problem at hands), where a given decision will lead us down a different classification path.

Corpora

Youtube search... ...Google search

• Google Books Corpus
• Oxford English Corpus

A corpus (plural corpora) or text corpus is a large and structured set of texts (nowadays usually electronically stored and processed). In corpus linguistics, they are used to do statistical analysis and hypothesis testing, checking occurrences or validating linguistic rules within a specific language territory.

Topic Modeling

Youtube search... ...Google search

• Doc2Vec
• Latent Dirichlet Allocation (LDA)

A type of statistical modeling for discovering the abstract “topics” that occur in a collection of documents. Latent Dirichlet Allocation (LDA) is an example of topic model and is used to classify text in a document to a particular topic

Word Embeddings

Youtube search... ...Google search

• Word2Vec
• Sense2Vec | Matthew Honnibal
• Global Vectors for Word Representation (GloVe)
• Word Embeddings Demo
• Representation Learning

The collective name for a set of language modeling and feature learning techniques in natural language processing (NLP) where words or phrases from the vocabulary are mapped to vectors of real numbers.

Ontologies

Youtube search... ...Google search

(aka knowledge graph) can incorporate computable descriptions that can bring insight in a wide set of compelling applications including more precise knowledge capture, semantic data integration, sophisticated query answering, and powerful association mining - thereby delivering key value for health care and the life sciences.

Natural Language Inference (NLI) and Recognizing Textual Entailment (RTE)

Youtube search... ...Google search

Identifying whether one piece of text can be plausibly inferred from another - automatic acquisition of paraphrases, lexical semantic relationships, inference methods, knowledge representations for applications such as question answering, information extraction and summarization.

https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcR8BEISn7dsKtnyHEKoVqydi8-SkuYE6tFAiJNe57lnqiy__lF2

Semantic Role Labeling (SRL)

Youtube search... ...Google search

• Illinois Semantic Role Labeler (SRL) | Cognitive Computation Group
  • Semantic Role Labeling Demo

identifies shallow semantic information in a given sentence. The tool labels verb-argument structure, identifying who did what to whom by assigning roles that indicate the agent, patient, and theme of each verb to constituents of the sentence representing entities related by the verb.

Deep Learning Algorithms

Youtube search... ...Google search

• Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Recurrent Neural Network (RNN)
• Deep Q Learning (DQN)
• 7 types of Artificial Neural Networks for Natural Language Processing
• Combination of Convolutional and Recurrent Neural Network for Sentiment Analysis of Short Texts | Xingyou Wang, Weijie Jiang, Zhiyong Luo
• Autoencoders / Encoder-Decoders
• LSTM
• GRU

Evaluation Measures - Classification Performance

Youtube search... ...Google search

• Evaluation Measures - Classification Performance

Confusion Matrix, Precision, Recall, F Score, ROC Curves, trade off between True Positive Rate and False Positive Rate.

Capabilities

• AI-Powered Search

Summarizer

Youtube search... ...Google search

Sentiment Analysis

Youtube search... ...Google search

Wikifier

Youtube search... ...Google search

• Illinois Wikifier
• Wikifier Demo - Semantic Annotation Service for 100 Languages | Wikifier.org
  • The JSI Wikifier is a web service that takes a text document as input and annotates it with links to relevant Wikipedia concepts.

Sapir Whorf Hypothesis - How language shapes the way we think

Youtube search... ...Google search