Human-in-the-Loop (HITL) Learning

YouTube search... ...Google search

• Learning Techniques
• Human-in-the-loop | Wikipedia
• Human In-The-Loop Vs. Out-of-The-Loop in AI Systems: The Case of AI Self-Driving Cars | Lance Eliot - AI Trends
• What Is Human in The Loop (HITL) Machine Learning? | Jonathan Johnson - bmc blogs

Human-in-the-loop (HITL), basically you can say, is the process of leveraging the power of the machine and human intelligence to create machine learning-based AI models. HITL describes the process when the machine or computer system is unable to solve a problem, needs human intervention like involving in both the training and testing stages of building an algorithm, for creating a continuous feedback loop allowing the algorithm to give every time better results. What is Human in the Loop Machine Learning: Why & How Used in AI? | Vikram Singh Bisen - Medium

Example use case:

• Limited data for use
• Uncomprehensive data
• Interpretation required
• High liability mistakes
• Rare objectives
• Uncommon objectives
• AI functional inexperienced

Human in the loop Workflow Automation A talk given by Tina Huang from Transposit at the 2019 Platform Summit in Stockholm. Manual workflows are productivity killers. Automation has become our North Star. But many workflows can only be partially automated; they may benefit from or require human intervention. APIs give us the levers we need to build great automation. Applications provide the interfaces that pull humans into the loop at critical junctures. Composition is how we turn those APIs into apps that integrate disparate applications, simplify workflows, notify people, and respond to interactions. In this talk, we’ll demonstrate how we’ve built apps that allow for human-in-the-loop automation and show you how using an API composition platform like Transposit can help you efficiently build apps and bots that automate the tedium and let humans add maximal value.

dabl: Automatic Machine Learning with a Human in the Loop |SciPy 2020| Andreas Mueller In many real-world applications, data quality and curation and domain knowledge play a much larger role in building successful models than coming up with complex processing techniques and tweaking hyper-parameters. Therefore, a machine learning toolbox should enable users to understand both data and model, and not burden the practitioner with picking preprocessing steps and hyperparameters. The dabl library is a first step in this direction. It provides automatic visualization routines and model inspection capabilities while automating away model selection. This talk will introduce the dabl library and show how to use it to quickly create supervised models and identify modeling and data quality issues.

Easily Implement Human in the Loop into Your Machine Learning Predictions with Amazon A2I Companies have millions of documents to process along with various types of documents. Often times, these documents are hard to read or have specific data points which are required to complete the business process. Using Amazon Augmented AI, you can now implement human reviews to review your machine learning predictions from Amazon Textract, Amazon Rekognition, Amazon SageMaker and many AWS AI/ ML services. In this tech talk, we walk through how to implement human reviews as well as showcase a use case by DealNet Capital on how they were able to reduce review time by 80% implementing Amazon A2I. Learning Objectives: Learn how to implement human reviews, Understand how Amazon A2I can work with other machine learning services, Learn how DealNet used Amazon Textract and Amazon A2I to process loan applications. To learn more about the services featured in this talk, please visit: http://aws.amazon.com/augmented-ai

MLOps #15 - Scaling Human in the Loop Machine Learning with Robert Munro Human In The Loop Machine Learning and how to scale it. This conversation talked about the components of Human-in-the-Loop Machine Learning systems and the challenges when scaling them. Most machine learning applications learn from human examples. For example, autonomous vehicles know what a pedestrian looks like because people have spent 1000s of hours labeling “pedestrians” in videos; your smart device understands you because people have spent 1000s of hours labeling the intent of speech recordings; and machine translation services work because they are trained on 1000s of sentences that have been manually translated between languages. If you have a machine learning system that is learning from human-feedback in real-time, then there are many components to support and scale, from the machine learning models to the human interfaces and the processes for quality control. Robert Munro is an expert in combining Human and Machine Intelligence, working with Machine Learning approaches to Text, Speech, Image and Video Processing. Robert has founded several AI companies, building some of the top teams in Artificial Intelligence. He has worked in many diverse environments, from Sierra Leone, Haiti and the Amazon, to London, Sydney and Silicon Valley, in organizations ranging from startups to the United Nations. He has shipped Machine Learning Products at startups and at/with Amazon, Google, IBM & Microsoft. Robert has published more than 50 papers on Artificial Intelligence and is a regular speaker about technology in an increasingly connected world. He has a PhD from Stanford University. Robert is the author of Human-in-the-Loop Machine Learning (Manning Publications, 2020)

Humans-in-the-loop: improving artificial intelligence through human intelligence "A colleague approaches you and requests help with an anomaly detection system on some unstructured data. They tell you the data quantity is sparse but they need to get some sort of solution in place as soon as possible. What now? Often times when solving problems for customers, the initial data is small, imbalanced in class, and there may be little room for error. This is especially true when looking to catch infrequent anomalies. There are many techniques that can aid with these issues such as transfer learning, data augmentation, and data synthesis. However, these techniques may only get you so far initially with a generalized model. In order to get the most out of existing data as well as leverage the expertise internal to the business, human-in-the-loop (HITL) machine learning systems can aid in the effort. In this topic, we'll discuss how HITL systems can be structured, how they help drive more customer engagement, and help deliver more robust solutions from your data team."

MobiSys 2020 - Human-In-The-Loop Reinforcement Learning (RL) with an EEG Wearable Headset Mohit Agarwal @ ACM SIGMOBILE ONLINE Presented at MobiSys 2020