Evaluation

YouTube search... ...Google search

• Evaluation Measures - Classification Performance
• Benchmarks
• Train, Validate, and Test
• Model Monitoring
• Bias and Variances
• Explainable Artificial Intelligence (XAI)
• Imbalanced Data
• Five ways to evaluate AI systems | Felix Wetzel - Recruiting Daily
• Cyber Security Evaluation Tool (CSET®) ...provides a systematic, disciplined, and repeatable approach for evaluating an organization’s security posture.

Many products today leverage artificial intelligence for a wide range of industries, from healthcare to marketing. However, most business leaders who need to make strategic and procurement decisions about these technologies have no formal AI background or academic training in data science. The purpose of this article is to give business people with no AI expertise a general guideline on how to assess an AI-related product to help decide whether it is potentially relevant to their business. How to Assess an Artificial Intelligence Product or Solution (Even if You’re Not an AI Expert) | Daniel Faggella - Emerj

Evaluating AI- and ML-Based Security Products Anup Ghosh, Founder and CEO, Invincea Liam Randall, President, Critical Stack, A Division of Capital One Chad Skipper, VP Competitive Intelligence and Product Testing, Cylance Mike Spanbauer, Vice President of Research and Strategy, NSS Labs With endless AI or machine learning product claims, buyers are left bewildered with how to test these claims. It falls to independent third-party test organizations to develop and update traditional test protocols to test and validate AI and ML product capability claims. This panel will tackle the key issues that third-party testing must address to validate AI and ML security products.

How Should We Evaluate Machine Learning for AI?: Percy Liang Machine learning has undoubtedly been hugely successful in driving progress in AI, but it implicitly brings with it the train-test evaluation paradigm. This standard evaluation only encourages behavior that is good on average; it does not ensure robustness as demonstrated by adversarial examples, and it breaks down for tasks such as dialogue that are interactive or do not have a correct answer. In this talk, I will describe alternative evaluation paradigms with a focus on natural language understanding tasks, and discuss ramifications for guiding progress in AI in meaningful directions. Percy Liang is an Assistant Professor of Computer Science at Stanford University (B.S. from MIT, 2004; Ph.D. from UC Berkeley, 2011). His research spans machine learning and natural language processing, with the goal of developing trustworthy agents that can communicate effectively with people and improve over time through interaction. Specific topics include question answering, dialogue, program induction, interactive learning, and reliable machine learning. His awards include the IJCAI Computers and Thought Award (2016), an NSF CAREER Award (2016), a Sloan Research Fellowship (2015), and a Microsoft Research Faculty Fellowship (2014).

Buying

Build or buy AI? You're asking the wrong question Evan Kohn, chief business officer and head of marketing at Pypestream, talks with Tonya Hall about why companies need to turn to staffing for AI and building data sets.

Why you should Buy Open-Source AI Considering an AI assistant in your home? Before you auto-buy that pretty picture in front of you, be sure to check out the open-source offerings as well.