Difference between revisions of "Generative Pre-trained Transformer (GPT)"

From
Jump to: navigation, search
m (→‎GPT Impact to Development)
m
 
(225 intermediate revisions by the same user not shown)
Line 2: Line 2:
 
|title=PRIMO.ai
 
|title=PRIMO.ai
 
|titlemode=append
 
|titlemode=append
−
|keywords=artificial, intelligence, machine, learning, models, algorithms, data, singularity, moonshot, Tensorflow, Google, Nvidia, Microsoft, Azure, Amazon, AWS  
+
|keywords=ChatGPT, artificial, intelligence, machine, learning, NLP, NLG, NLC, NLU, models, data, singularity, moonshot, Sentience, AGI, Emergence, Moonshot, Explainable, TensorFlow, Google, Nvidia, Microsoft, Azure, Amazon, AWS, Hugging Face, OpenAI, Tensorflow, OpenAI, Google, Nvidia, Microsoft, Azure, Amazon, AWS, Meta, LLM, metaverse, assistants, agents, digital twin, IoT, Transhumanism, Immersive Reality, Generative AI, Conversational AI, Perplexity, Bing, You, Bard, Ernie, prompt Engineering LangChain, Video/Image, Vision, End-to-End Speech, Synthesize Speech, Speech Recognition, Stanford, MIT |description=Helpful resources for your journey with artificial intelligence; videos, articles, techniques, courses, profiles, and tools
−
|description=Helpful resources for your journey with artificial intelligence; videos, articles, techniques, courses, profiles, and tools  
+
 
 +
<!-- Google tag (gtag.js) -->
 +
<script async src="https://www.googletagmanager.com/gtag/js?id=G-4GCWLBVJ7T"></script>
 +
<script>
 +
  window.dataLayer = window.dataLayer || [];
 +
  function gtag(){dataLayer.push(arguments);}
 +
  gtag('js', new Date());
 +
 
 +
  gtag('config', 'G-4GCWLBVJ7T');
 +
</script>
 
}}
 
}}
−
[http://www.youtube.com/results?search_query=Generative+Pre+trained+Transformer-2+GPT+generation+nlg+natural+language+semantics YouTube search...]
+
[https://www.youtube.com/results?search_query=Generative+Pre+trained+Transformer+GPT YouTube]
−
[http://www.google.com/search?q=Generative+Pre+trained+Transformer-2+GPT+generation+nlg+natural+language+semantics ...Google search]
+
[https://www.quora.com/search?q=Generative%20Pre%20trained%20Transformer%20%GPT ... Quora]
 +
[https://www.google.com/search?q=Generative+Pre+trained+Transformer+GPT ...Google search]
 +
[https://news.google.com/search?q=Generative+Pre+trained+Transformer+GPT ...Google News]
 +
[https://www.bing.com/news/search?q=Generative+Pre+trained+Transformer+GPT&qft=interval%3d%228%22 ...Bing News]
  
−
* [[Case Studies]]
+
* [[Large Language Model (LLM)]] ... [[Large Language Model (LLM)#Multimodal|Multimodal]] ... [[Foundation Models (FM)]] ... [[Generative Pre-trained Transformer (GPT)|Generative Pre-trained]] ... [[Transformer]] ... [[Attention]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]]
−
** [[Writing]]
+
* [[Conversational AI]] ... [[ChatGPT]] | [[OpenAI]] ... [[Bing/Copilot]] | [[Microsoft]] ... [[Gemini]] | [[Google]] ... [[Claude]] | [[Anthropic]] ... [[Perplexity]] ... [[You]] ... [[phind]] ... [[Ernie]] | [[Baidu]]
−
** [[Publishing]]
+
* [[Natural Language Processing (NLP)]] ... [[Natural Language Generation (NLG)|Generation (NLG)]] ... [[Natural Language Classification (NLC)|Classification (NLC)]] ... [[Natural Language Processing (NLP)#Natural Language Understanding (NLU)|Understanding (NLU)]] ... [[Language Translation|Translation]] ... [[Summarization]] ... [[Sentiment Analysis|Sentiment]] ... [[Natural Language Tools & Services|Tools]]
 +
* [[Agents]] ... [[Robotic Process Automation (RPA)|Robotic Process Automation]] ... [[Assistants]] ... [[Personal Companions]] ... [[Personal Productivity|Productivity]] ... [[Email]] ... [[Negotiation]] ... [[LangChain]]
 +
* [[What is Artificial Intelligence (AI)? | Artificial Intelligence (AI)]] ... [[Generative AI]] ... [[Machine Learning (ML)]] ... [[Deep Learning]] ... [[Neural Network]] ... [[Reinforcement Learning (RL)|Reinforcement]] ... [[Learning Techniques]]
 +
* [[Sequence to Sequence (Seq2Seq)]]
 +
* [[Recurrent Neural Network (RNN)]] 
 +
* [[Long Short-Term Memory (LSTM)]]
 +
* [[ELMo]]
 +
* [[Bidirectional Encoder Representations from Transformers (BERT)]]  ... a better model, but less investment than the larger [[OpenAI]] organization
 +
* [https://openai.com/blog/gpt-2-6-month-follow-up/ OpenAI Blog] | [[OpenAI]]
 
* [[Text Transfer Learning]]
 
* [[Text Transfer Learning]]
−
* [[Natural Language Generation (NLG)]]
+
* [[Video/Image]] ... [[Vision]] ... [[Enhancement]] ... [[Fake]] ... [[Reconstruction]] ... [[Colorize]] ... [[Occlusions]] ... [[Predict image]] ... [[Image/Video Transfer Learning]]
−
* [[Generated Image]]
+
[[Writing/Publishing#SynthPub|Writing/Publishing - SynthPub]]
−
* [http://openai.com/blog/gpt-2-6-month-follow-up/ OpenAI Blog] | [[OpenAI]]
+
* [https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf Language Models are Unsupervised Multitask Learners | Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever]
−
* [http://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf Language Models are Unsupervised Multitask Learners | Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever]
+
* [https://neural-monkey.readthedocs.io/en/latest/machine_translation.html Neural Monkey | Jindřich Libovický, Jindřich Helcl, Tomáš Musil] Byte Pair Encoding (BPE) enables NMT model translation on open-vocabulary by encoding rare and unknown words as sequences of subword units.
−
* [http://neural-monkey.readthedocs.io/en/latest/machine_translation.html Neural Monkey | Jindřich Libovický, Jindřich Helcl, Tomáš Musil] Byte Pair Encoding (BPE) enables NMT model translation on open-vocabulary by encoding rare and unknown words as sequences of subword units.
+
* [https://github.com/openai/gpt-2 Language Models are Unsupervised Multitask Learners - GitHub]
−
* [[Attention]] Mechanism/[[Transformer]] Model
+
* [https://www.infoq.com/news/2019/11/microsoft-ai-conversation/ Microsoft Releases DialogGPT AI Conversation Model | Anthony Alford - InfoQ] - trained on over 147M dialogs  
−
* [[Bidirectional Encoder Representations from Transformers (BERT)]]
+
* [https://github.com/karpathy/minGPT minGPT | Andrej Karpathy - GitHub]
−
* [[ELMo]]
+
* [https://sambanova.ai/solutions/gpt/ SambaNova Systems] ... Dataflow-as-a-Service GPT
−
* [http://github.com/openai/gpt-2 Language Models are Unsupervised Multitask Learners - GitHub]
+
* [https://www.reuters.com/technology/facebook-owner-meta-opens-access-ai-large-language-model-2022-05-03/  [[Meta|Facebook]]-owner Meta opens access to AI large language model | Elizabeth Culliford - Reuters] ... [[Meta|Facebook]] 175-billion-parameter language model - Open Pretrained Transformer (OPT-175B)
−
* [http://www.infoq.com/news/2019/11/microsoft-ai-conversation/ Microsoft Releases DialogGPT AI Conversation Model | Anthony Alford - InfoQ] - trained on over 147M dialogs  
+
* [https://lilianweng.github.io/posts/2018-06-24-attention/ Resource on Transformers | Lilian Weng - Lil'Log]
−
* [http://github.com/karpathy/minGPT minGPT | Andrej Karpathy - GitHub]
 
  
−
http://cdn-images-1.medium.com/max/800/1*jbcwhhB8PEpJRk781rML_g.png
+
<img src=https://production-media.paperswithcode.com/methods/Screen_Shot_2020-05-27_at_12.41.44_PM.png width="1000">
 +
* [https://paperswithcode.com/method/gpt GPT | Papers With Code]
  
−
= Generative Pre-trained Transformer (GPT-3) =
 
−
 
−
* [http://arxiv.org/abs/2005.14165 Language Models are Few-Shot Learners | T. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei - arXiv.org]
 
−
* [http://towardsdatascience.com/gpt-3-demos-use-cases-implications-77f86e540dc1 GPT-3: Demos, Use-cases, Implications | Simon O'Regan - Towards Data Science]
 
−
* [http://openai.com/blog/openai-api/ OpenAI API] ...today the API runs models with weights from the GPT-3 family with many speed and throughput improvements.
 
−
* [http://medium.com/@praveengovi.analytics/gpt-3-by-openai-outlook-and-examples-f234f9c62c41 GPT-3 by OpenAI – Outlook and Examples | Praveen Govindaraj | Medium]
 
−
* [http://www.gwern.net/GPT-3 GPT-3 Creative Fiction | R. Gwern]
 
  
  
Line 38: Line 51:
  
  
−
==== <span id="Try"></span>Try... ====
+
<b><span id="Try"></span>Try...</b>
  
 +
* [https://twitter.com/sushant_kumar Sushant Kumar]'s micro-site - Replace your 'word' in the following URL to see what GPT-3 generates: https://thoughts.sushant-kumar.com/word
 +
* [https://serendipityrecs.com/ Serendipity] ...an AI powered recommendation engine for anything you want.
 +
* [https://www.taglines.ai/ Taglines.ai] ... just about every business has a tagline — a short, catchy phrase designed to quickly communicate what it is that they do.
 +
* [https://www.simplify.so/ Simplify.so] ...simple, easy-to-understand explanations for everything
  
−
* [http://twitter.com/sushant_kumar Sushant Kumar]'s micro-site - Replace your 'word' in the following URL to see what GPT-3 generates: http://thoughts.sushant-kumar.com/word
 
−
 
−
* [http://serendipityrecs.com/ Serendipity] ...an AI powered recommendation engine for anything you want.
 
−
 
−
* [http://www.taglines.ai/ Taglines.ai] ... just about every business has a tagline — a short, catchy phrase designed to quickly communicate what it is that they do.
 
−
 
−
* [http://www.simplify.so/ Simplify.so] ...simple, easy-to-understand explanations for everything
 
  
 +
<hr>
  
−
<hr>
 
  
  
Line 59: Line 69:
 
<youtube>5fqxPOaaqi0</youtube>
 
<youtube>5fqxPOaaqi0</youtube>
 
<b>What is GPT-3? Showcase, possibilities, and implications
 
<b>What is GPT-3? Showcase, possibilities, and implications
−
</b><br>What is going on in AI research lately? GPT-3 crashed the party, let’s see what it is and what it can do. Hoping we do not forget how problematic it might also become. GPT-3 Paper : Brown, Tom B., Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan et al. "Language models are few-shot learners." arXiv preprint arXiv:2005.14165 (2020). http://arxiv.org/pdf/2005.14165.pdf  
+
</b><br>What is going on in AI research lately? GPT-3 crashed the party, let’s see what it is and what it can do. Hoping we do not forget how problematic it might also become. GPT-3 Paper : Brown, Tom B., Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan et al. "Language models are few-shot learners." arXiv preprint arXiv:2005.14165 (2020). https://arxiv.org/pdf/2005.14165.pdf  
 
|}
 
|}
 
|<!-- M -->
 
|<!-- M -->
Line 68: Line 78:
 
<b>14 Cool Apps Built on [[OpenAI]]'s GPT-3 API
 
<b>14 Cool Apps Built on [[OpenAI]]'s GPT-3 API
 
</b><br>14 Cool applications just built on top of [[OpenAI]]'s GPT-3 (generative predictive transformer) API (currently in private beta).  
 
</b><br>14 Cool applications just built on top of [[OpenAI]]'s GPT-3 (generative predictive transformer) API (currently in private beta).  
−
|}
 
−
|}<!-- B -->
 
−
{|<!-- T -->
 
−
| valign="top" |
 
−
{| class="wikitable" style="width: 550px;"
 
−
||
 
−
<youtube>lQnLwUfwgyA</youtube>
 
−
<b>This text generation AI is INSANE (GPT-3)
 
−
</b><br>An overview of the gpt-3 machine learning model, why everyone should understand it, and why some (including its creator, open AI) think it's dangerous.
 
−
|}
 
−
|<!-- M -->
 
−
| valign="top" |
 
−
{| class="wikitable" style="width: 550px;"
 
−
||
 
−
<youtube>qqbqW4aVvHo</youtube>
 
−
<b>GPT-3 Demo Installation -Generative pretrained Transformer model (Third generation of [[OpenAI]])
 
−
</b><br>[[Python]]code.
 
−
|}
 
−
|}<!-- B -->
 
−
{|<!-- T -->
 
−
| valign="top" |
 
−
{| class="wikitable" style="width: 550px;"
 
−
||
 
−
<youtube>SY5PvZrJhLE</youtube>
 
−
<b>How Artificial Intelligence Changed the Future of Publishing | [[OpenAI]] GPT-3 and the Future of Books
 
−
</b><br>Go from content chaos to clear, compelling writing that influences people to act without them realizing it: http://bit.ly/thebestwaytosayit  As Ed Leon Klinger shows in his GPT 3 demo and GPT 3 examples thread
 
−
|}
 
−
|<!-- M -->
 
−
| valign="top" |
 
−
{| class="wikitable" style="width: 550px;"
 
−
||
 
−
<youtube>pXOlc5CBKT8</youtube>
 
−
<b>GPT-3: Language Models are Few-Shot Learners (Paper Explained)
 
−
</b><br>How far can you go with ONLY language modeling? Can a large enough language model perform [[Natural Language Processing (NLP)]] task out of the box? [[OpenAI]] take on these and other questions by training a transformer that is an order of magnitude larger than anything that has ever been built before and the results are astounding.
 
−
|}
 
−
|}<!-- B -->
 
−
{|<!-- T -->
 
−
| valign="top" |
 
−
{| class="wikitable" style="width: 550px;"
 
−
||
 
−
<youtube>_8yVOC4ciXc</youtube>
 
−
<b>GPT3: An Even Bigger Language Model - Computerphile
 
−
</b><br>Basic mathematics from a language model? Rob Miles on GPT3, where it seems like size does matter!  More from Rob Miles: http://bit.ly/Rob_Miles_YouTube  This video was filmed and edited by Sean Riley. Computer Science at the University of Nottingham: https://bit.ly/nottscomputer
 
−
|}
 
−
|<!-- M -->
 
−
| valign="top" |
 
−
{| class="wikitable" style="width: 550px;"
 
−
||
 
−
<youtube>OznMk5Jexu8</youtube>
 
−
<b>GPT-3 from [[OpenAI]] is here and it's a MONSTER!
 
−
</b><br>GPT-3 is the largest language model to date with 175 billion parameters. It is able to do various [[Natural Language Processing (NLP)]] tasks (translation, question answering) without additional finetuning.
 
−
|}
 
−
|}<!-- B -->
 
−
{|<!-- T -->
 
−
| valign="top" |
 
−
{| class="wikitable" style="width: 550px;"
 
−
||
 
−
<youtube>kpiY_LemaTc</youtube>
 
−
<b>GPT-3 vs Human Brain
 
−
</b><br>GPT-3 has 175 billion parameters/synapses. Human brain has 100 trillion synapses. How much will it cost to train a language model the size of the human brain?  REFERENCES:
 
−
 
−
[1] [http://arxiv.org/abs/2005.14165 GPT-3 paper: Language Models are Few-Shot Learners]
 
−
 
−
[2] [http://lambdalabs.com/blog/demystifying-gpt-3/ OpenAI's GPT-3 Language Model: A Technical Overview]
 
−
 
−
[3] [http://arxiv.org/abs/2005.04305 Measuring the Algorithmic Efficiency of Neural Networks]
 
−
|}
 
−
|<!-- M -->
 
−
| valign="top" |
 
−
{| class="wikitable" style="width: 550px;"
 
−
||
 
−
<youtube>0ZVOmBp29E0</youtube>
 
−
<b>Steve Omohundro on GPT-3
 
−
</b><br>In this research meeting, guest Stephen Omohundro gave a fascinating talk on GPT-3, the new massive [[OpenAI]] Natural Language Processing model. He reviewed the network architecture, training process, and results in the context of past work. There was extensive discussion on the implications for [[Natural_Language_Processing_(NLP)]] and for Machine Intelligence / AGI.
 
−
|}
 
−
|}<!-- B -->
 
−
{|<!-- T -->
 
−
| valign="top" |
 
−
{| class="wikitable" style="width: 550px;"
 
−
||
 
−
<youtube>8psgEDhT1MM</youtube>
 
−
<b>GPT 3 Demo and Explanation - An AI revolution from [[OpenAI]]
 
−
</b><br>GPT 3 can write poetry, translate text, chat convincingly, and answer abstract questions. It's being used to code, design and much more.  I'll give you a demo of some of the latest in this technology and some of how it works. GPT3 comes from a company called [[OpenAI]]. [[OpenAI]] was founded by Elon Musk and Sam Altman (former president of Y-combinator the startup accelerator). [[OpenAI]] was founded with over a Billion invested to collaborate and create human-level AI for the benefit of society. GPT 3 has been developed for a number of years. One of the early papers published was on Generative Pre-Training.  The idea behind generative pre-training (GPT) is that while most AI's are trained on labeled data, there's a ton of data that isn't labeled.  If you can evaluate the words and use them to train and tune the AI it can start to create predictions of future text on the unlabeled data.  You repeat the process until predictions start to converge.  The newest GPT is able to do a ton. Some of the demos include:  - GPT 3 demo of how to design a user interface using AI  - GPT 3 demo of how to code a react application using AI  - GPT 3 demo of an excel plug-in to fill data using AI    - GPT 3 demo of a search engine/answer engine using AI  - GPT3 demo of command line auto-complete from English to shell commands 
 
−
|}
 
−
|<!-- M -->
 
−
| valign="top" |
 
−
{| class="wikitable" style="width: 550px;"
 
−
||
 
−
<youtube>aDFLp4A1EmY</youtube>
 
−
<b>Panel discussion - GPT-3 and Artificial General Intelligence 27 Aug 2020
 
−
</b><br>Is GPT-3 a step towards creating artificial general intelligence?  Chair: Associate Professor Kate Devitt - Chief Scientist, Trusted Autonomous Systems
 
−
 
−
Panel:
 
−
• Professor David Chalmers (NYU)
 
−
• Professor Susan Schneider (NASA and Florida Atlantic University)
 
−
• Professor Marcus Hutter (ANU)
 
−
 
−
A philosophical discussion on the development of artificial intelligence and specifically advances in Generative Pre-trained Transformer-3 (GPT-3). GPT-3 is an auto-complete algorithm created by OpenAI as part of their endeavour to develop artificial general intelligence. GPT-3 is the third in a series of autocomplete tools designed by OpenAI. (GPT stands for “generative pre-trained transformer.”). GPT-3 is fed on an unimaginatively large corpus of human knowledge including all of Wikipedia, millions of books, websites and other materials including philosophy texts. In fact, any type of information uploaded to the internet is possible food for GPT-3's artificial mind to dwell on. The result? Eerily coherent, complex and interesting thoughts about almost any topic. The sophisticated, nuanced text produced by GPT-3 seems to pass the Turing Test for many--including philosophers. Some of GPT-3's answers are shedding new light on enduring philosophical questions. Is GPT-3 the beginnings of an artificial general intelligence. Does it create ideas like a human mind, or even better than a human mind? Is human cognition similarly some sort of autocomplete program in our brains? Is it possible that GPT-3 one day becomes consciousness or is it already conscious?--How could we tell. If an AI passes our tests for consciousness, do we then have an obligation to accord it rights? If so, what sorts of rights might it deserve. Independently of rights, how should humans manage an AI that has access to everything that is posited and known and can trick humans into believing that another rational agent is communicating with them? The panel considers what GPT-3 tell us about the ambition to build an artificial general intelligence, consciousness, human thought and how we should treat AI in an increasingly digital and disembodied world rife with mis- and disinformation.
 
 
|}
 
|}
 
|}<!-- B -->
 
|}<!-- B -->
  
−
= <span id="GPT Impact to Development"></span>GPT Impact to Development =
+
== <span id="GPT Impact to Development"></span>GPT Impact to Development ==
−
* [[Development]]
+
* * [[Development]]  ...[[Development#AI Pair Programming Tools|AI Pair Programming Tools]] ... [[Analytics]]  ... [[Visualization]]  ... [[Diagrams for Business Analysis]]
−
* http://analyticsindiamag.com/will-the-much-hyped-gpt-3-impact-the-coders/ Will The Much-Hyped GPT-3 Impact The Coders? | Analytics India Magazine]
+
* https://analyticsindiamag.com/will-the-much-hyped-gpt-3-impact-the-coders/ Will The Much-Hyped GPT-3 Impact The Coders? | Analytics India Magazine]
−
* [http://twitter.com/sharifshameem/status/1283322990625607681 With GPT-3, I built a layout generator where you just describe any layout you want, and it generates the JSX code for you. | Sharif Shameem] - [http://debuild.co/ debuild]
+
* [https://twitter.com/sharifshameem/status/1283322990625607681 With GPT-3, I built a layout generator where you just describe any layout you want, and it generates the JSX code for you. | Sharif Shameem] - [https://debuild.co/ debuild]
  
 
{|<!-- T -->
 
{|<!-- T -->
Line 188: Line 100:
 
<youtube>utuz7wBGjKM</youtube>
 
<youtube>utuz7wBGjKM</youtube>
 
<b>[[OpenAI]] Model Generates Python Code
 
<b>[[OpenAI]] Model Generates Python Code
−
</b><br>This code completion engine can write an entire function from just the name! OpenAI demonstrates what happens when you learn a language model on thousands of GitHub Python repositories. Source Clip: http://youtu.be/fZSFNUT6iY8
+
</b><br>This code completion engine can write an entire function from just the name! [[OpenAI]] demonstrates what happens when you learn a language model on thousands of GitHub [[Python]] repositories. Source Clip: https://youtu.be/fZSFNUT6iY8
 
|}
 
|}
 
|}<!-- B -->
 
|}<!-- B -->
Line 195: Line 107:
 
{| class="wikitable" style="width: 550px;"
 
{| class="wikitable" style="width: 550px;"
 
||
 
||
−
<youtube>y5-wzgIySb4</youtube>
+
<youtube>3P3TcKaegbA</youtube>
−
<b>[[OpenAI]] and [[Microsoft]] Can Generate [[Python]] Code
+
<b>Generative Python Code with GPT
−
</b><br>Open AI language model was trained on thousands of GitHub repositories using the same unsupervised learning as the GPT models. Build 2020
+
</b><br>In a quest to teach neural networks via transformers to write Python code. Project name: Generative Python Transformers!
 
|}
 
|}
 
|<!-- M -->
 
|<!-- M -->
Line 204: Line 116:
 
||
 
||
 
<youtube>6SGj2OrTpbI</youtube>
 
<youtube>6SGj2OrTpbI</youtube>
−
<b>HH6
+
<b>GPT 3 Explanation And Demo Reaction | Should You Be Scared ? Techies Reaction To GPT3 AI [[OpenAI]]
−
</b><br>BB6
+
</b><br>In this video we will look at some of the demos and reactions across social media on GPT-3. The links to the tweets and demo you see in this video have been linked below so please do react out to them if you have any questions.
 
|}
 
|}
 
|}<!-- B -->
 
|}<!-- B -->
Line 226: Line 138:
 
|}<!-- B -->
 
|}<!-- B -->
  
−
= Generative Pre-trained Transformer (GPT-2) =
+
= <span id="Custom GPTs"></span>Custom GPTs =
−
* GitHub
+
* [[Agents]] ... [[Robotic Process Automation (RPA)|Robotic Process Automation]] ... [[Assistants]] ... [[Personal Companions]] ... [[Personal Productivity|Productivity]] ... [[Email]] ... [[Negotiation]] ... [[LangChain]]
−
** [http://github.com/openai/gpt-2/blob/master/README.md  (117M parameter) version of GPT-2]
 
−
** [http://github.com/openai/gpt-2 openai/gpt-2 GPT-2]
 
−
* [http://analyticsindiamag.com/how-to-get-started-with-openais-gpt-2-for-text-generation/ How to Get Started with OpenAIs GPT-2 for Text Generation | Amal Nair - Analytics India Magazine]
 
−
* [http://aiweirdness.com/post/182824715257/gpt-2-it-learned-on-the-internet GPT-2: It learned on the Internet | Janelle Shane]
 
−
* [http://towardsdatascience.com/too-powerful-nlp-model-generative-pre-training-2-4cc6afb6655 Too powerful NLP model (GPT-2): What is Generative Pre-Training | Edward Ma]
 
−
* [http://medium.com/@ajitrajasekharan/gpt-2-a-promising-but-nascent-transfer-learning-method-that-could-reduce-or-even-eliminate-in-some-48ea3370cc21 GPT-2 A nascent transfer learning method that could eliminate supervised learning some NLP tasks | Ajit Rajasekharan - Medium]
 
−
* [http://insights.dice.com/2019/02/19/openai-platform-generating-fake-news-wonderful OpenAI Creates Platform for Generating Fake News. Wonderful | Nick Kolakowski - Dice]
 
−
* [http://inferkit.com/ InferKit | Adam D King]- completes your text.
 
  
 +
Custom GPTs are personalized versions of AI models like [[ChatGPT]] that can be tailored for specific tasks or projects. They represent a significant advancement in AI implementation, allowing businesses and individuals to customize AI tools to meet unique challenges and operational needs.
  
−
== Coding Train Late Night 2 ==
+
== <span id="OpenAI Platform"></span>OpenAI Platform ==
 +
* [https://chat.openai.com/create OpenAI Platform]
 +
 
 +
[[OpenAI]] allows Plus and Enterprise users to create custom GPTs that can browse the web, create images, and run code. Users can upload knowledge files, modify the GPT's appearance, and define its actions
 +
 
 +
=== <span id="OpenAI GPT Store"></span>OpenAI GPT Store ===
 +
* [https://chatgpt.com/gpts GPT Store]
 +
 
 +
The [[OpenAI]] GPT Store provides a platform for users to create, share, and monetize their custom GPTs, expanding the capabilities and possibilities of AI assistants like [[ChatGPT]]. It allows users of [[ChatGPT]] Plus to create and share their own custom chatbots, known as GPTs (Generative Pre-trained Transformers). The GPT Store offers a platform for developers to monetize their custom GPTs and provides a wide range of AI tools and capabilities for users to explore and enhance their AI assistant capabilities
 +
 
 +
<youtube>2wYcJEcKVPk</youtube>
 +
<youtube>amjnJrfByS0</youtube>
 +
<youtube>VudB3E9tSbc</youtube>
 +
<youtube>SVA-OBl44m4</youtube>
 +
 
 +
=== <span id="OpenAI GPT Builder"></span>OpenAI GPT Builder ===
 +
 
 +
With the GPT Builder, users can tailor GPTs for specific tasks or topics by combining instructions, knowledge, and capabilities. It enables users to build AI agents without the need for coding skills, making it accessible to a wide range of individuals, including educators, coaches, and anyone interested in building helpful tools.
 +
 
 +
To create a GPT using the GPT Builder, users can access the builder interface through the [[OpenAI]] platform at chat.openai.com/gpts/editor or by selecting "My GPTs" after logging in. The builder interface provides a split screen with a Create panel where users can enter prompts and instructions to build their chatbot, and a Preview panel that allows users to interact with the chatbot as they build it, making it easier to refine and customize the GPT.
 +
 
 +
The GPT Builder also offers features such as the ability to add images to the GPT, either by asking the builder to create an image or by uploading custom images. Additionally, GPTs can be granted access to web browsing, [[Video/Image#DALL-E | DALL-E]]  (an image generation model), and [[OpenAI]]'s Code Interpreter tool for writing and executing software. The builder interface also includes a Knowledge section where users can upload custom data to enhance the capabilities of their GPTs .
 +
 
 +
<youtube>f2uPl2MlV24</youtube>
 +
<youtube>SjJsXyBTPUc</youtube>
 +
 
 +
= <span id="Let's build GPT: from scratch, in code, spelled out - Andrej Karpathy"></span>Let's build GPT: from scratch, in code, spelled out - Andrej Karpathy =
 +
* [[What is Artificial Intelligence (AI)? | Artificial Intelligence (AI)]] ... [[Generative AI]] ... [[Machine Learning (ML)]] ... [[Deep Learning]] ... [[Neural Network]] ... [[Reinforcement Learning (RL)|Reinforcement]] ... [[Learning Techniques]]
 +
* [[Development]] ... [[Notebooks]] ... [[Development#AI Pair Programming Tools|AI Pair Programming]] ... [[Codeless Options, Code Generators, Drag n' Drop|Codeless, Generators, Drag n' Drop]] ... [[Algorithm Administration#AIOps/MLOps|AIOps/MLOps]] ... [[Platforms: AI/Machine Learning as a Service (AIaaS/MLaaS)|AIaaS/MLaaS]]
  
 
{|<!-- T -->
 
{|<!-- T -->
Line 244: Line 177:
 
{| class="wikitable" style="width: 550px;"
 
{| class="wikitable" style="width: 550px;"
 
||
 
||
−
<youtube>0LZUSkwCYfU</youtube>
+
<youtube>kCc8FmEb1nY</youtube>
−
<b>Coding Train Late Night 2: Fetch, GPT-2 and RunwayML
+
<b>Let's build GPT: from scratch, in code, spelled out.
−
</b><br>The Coding Train
+
</b><br>[[Creatives#Andrej Karpathy |Andrej Karpathy]]  ...We build a Generatively Pretrained Transformer (GPT), following the paper "Attention is All You Need" and OpenAI's GPT-2 / GPT-3. We talk about connections to ChatGPT, which has taken the world by storm. We watch GitHub Copilot, itself a GPT, help us write a GPT (meta :D!) . I recommend people watch the earlier makemore videos to get comfortable with the autoregressive language modeling framework and basics of tensors and PyTorch nn, which we take for granted in this video.
−
0:00 Live Stream Starts
+
 
−
3:51 Introduction With Dad Jokes
+
Links:
−
11:29 Coding Late At Night Projects and Notes
+
* [https://colab.research.google.com/drive/1JMLa53HDuA-i7ZBmqV7ZnA3c_fvtXnx-?usp=sharing  Google colab for the video]
−
16:48 Scraping Dad Jokes With Fetch
+
* [https://github.com/karpathy/ng-video-lecture GitHub repo for the video]
−
50:10 Training a Model With Runway
+
* [https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ&index=2 Playlist of the whole Zero to Hero series so far]
−
57:52 Small Break
+
* [https://github.com/karpathy/nanoGPT nanoGPT repo]
−
1:00:15 Controlling Hue Lights
+
* [https://karpathy.ai my website]
−
1:20:00 Dad Joke Model
+
* [https://twitter.com/karpathy my twitter]
−
1:32:27 Skip: Audio Glitch (LOUD)
+
* [https://discord.gg/3zy8kqD9Cp our Discord channel]
−
1:35:00 Dad Joke Model
 
−
1:49:25 Dad Joke Generator
 
−
1:54:25 Goodbyes and End of Stream
 
  
−
Website: http://thecodingtrain.com/  
+
Supplementary links:
 +
* [https://arxiv.org/abs/1706.03762 Attention is All You Need paper]
 +
* [https://arxiv.org/abs/2005.14165 OpenAI GPT-3 paper] 
 +
* [https://openai.com/blog/chatgpt/ OpenAI ChatGPT blog post]
 +
* The GPU I'm training the model on is from Lambda GPU Cloud, I think the best and easiest way to spin up an on-demand GPU instance in the cloud that you can ssh to: https://lambdalabs.com . If you prefer to work in notebooks, I think the easiest path today is Google Colab.
 +
 
 +
Suggested exercises:
 +
* EX1: The n-dimensional tensor mastery challenge: Combine the `Head` and `MultiHeadAttention` into one class that processes all the heads in parallel, treating the heads as another batch dimension (answer is in nanoGPT).
 +
* EX2: Train the GPT on your own dataset of choice! What other data could be fun to blabber on about? (A fun advanced suggestion if you like: train a GPT to do addition of two numbers, i.e. a+b=c. You may find it helpful to predict the digits of c in reverse order, as the typical addition algorithm (that you're hoping it learns) would proceed right to left too. You may want to modify the data loader to simply serve random problems and skip the generation of train.bin, val.bin. You may want to mask out the loss at the input positions of a+b that just specify the problem using y=-1 in the targets (see CrossEntropyLoss ignore_index). Does your Transformer learn to add? Once you have this, swole doge project: build a calculator clone in GPT, for all of +-*/. Not an easy problem. You may need Chain of Thought traces.)
 +
* EX3: Find a dataset that is very large, so large that you can't see a gap between train and val loss. Pretrain the transformer on this data, then initialize with that model and finetune it on tiny shakespeare with a smaller number of steps and lower learning rate. Can you obtain a lower validation loss by the use of pretraining?
 +
* EX4: Read some transformer papers and implement one additional feature or change that people seem to use. Does it improve the performance of your GPT?
 +
 
 +
Chapters:
 +
* 00:00:00 intro: ChatGPT, Transformers, nanoGPT, Shakespeare baseline language modeling, code setup
 +
* 00:07:52 reading and exploring the data
 +
* 00:09:28 tokenization, train/val split
 +
* 00:14:27 data loader: batches of chunks of data
 +
* 00:22:11 simplest baseline: bigram language model, loss, generation
 +
* 00:34:53 training the bigram model
 +
* 00:38:00 port our code to a script Building the "self-attention"
 +
* 00:42:13 version 1: averaging past [[context]] with for loops, the weakest form of aggregation
 +
* 00:47:11 the trick in self-attention: matrix multiply as weighted aggregation
 +
* 00:51:54 version 2: using matrix multiply
 +
* 00:54:42 version 3: adding softmax
 +
* 00:58:26 minor code cleanup
 +
* 01:00:18 positional encoding
 +
* 01:02:00 THE CRUX OF THE VIDEO: version 4: self-attention
 +
* 01:11:38 note 1: attention as [[Agents#Communication | communication]]
 +
* 01:12:46 note 2: attention has no notion of space, operates over sets
 +
* 01:13:40 note 3: there is no [[Agents#Communication | communication]] across batch dimension
 +
* 01:14:14 note 4: encoder blocks vs. decoder blocks
 +
* 01:15:39 note 5: attention vs. self-attention vs. cross-attention
 +
* 01:16:56 note 6: "scaled" self-attention. why divide by sqrt(head_size) Building the Transformer
 +
* 01:19:11 inserting a single self-attention block to our network
 +
* 01:21:59 multi-headed self-attention
 +
* 01:24:25 feedforward layers of transformer block
 +
* 01:26:48 residual connections
 +
* 01:32:51 layernorm (and its relationship to our previous batchnorm)
 +
* 01:37:49 scaling up the model! creating a few variables. adding dropout Notes on Transformer
 +
* 01:42:39 encoder vs. decoder vs. both (?) Transformers
 +
* 01:46:22 super quick walkthrough of nanoGPT, batched multi-headed self-attention
 +
* 01:48:53 back to ChatGPT, GPT-3, pretraining vs. finetuning, RLHF
 +
* 01:54:32 conclusions
 +
 
 +
Corrections:
 +
* 00:57:00 Oops "tokens from the future cannot communicate", not "past". Sorry! :)
 +
* 01:20:05 Oops I should be using the head_size for the normalization, not C
 
|}
 
|}
 
|<!-- M -->
 
|<!-- M -->
Line 266: Line 242:
 
{| class="wikitable" style="width: 550px;"
 
{| class="wikitable" style="width: 550px;"
 
||
 
||
−
<youtube>kWsDL-6D-nk</youtube>
+
<youtube>9uw3F6rndnA</youtube>
−
<b>Coding Train Late Night 3: GPT-2, Hue Lights, Discord Bot
+
<b>Transformers: The best idea in AI | Andrej Karpathy and Lex Fridman
−
</b><br>The Coding Train
+
</b><br>[https://www.youtube.com/watch?v=cdiD-9MMpb0 Lex Fridman Podcast full episode]
−
0:00 Live Stream Starts
+
Please support this podcast by checking out our sponsors:
−
3:50 Introduction
+
* [https://www.eightsleep.com/lex Eight Sleep: to get special savings]
−
9:50 AI Joke Generator
+
* [https://betterhelp.com/lex BetterHelp: to get 10% off]
−
13:30 Live Stream Notes
+
* [https://fundrise.com/lex Fundrise]
−
19:50 Generative Text Training with GPT-2
+
* [https://athleticgreens.com/lex Athletic Greens: to get 1 month of fish oil]
−
29:40 Dad Joke Model Training
+
 
−
1:11:27 Using Hue Lights API
+
GUEST BIO:
−
1:31:50 More Dad Joke Generator
+
[[Creatives#Andrej Karpathy |Andrej Karpathy]] is a legendary AI researcher, engineer, and educator. He's the former director of AI at Tesla, a founding member of [[OpenAI]], and an educator at Stanford.
−
1:37:33 Discord Bot
+
 
−
2:15:04 Goodbyes and End of Stream
+
PODCAST INFO:
 +
* [https://lexfridman.com/podcast Podcast website]
 +
* [https://apple.co/2lwqZIr Apple Podcasts]
 +
* [https://spoti.fi/2nEwCF8 Spotify]
 +
* [https://lexfridman.com/feed/podcast/ RSS]
 +
* [https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4 Full episodes playlist]
 +
Clips playlist: https://www.youtube.com/playlist?list...
  
−
Website: http://thecodingtrain.com/  
+
SOCIAL:  
 +
* [https://twitter.com/lexfridman Twitter] 
 +
* [https://www.linkedin.com/in/lexfridman LinkedIn]
 +
* [https://www.facebook.com/lexfridman [[Meta|Facebook]]]
 +
* [https://www.instagram.com/lexfridman Instagram]
 +
* [https://medium.com/@lexfridman Medium]
 +
* [https://reddit.com/r/lexfridman Reddit]
 +
* [https://www.patreon.com/lexfridman Support on Patreon]
 +
 
|}
 
|}
 
|}<!-- B -->
 
|}<!-- B -->
−
 
−
== r/SubSimulator ==
 
−
 
−
Subreddit populated entirely by AI personifications of other subreddits -- all posts  and comments are generated automatically using:
 
−
 
−
* [http://www.reddit.com/r/SubredditSimulator/ Markov Chain Model]
 
−
* [http://www.reddit.com/r/SubSimulatorGPT2/ GPT-2 Language]
 
−
 
−
results in coherent and realistic simulated content.
 
−
 
−
 
−
== GetBadNews ==
 
−
 
−
* [http://getbadnews.com Get Bad News] game - Can you beat my score? Play the fake news game! Drop all pretense of ethics and choose the path that builds your persona as an unscrupulous media magnate. Your task is to get as many followers as you can while
 
−
 
−
<img src="http://www.getbadnews.com/wp-content/uploads/2018/02/share-score.png" width="500" height="250">
 

Latest revision as of 09:06, 28 May 2025

YouTube ... Quora ...Google search ...Google News ...Bing News

Writing/Publishing - SynthPub



Try...



What is GPT-3? Showcase, possibilities, and implications
What is going on in AI research lately? GPT-3 crashed the party, let’s see what it is and what it can do. Hoping we do not forget how problematic it might also become. GPT-3 Paper : Brown, Tom B., Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan et al. "Language models are few-shot learners." arXiv preprint arXiv:2005.14165 (2020). https://arxiv.org/pdf/2005.14165.pdf

14 Cool Apps Built on OpenAI's GPT-3 API
14 Cool applications just built on top of OpenAI's GPT-3 (generative predictive transformer) API (currently in private beta).

GPT Impact to Development

OpenAI Model Generates Python Code
Credits: Microsoft, OpenAI

OpenAI Model Generates Python Code
This code completion engine can write an entire function from just the name! OpenAI demonstrates what happens when you learn a language model on thousands of GitHub Python repositories. Source Clip: https://youtu.be/fZSFNUT6iY8

Generative Python Code with GPT
In a quest to teach neural networks via transformers to write Python code. Project name: Generative Python Transformers!

GPT 3 Explanation And Demo Reaction | Should You Be Scared ? Techies Reaction To GPT3 AI OpenAI
In this video we will look at some of the demos and reactions across social media on GPT-3. The links to the tweets and demo you see in this video have been linked below so please do react out to them if you have any questions.

Code 10x Faster With This CRAZY New AI Tool (GPT-3)
In this FREE LIVE training, Aaron and Naz will show you the new cutting edge machine learning AI, OpenAI's GPT-3.

Build 2 projects using GPT-3 in just a couple of minutes. Bare bones: branding generator, chat bot
Co-founded by Elon Musk OpenAI wants to make AI safe and accessible. A year ago the startup released GPT-2. That language model was at that time deemed too powerful to release. Eventually OpenAI made the model available. This year they've trained a much more powerful model at least 1 magnitude larger than GPT-2. I was one of the lucky 750+ people granted access as Beta testers by OpenAI to see what can be built using the GPT-3 API. The model costs millions of dollars to train which makes it out of reach for most organizations. This video skims the surface of what you can get done with this amazing new model.

Custom GPTs

Custom GPTs are personalized versions of AI models like ChatGPT that can be tailored for specific tasks or projects. They represent a significant advancement in AI implementation, allowing businesses and individuals to customize AI tools to meet unique challenges and operational needs.

OpenAI Platform

OpenAI allows Plus and Enterprise users to create custom GPTs that can browse the web, create images, and run code. Users can upload knowledge files, modify the GPT's appearance, and define its actions

OpenAI GPT Store

The OpenAI GPT Store provides a platform for users to create, share, and monetize their custom GPTs, expanding the capabilities and possibilities of AI assistants like ChatGPT. It allows users of ChatGPT Plus to create and share their own custom chatbots, known as GPTs (Generative Pre-trained Transformers). The GPT Store offers a platform for developers to monetize their custom GPTs and provides a wide range of AI tools and capabilities for users to explore and enhance their AI assistant capabilities

OpenAI GPT Builder

With the GPT Builder, users can tailor GPTs for specific tasks or topics by combining instructions, knowledge, and capabilities. It enables users to build AI agents without the need for coding skills, making it accessible to a wide range of individuals, including educators, coaches, and anyone interested in building helpful tools.

To create a GPT using the GPT Builder, users can access the builder interface through the OpenAI platform at chat.openai.com/gpts/editor or by selecting "My GPTs" after logging in. The builder interface provides a split screen with a Create panel where users can enter prompts and instructions to build their chatbot, and a Preview panel that allows users to interact with the chatbot as they build it, making it easier to refine and customize the GPT.

The GPT Builder also offers features such as the ability to add images to the GPT, either by asking the builder to create an image or by uploading custom images. Additionally, GPTs can be granted access to web browsing, DALL-E (an image generation model), and OpenAI's Code Interpreter tool for writing and executing software. The builder interface also includes a Knowledge section where users can upload custom data to enhance the capabilities of their GPTs .

Let's build GPT: from scratch, in code, spelled out - Andrej Karpathy

Let's build GPT: from scratch, in code, spelled out.
Andrej Karpathy ...We build a Generatively Pretrained Transformer (GPT), following the paper "Attention is All You Need" and OpenAI's GPT-2 / GPT-3. We talk about connections to ChatGPT, which has taken the world by storm. We watch GitHub Copilot, itself a GPT, help us write a GPT (meta :D!) . I recommend people watch the earlier makemore videos to get comfortable with the autoregressive language modeling framework and basics of tensors and PyTorch nn, which we take for granted in this video.

Links:

Supplementary links:

Suggested exercises:

  • EX1: The n-dimensional tensor mastery challenge: Combine the `Head` and `MultiHeadAttention` into one class that processes all the heads in parallel, treating the heads as another batch dimension (answer is in nanoGPT).
  • EX2: Train the GPT on your own dataset of choice! What other data could be fun to blabber on about? (A fun advanced suggestion if you like: train a GPT to do addition of two numbers, i.e. a+b=c. You may find it helpful to predict the digits of c in reverse order, as the typical addition algorithm (that you're hoping it learns) would proceed right to left too. You may want to modify the data loader to simply serve random problems and skip the generation of train.bin, val.bin. You may want to mask out the loss at the input positions of a+b that just specify the problem using y=-1 in the targets (see CrossEntropyLoss ignore_index). Does your Transformer learn to add? Once you have this, swole doge project: build a calculator clone in GPT, for all of +-*/. Not an easy problem. You may need Chain of Thought traces.)
  • EX3: Find a dataset that is very large, so large that you can't see a gap between train and val loss. Pretrain the transformer on this data, then initialize with that model and finetune it on tiny shakespeare with a smaller number of steps and lower learning rate. Can you obtain a lower validation loss by the use of pretraining?
  • EX4: Read some transformer papers and implement one additional feature or change that people seem to use. Does it improve the performance of your GPT?

Chapters:

  • 00:00:00 intro: ChatGPT, Transformers, nanoGPT, Shakespeare baseline language modeling, code setup
  • 00:07:52 reading and exploring the data
  • 00:09:28 tokenization, train/val split
  • 00:14:27 data loader: batches of chunks of data
  • 00:22:11 simplest baseline: bigram language model, loss, generation
  • 00:34:53 training the bigram model
  • 00:38:00 port our code to a script Building the "self-attention"
  • 00:42:13 version 1: averaging past context with for loops, the weakest form of aggregation
  • 00:47:11 the trick in self-attention: matrix multiply as weighted aggregation
  • 00:51:54 version 2: using matrix multiply
  • 00:54:42 version 3: adding softmax
  • 00:58:26 minor code cleanup
  • 01:00:18 positional encoding
  • 01:02:00 THE CRUX OF THE VIDEO: version 4: self-attention
  • 01:11:38 note 1: attention as communication
  • 01:12:46 note 2: attention has no notion of space, operates over sets
  • 01:13:40 note 3: there is no communication across batch dimension
  • 01:14:14 note 4: encoder blocks vs. decoder blocks
  • 01:15:39 note 5: attention vs. self-attention vs. cross-attention
  • 01:16:56 note 6: "scaled" self-attention. why divide by sqrt(head_size) Building the Transformer
  • 01:19:11 inserting a single self-attention block to our network
  • 01:21:59 multi-headed self-attention
  • 01:24:25 feedforward layers of transformer block
  • 01:26:48 residual connections
  • 01:32:51 layernorm (and its relationship to our previous batchnorm)
  • 01:37:49 scaling up the model! creating a few variables. adding dropout Notes on Transformer
  • 01:42:39 encoder vs. decoder vs. both (?) Transformers
  • 01:46:22 super quick walkthrough of nanoGPT, batched multi-headed self-attention
  • 01:48:53 back to ChatGPT, GPT-3, pretraining vs. finetuning, RLHF
  • 01:54:32 conclusions

Corrections:

  • 00:57:00 Oops "tokens from the future cannot communicate", not "past". Sorry! :)
  • 01:20:05 Oops I should be using the head_size for the normalization, not C

Transformers: The best idea in AI | Andrej Karpathy and Lex Fridman
Lex Fridman Podcast full episode Please support this podcast by checking out our sponsors:

GUEST BIO: Andrej Karpathy is a legendary AI researcher, engineer, and educator. He's the former director of AI at Tesla, a founding member of OpenAI, and an educator at Stanford.

PODCAST INFO:

Clips playlist: https://www.youtube.com/playlist?list...

SOCIAL:

Retrieved from "https://primo.ai/index.php?title=Generative_Pre-trained_Transformer_(GPT)&oldid=39173"