LLMs

This module explores Large Language Models (LLMs), focusing on their development and applications. It covers foundational concepts in natural language processing, the architecture of models like GPT (Generative Pre-trained Transformer), and their applications in tasks like text generation, translation, and content creation, along with ethical considerations and potential biases.

Portal > Artificial Intelligence > LLMs

Curriculum Builder

View Curriculum

Manning, Christopher D. “Human Language Understanding & Reasoning.” Daedalus (Cambridge, Mass.) 151, no. 2 (2022): 127–38. doi:10.1162/daed_a_01905.

Peters et al., “Deep Contextualised Word Representations,” 2018. doi:10.48550/arxiv.1802.05365.

Radford et al,. “Improving Language Understanding by Generative Pre-Training.” 2018.

Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” 2019.

Clark, Kevin, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. “ELECTRA: Pre-Training Text Encoders as Discriminators Rather Than Generators,” 2020. doi:10.48550/arxiv.2003.10555.

Zhang, Susan, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, et al. “OPT: Open Pre-Trained Transformer Language Models,” 2022. doi:10.48550/arxiv.2205.01068.

He, Junxian, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. “Towards a Unified View of Parameter-Efficient Transfer Learning,” 2022. doi:10.48550/arxiv.2110.04366.

Garg, Shivam, Dimitris Tsipras, Percy Liang, and Gregory Valiant. “What Can Transformers Learn In-Context? A Case Study of Simple Function Classes,” 2023. doi:10.48550/arxiv.2208.01066.

Stiennon, Nisan, Long Ouyang, Jeff Wu, Daniel M Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. “Learning to Summarize from Human Feedback,” 2022. doi:10.48550/arxiv.2009.01325.

Li, Yujia et al. “Competition-Level Code Generation with AlphaCode.” Science (American Association for the Advancement of Science) 378, no. 6624 (2022): 1092–97. doi:10.1126/science.abq1158.