How do software programs that automatically extract information from web pages actually work? This video course, based on content from the book "Mining the Social Web" (O'Reilly Media) by Matthew Russell, teaches you how to create machines that can navigate the internet, cut through the noise, and extract the most important textual content from any web page or group of web pages. You'll learn how to use Python to write programs that can crawl, scrape, and parse the web; as well as discover how to extract key terms and sentences from web mined documents, explore document summarization techniques used in natural language processing and artificial intelligence, and gain experience using Python’s Natural Language Toolkit (NLTK) to auto-summarize web articles. Learners should have a basic understanding of Python.

Understand how helpful (and malicious) web bots crawl, parse, and index the web
Learn how to scrape content, extract links, and parse information from web pages and blog feeds
Discover how Python’s Natural Language Toolkit (NLTK) extracts and summarizes content
After completing his PhD in astrophysics, Mikhail Klassen transitioned to data science and refined his expertise in data mining, data analysis, and machine learning. He's now the Chief Data Scientist for Paladin: Paradigm Knowledge Solutions in Montreal, where he combines data mining and artificial intelligence to deliver personalized training for the aerospace industry.

Table of Contents
Mining Web Pages
Welcome To The Course 00:02:06
About The Author 00:00:52
Why Mine The Web? 00:02:03
Scraping, Crawling, And Parsing The Web 00:07:22
Applying NLTK To The Web 00:07:48
Document Summarization 00:07:27
Detecting Entities 00:03:21
Wrap Up 00:01:22