Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. Web scraping software may access the World Wide Web directly using the Hypertext Transfer Protocol, or through a web browser. While web scraping can be done manually by a software user, the term typically refers to automated processes implemented using a bot or web crawler. It is a form of copying, in which specific data is gathered and copied from the web, typically into a central local database or spreadsheet, for later retrieval or analysis.
In this project we craped several websites and collected useful data . There are seveal libraries in python which can be used for web scraping such as beautiful soup, pandas and selenium. We've used beautiful soups and selenium in this project to scrap different websites.