NLP-spyder

This is a repository for spyder projects maintained by members of WIE.

project introductions:

• This is an index of all projects.
  
• For more complex projects, there are more detailed descriptions and operating instructions in the folder.

Guba（股吧）：

• URL：
  http://guba.eastmoney.com/
• Developer：
  Haojun Gao
• Description：
  This is a project to crawl the stock review website. In the project, I climbed all the comments on the 300 stocks in the HS300 index from January 1, 2015 to the present. In this project I use the framework: pyspider and database operations: mongoDB.
• Dependencies：
  pyspider / mongoDB / redis / lxml / datatime / re
• Operation：
  For more detailed operations, please open the readme in the folder Guba.

Sina Weibo（新浪微博）：

• URL：
  https://weibo.com/
• Developer：
  Zhiqi Liu
• Description：
  This code was designed for 'sina weibo', which is used to grap the content of certain weibo's comment. The only thing you have to do is to decide the range of each user's uid. This code can search user's uid in certain range. But remember that the address shouldn't have Chinese strings.
• Dependencies：
  python3.6

Bilibili（哔哩哔哩动画）：

• URL：
  https://www.bilibili.com/
• Developer：
  Wei Sun
• Description：
  This is a code that can automatically perform corresponding operations on the Bilibili station. The operations that can be performed are following, sending the bullet screen, making comments and saving to favorites.
• Dependencies：
  urllib / requests / ssl / re / time

News_Sina（新浪新闻）：

• URL：
  http://news.sina.com.cn/
• Developer：
  Yexi Huang
• Description：
  This is a simple threaded Python crawler, using BeautifulSoup to make links and news text crawling to Sina News homepage and store it in MySQL. To slightly improve the performance, the crawler crawls only the news text with more than five characters.
• Dependencies：
  Python 3.6 / MySQL 8.0 / bs4 / requests / pymysql / os
• Operation：
  For more detailed operations, please open the readme in the folder News_Sina.

Mobike（摩拜单车）：

• Developer：
  Yuzan Liu
• Description：
  This is a spyder program for mobike location, you can get the location of mobikes of a certain area.
• Dependencies：
  Windows/Linux
  Python2
• Operation：
  For more detailed operations, please open the readme in the folder mobike.