INFS7203-PROJ

UQ ITEE - INFS7203 Project (Sem2, 2016)

What's in it

1. Collecting data of users and the Tweets of some specific users

2. TF-IDF and Agglomerative Clustering to find the clusters of Tweets

3. LDA to analysis the topics

Package Used

• sklearn, gensim, numpy, nltk, pymongo, mongoengine, Tweepy

How to use

1. Create your app on Twitter Developer's website and copy your tokens to twitter_client.py

2. (Optional) If you want to collect the information of a list of users, and you have created a Twitter list with users, change the list name and run text/get_members.py. Remember to set up you local MongoDB because it is needed in this step. The MongoDB schema is in mongodb.py.

3. Run text/get_timeline.py. You can get the timeline of multiple users if you have already finished step 2. If you want to collect the timeline of a specific user, just modify the user's screen name.

4. Run text/text_mining.py. If you want to collect the timeline of a specific user, just modify the user's screen name.