Skip to content

v0.6.0: Threaded Update

Latest
Compare
Choose a tag to compare
@ProgrammingIncluded ProgrammingIncluded released this 25 Nov 09:00
738caec

This update is all about threads⚡! Three main large changes:

  • Adds twitter threads tracking via --depth 2
    *Tweets within a thread now have their tweets who they are replying to saved as parent_id.
    • Screenshots are taken for each thread.
  • Adds multi-threading support via --multi-threading N
  • Massive internal architectural changes.

Changes

  • Added --depth and -d for archiving threads.
  • Added --multi-threading and -t to specify number of threads to use.
  • Added parent_id to tweets.json which tracks direct tweet responses.
  • Added threading.py which manages thread workers and queues.
  • Added pages.py which abstracts away a page on Twitter. Useful for multi-threading.
    • Added TwitterPage used for representing a page in Twitter.
    • Added TwitterBio which is used for fetching and obtaining bio pages.
    • Added TwitterThread which represents a tweet thread page.
  • Added TweetExtractor which attempts to fetch all available Tweets on a page.
  • Added Scroller which abstracts away scrolling metrics.
  • Added thread expansion functions: hit_more_replies() and get_recommend_tweets_height().
  • Changed random messages from ChromeDriver, makes logs cleaner.
  • Fixed potential for some posts to be skipped on ad removal.
  • Fixed logger not outputting any info.
  • Fixed logger not printing debug.