Library Guides: Text Mining & Computational Text Analysis: Social Media & Web

Social Media & Internet Sources

Blogger Corpus (2004)
The collected posts of 19,320 bloggers gathered from blogger.com in August 2004. The corpus incorporates a total of 681,288 posts.
Internet Archive
How to download files from archive.org in an automated way using wget.
reddit APIs
Access data from posts, threads, comments, users and more from reddit and subreddits.
Historical Reddit data has been collected at http://files.pushshift.io/reddit/ as monthly CSV downloads.
Stanford Large Network Dataset Collection
The SNAP library collects data on large social and information networks since 2004.
Twitter Streaming APIs
Public streams provide access to public data flowing through Twitter. Suitable for following specific users or topics, and data mining. You can also access single-user streams, containing roughly all of the data corresponding with a single user’s view of Twitter.
Wikipedia Data Dumps
Monthly database backups of all Wikimedia wikis in various formats.
Yelp API
Access to business data, including location, photos, Yelp rating, price levels, hours of operation, and types of transactions. Also includes a Review API, which returns up to 3 review excerpts for a business. See also: the Yelp Dataset Academic Challenge.