Observatory on Social Media (OSoMe)
The Observatory on Social Media highlights the results of a broad research project aimed to study information diffusion in social media. The public can use our tools to explore how people spread ideas through online social networks.
OSoMe (awe•some) is a set of tools resulting from a broad research project that seeks to study how memes spread online. Our first application was the study of astroturfing: memes promoted by social bots to create the appearance of a grassroots movement. Our first demo to visualze the spread of memes on Twitter was nicknamed Truthy. The word, suggested by a graduate student, comes from a term popularized by Stephen Colbert, truthiness, which describes claims that feel like they ought to be true, but aren't necessarily.
In the years following its creation, OSoMe has spawned many mashups and side projects, including our own Hoaxy. For more information on OSoMe or to use the OSoMe tools, visit the official OSoMe website.
Indiana University students and faculty can obtain access to enhanced OSoMe data featuring additional meta data and tweet content.
Background and how to apply
- How do I access the data?
- There is a form that any IU user can complete to request access to the dataset; requests are evaluated by IUNI's data steward. The full access policy is available here, together with the link to the form: http://iuni.iu.edu/resources/osome#enhanced-data
- Is this a free resource?
- Yes, but it can be used only for academic and educational purposes.
- I am an IU faculty/staff/student but I am not based on the Bloomington campus. May I still access this resource?
- Yes, you may. People from all IU campuses are eligible to apply for access to the data.
- Do I need to file for IRB review to use this resource?
- It depends. As a general rule of thumb, access to the OSoMe Enhanced Data API does not absolve a user from any responsibilities required by the Office of Research Compliance.
- Do I need to create a Twitter account to collect these data?
- No, you do not have to create a Twitter account.
- Am I going to breach the privacy of Twitter users by collecting these data?
- No. Only data from the public Twitter stream can be accessed via the Enhanced Data API. In particular, we comply with the terms of the Twitter Developer Agreement and Policy, and honor the intent of Twitter users to remove from our database any tweet that they decide to delete. Please note that in order to apply for access to the Enhanced Data API you also have to agree and comply with those terms.
Data characteristics, sampling, tools and resources
- Can I download the data? How large is the file?
- The data are only available for download through the OSoMe Enhanced Access API. File size will depend on the type of query you run.
- How much data is available?
- The API can return any tweets in our collection beginning on August 1st, 2016. The collection is based on a 10% sample of all public tweets.
- I need to access tweets that are older than Aug 1, 2016 months. Can you please make older data available?
- Unfortunately due to the limited size of our cluster we cannot maintain data older than Aug 1, 2016. If you are interested in supporting the expansion of the Observatory on Social Media please drop us a line and we would be happy to discuss it with you.
- How is this different from the standard Twitter API?
- The free Twitter Search API returns only results from the last seven days. The new Premium 30-day Search API extends to the last 30 days but is not free. OSoMe’s Enhanced Data API returns results starting in August 2016 and is free to any IU students, faculty, or staff, and includes a range of advanced data aggregation and filtering methods.
- Is the sample statistically valid?
- The API takes advantage of Twitter’s Enterprise real-time sampling algorithms to collect a statistically valid sample of 10% of all public tweets. Please note that this is different from the free Twitter Streaming API, which has been shown to be affected by sampling biases.
- I have used the OSoMe tools before. How does this differ the rest of the tools of the Observatory on Social Media?
- To comply with Twitter’s Developer Agreement, the amount of data that you can export from our current public front end tools is capped. Moreover, you can only export numerical tweet and user IDs. This means that even when your data fits within the limit, you still have to use the Twitter API to collect each individual tweet. The new Enhanced Access API is not restricted by these limits, but is available only to IU users by virtue of an academic partnership between Indiana University and Twitter, inc.
- I have heard that a lot of Twitter users are fake or automated. How can I tell which tweets are from human users?
- Please check Botometer, our free social bot detection tool.
- Can I visualize these data?
- With the Observatory on Social Media you can visualize temporal trends, geographic distributions, and diffusion networks, and even make movies. For the full list of public tools, please see osome.iuni.iu.edu/tools
- Can I read more about OSoMe? How can I acknowledge this resource in my work?
- For more information and/or a suggested citation, please refer to our open-access publication on the Observatory on Social Media:
Davis CA, Ciampaglia GL, Aiello LM, Chung K, Conover MD, Ferrara E, Flammini A, Fox GC, Gao X, Gonçalves B, Grabowicz PA, Hong K, Hui P, McCaulay S, McKelvey K, Meiss MR, Patil S, Peli Kankanamalage C, Pentchev V, Qiu J, Ratkiewicz J, Rudnick A, Serrette B, Shiralkar P, Varol O, Weng L, Wu T, Younge AJ, Menczer F. (2016) OSoMe: the IUNI observatory on social media. PeerJ Computer Science 2:e87 https://doi.org/10.7717/peerj-cs.87
If you use OSoMe and/or the new Enhanced Access API in your work, please acknowledge support from the Indiana University Network Science Institute. Suggested acknowledgement formula:
Access to Twitter data was obtained thanks to support from the Indiana University Network Science Institute through the Observatory on Social Media (osome.iuni.iu.edu), which is also supported by the Digital Science Center (dsc.sice.indiana.edu) and the Center for Complex Networks and Systems Research (cnets.indiana.edu).