Background

 

In July 2019, as part of X / Twitter’s licensing agreement with Altmetric, we made some updates to the Twitter data that is surfaced in the Altmetric Details Page and Altmetric Explorer APIs, as well as the Altmetric Explorer CSV exports. Below you will find information on exactly what has changed, and how you can work directly with Twitter to obtain any additional information you need going forward. 

 

The Altmetric APIs and data exports will only include the following Twitter-related data:

  • Tweet ID (Twitter’s ID for an individual tweet)

  • User ID (Twitter’s ID for an individual tweeter)

  • Aggregated tweet counts

  • Aggregated demographics (e.g., “member of the public”)

  • Aggregated tweet counts by country

 

Please note that these changes in Twitter data availability only apply to the Altmetric APIs and Explorer CSV exports. The changes do not change what you can see in the Altmetric Explorer or Altmetric Details Pages.


What Twitter data can you access through Altmetric data exports?


Altmetric displays lots of useful information about tweets on our details pages and in the interface of the Explorer, including the date of the tweet, a snippet of its text, and a link that takes you back to Twitter. However, there is a limit to what Altmetric can share in our data exports via CSV and API. The only fields you will find in these exports are the Tweet ID and the Tweeter ID.


What are those two fields? The Tweet ID is a numerical string that you’ll find in the URL of any tweet. For example, see this Tweet URL:


https://twitter.com/DataShift/status/1263804711637078017


The Tweet ID here is 1263804711637078017 - the end of the URL that will take you directly to the tweet itself.


The tweeter ID is also a numerical string, but it refers to the profile that sent the tweet.


Here is an example of that data in the Explorer API:


And in the Details Page API (fetch call):


In the Explorer API, you will find the tweet ID in a field called “external-id”, and in the Details Page API in a field called “tweet_id”. The tweeter ID is found in a field called “id” under “author” in the Explorer API, and in a field called “tweeter_id” in the Details Page API.

If you export the data to a CSV from the Explorer, you will find the Tweet ID in a column titled “External Mention ID,” and the Tweeter ID will be in a column titled “Outlet or Author”


Based on the Tweeter ID and the Tweet ID you could construct the URL to the original tweet like so:


https://twitter.com/895631986718461954/status/920932138668691457


How does Altmetric find geographic data for Tweets?



All of the Twitter data available through Altmetric’s platform and data exports comes from Twitter’s PowerTrack API.


Altmetric provides some geographic context to the twitter attention data displayed. We attempt to associate each Twitter profile with a country. Hence the country-level geographic location associated with a Tweet is based on the information provided in the Tweeter’s profile, and is not related to the individual tweet.


Where does this country association come from? When a user creates an account, they have the option to declare their location. Based on the information they provide in that field, Twitter attempts to derive a country - so if the user says they are in “California,” Twitter will work out that they are in the United States. When Twitter delivers data to us about new tweets, they include a two-letter country code for the profile which Altmetric converts into a full country name. Twitter’s API documentation about this is here: https://support.gnip.com/enrichments/profile_geo.html - we store the 'Activity Streams' fields.


Of course, because the location can be entered freely, a Twitter user could include a fictional place like “Narnia” as their location - in which case neither Altmetric nor Twitter would be able to say what country they were in; as a result there are always tweets for which we are unable to provide a location. This is the case, too, if the user doesn’t provide enough information about their location - for example, they might say they are located at “UCLA,” which Twitter wouldn’t recognize as meaning “Los Angeles” and therefore “California” and therefore the United States.


You can find some interesting background and analysis about our Twitter geolocation data in this blog post from 2018.

 

How to obtain additional Twitter information

 

If you are working on a project that requires information about tweets or tweeters, that is not available via Altmetric’s APIs, then you will need to request the additional data directly from Twitter’s own API services. You can utilize the Tweet IDs and User IDs that you obtain from Altmetric’s APIs to then query the Twitter API for additional information. If you are using Twitter data for your projects, please ensure that you are compliant with Twitter's Developer Policy and Twitter's Terms of Service.

 

First, you will need to apply for developer access to Twitter’s APIs here.

 

Once you have obtained access to Twitter’s Developer Portal, you will be able to send requests to the Twitter API. To access detailed information for individual tweets (which Twitter refers to as “statuses”), you can query the Twitter API using a single Tweet ID.

 

Alternatively, you can request detailed information for a batch of up to 100 tweets at a time using multiple Tweet IDs.


Hydrate Tweets IDs into actual Tweets

 

For hydration, you can use “Hydrator” – see https://github.com/DocNow/hydrator. This great multi-platform app takes as input a bunch of data (e.g., tweet IDs and your own developer key) and in turn goes to the Twitter API, retrieving all the tweets that are still available online. It also manages the download process, including download rate limits.

 

Check the Content Distribution section on this page for more info on Twitter’s platform terms and conditions.