Background

 

In July 2019, as part of X licensing agreement with Altmetric, we made some updates to the X data that is surfaced in the Altmetric Details Page and Altmetric Explorer APIs, as well as the Altmetric Explorer CSV exports. Below you will find information on exactly what has changed, and how you can work directly with X to obtain any additional information you need going forward. 


The API and CSV exports currently will display X details as Tweet/Twitter. This is due to be updated in the near future inline with the recent platform changes.

 

The Altmetric APIs and CSV data exports will only include the following X related data where applicable:

  • Post/Tweet ID (X ID for an individual post)

  • User ID (X ID for an individual X User)

  • Aggregated post counts

  • Aggregated demographics (e.g., “member of the public”)

  • Aggregated post counts by country

 

Please note that these changes in X data availability only apply to the Altmetric APIs and Explorer CSV exports. The changes do not change what you can see in the Altmetric Explorer or on the Altmetric Details Pages.


What X data can you access through Altmetric data exports?


Altmetric displays lots of useful information about X posts on our details pages and in the interface of the Explorer, including the date of the post, a snippet of its text, and a link that takes you back to X. However, there is a limit to what Altmetric can share in our data exports via CSV and API. The only fields you will find in these exports are the post/tweet ID and the user ID.


What are those two fields? The post ID is a numerical string that you’ll find in the URL of any X post. For example, see this post URL:


https://twitter.com/DataShift/status/1263804711637078017


The post/tweet ID here is 1263804711637078017 - the end of the URL that will take you directly to the post itself.


The user ID is also a numerical string, but it refers to the profile that sent the post.


Here is an example of that data in the Explorer API:


And in the Details Page API (fetch call):


In the Explorer API, you will find the Post/Tweet ID in a field called “external-id”, and in the Details Page API in a field called “tweet_id”. The tweeter ID is found in a field called “id” under “author” in the Explorer API, and in a field called “tweeter_id” in the Details Page API. These will soon be updated to accommodate the  recent name change.


If you export the data to a CSV from the Explorer, you will find the Post/Tweet ID in a column titled “External Mention ID,” and the User/Tweeter ID will be in a column titled “Outlet or Author”


Based on these IDs you could construct the URL to the original post like so:


https://twitter.com/895631986718461954/status/920932138668691457


How does Altmetric find geographic data for X posts?



All of the X data available through Altmetric’s platform and data exports comes from the X PowerTrack API.


Altmetric provides some geographic context to the X attention data displayed. We attempt to associate each X profile with a country. Hence the country-level geographic location associated with a post is based on the information provided in the users profile, and is not related to the individual post.


Where does this country association come from? When a user creates an account, they have the option to declare their location. Based on the information they provide in that field, X attempts to derive a country - so if the user says they are in “California,” X will work out that they are in the United States. When X delivers data to us about new posts, they include a two-letter country code for the profile which Altmetric converts into a full country name. X's API documentation about this is here: https://support.gnip.com/enrichments/profile_geo.html - we store the 'Activity Streams' fields.


Of course, because the location can be entered freely, an X user could include a fictional place like “Narnia” as their location - in which case neither Altmetric nor X would be able to say what country they were in; as a result there are always posts for which we are unable to provide a location. This is the case, too, if the user doesn’t provide enough information about their location - for example, they might say they are located at “UCLA,” which X wouldn’t recognize as meaning “Los Angeles” and therefore “California” and therefore the United States.


You can find some interesting background and analysis about our X geolocation data in this blog post from 2018.

 

How to obtain additional X information

 

If you are working on a project that requires information about posts or users, that is not available via Altmetric’s APIs, then you will need to request the additional data directly from X's own API services. You can utilize the Tweet/Post IDs and User IDs that you obtain from Altmetric’s APIs to then query the X API for additional information. If you are using X data for your projects, please ensure that you are compliant with X's Developer Policy and X's Terms of Service.

 

First, you will need to apply for developer access to X’s APIs here.

 

Once you have obtained access to X’s Developer Portal, you will be able to send requests to the X API. To access detailed information for individual posts (which X refers to as “statuses”), you can query the X API using a single post/tweet ID.

 

Alternatively, you can request detailed information for a batch of up to 100 posts at a time using multiple post/tweet IDs.


Hydrate Post IDs into actual Posts/Tweets

 

For hydration, you can use “Hydrator” – see https://github.com/DocNow/hydrator. This great multi-platform app takes as input a bunch of data (e.g., post IDs and your own developer key) and in turn goes to the X API, retrieving all the posts that are still available online. It also manages the download process, including download rate limits.

 

Check the Content Distribution section on this page for more info on X’s platform terms and conditions.