Using Dimensions organization groups with the API¶
This tutorial shows how use the organization groups in Dimensions (e.g. the funder groups) in order to construct API queries.
The Dimensions team maintains various organization groups definitions in the main Dimensions web application. These groups are not available directly via the API, but since they are a simple list of GRID identifiers, they can be easily downloaded as a CSV file. Once you have a CSV file, it is possible to parse it with Python and use its contents in an API query.
Outline
Downloading Dimensions’ organization groups as a CSV file.
Constructing API queries using a list of GRID IDs
[1]:
import datetime
print("==\nCHANGELOG\nThis notebook was last run on %s\n==" % datetime.date.today().strftime('%b %d, %Y'))
==
CHANGELOG
This notebook was last run on Feb 21, 2022
==
Prerequisites¶
This notebook assumes you have installed the Dimcli library and are familiar with the Getting Started tutorial.
[5]:
!pip install dimcli plotly --quiet
import dimcli
from dimcli.utils import *
import json
import sys
import pandas as pd
print("==\nLogging in..")
ENDPOINT = "https://app.dimensions.ai"
if 'google.colab' in sys.modules:
import getpass
KEY = getpass.getpass(prompt='API Key: ')
dimcli.login(key=KEY, endpoint=ENDPOINT)
else:
KEY = ""
dimcli.login()
dsl = dimcli.Dsl()
Searching config file credentials for default 'live' instance..
==
Logging in..
Dimcli - Dimensions API Client (v0.9.6)
Connected to: <https://app.dimensions.ai/api/dsl/v2> - DSL v2.0
Method: dsl.ini file
1. Downloading groups data from Dimensions¶
This can be done in three simple steps, using the Dimensions web application:
Go to the ‘Browse Groups’ area of Dimensions and open the group page you are interested in (eg this is the funder groups page).
Use the ‘Copy to my Groups’ command to create a copy of that group in your personal space.
Go to ‘My Groups’, where you can select ‘Export group definitions’ to download a CSV file containing the groups details including GRID IDs.
See below a screenshot of the Dimensions’ groups page.
NOTE if you have multiple groups in your personal area, you may want to edit out the ones you are not using via the API e.g. via Excel of some other CSV editor.
[7]:
from IPython.display import Image
Image(url= "http://api-sample-data.dimensions.ai/data/funder-groups/dimensions-funder-groups-page.jpg", width=800)
[7]:
2. Using group data with the API¶
For this part, we’ll be using a sample CSV export of NSF organizations.
We can load it with pandas as follows:
[9]:
# load a CSV containing funder group infos, exported from Dimensions
data = pd.read_csv("http://api-sample-data.dimensions.ai/data/funder-groups/funder-groups-export-sample.csv")
data
[9]:
| Filter type | Group | Name | ID | |
|---|---|---|---|---|
| 0 | Funder | NSF-mine | Directorate for Biological Sciences (NSF BIO) | grid.457768.f |
| 1 | Funder | NSF-mine | Directorate for Computer & Information Science... | grid.457785.c |
| 2 | Funder | NSF-mine | Directorate for Education & Human Resources (N... | grid.457799.1 |
| 3 | Funder | NSF-mine | Directorate for Engineering (NSF ENG) | grid.457810.f |
| 4 | Funder | NSF-mine | Directorate for Geosciences (NSF GEO) | grid.457836.b |
| 5 | Funder | NSF-mine | Directorate for Mathematical & Physical Scienc... | grid.457875.c |
| 6 | Funder | NSF-mine | Directorate for Social, Behavioral & Economic ... | grid.457916.8 |
| 7 | Funder | NSF-mine | Division of Advanced Cyberinfrastructure (NSF ... | grid.457789.0 |
| 8 | Funder | NSF-mine | Division of Chemical, Bioengineering, Environm... | grid.457813.c |
| 9 | Funder | NSF-mine | Division of Civil, Mechanical & Manufacturing ... | grid.457814.b |
| 10 | Funder | NSF-mine | Division of Earth Sciences (NSF EAR) | grid.457842.8 |
| 11 | Funder | NSF-mine | Division of Engineering Education & Centers (N... | grid.457821.d |
| 12 | Funder | NSF-mine | Division of Environmental Biology (NSF DEB) | grid.457772.4 |
| 13 | Funder | NSF-mine | Division of Graduate Education (NSF DGE) | grid.457801.f |
| 14 | Funder | NSF-mine | Division of Materials Research (NSF DMR) | grid.457891.6 |
| 15 | Funder | NSF-mine | Division of Mathematical Sciences (NSF DMS) | grid.457892.5 |
| 16 | Funder | NSF-mine | Division of Ocean Sciences (NSF OCE) | grid.457845.f |
| 17 | Funder | NSF-mine | Division of Social and Economic Sciences (NSF ... | grid.457922.f |
| 18 | Funder | NSF-mine | National Science Board (NSF NSB) | grid.457896.1 |
| 19 | Funder | NSF-mine | National Science Foundation (NSF) | grid.431093.c |
| 20 | Funder | NSF-mine | Office of Budget, Finance and Award Management... | grid.457758.c |
| 21 | Funder | NSF-mine | Office of Information and Resource Management ... | grid.457907.8 |
| 22 | Funder | NSF-mine | Office of Inspector General (OIG) | grid.473792.c |
| 23 | Funder | NSF-mine | Office of Polar Programs (NSF PLR) | grid.457846.c |
| 24 | Funder | NSF-mine | Office of the Director (NSF OD) | grid.457898.f |
Let’s get the GRID IDs for the NSF and put them into a Python list.
Then we can generate queries programmatically using this list.
For more background on this topic, see the Working with lists in the Dimensions API tutorial.
[11]:
nsfgrids = data['ID'].to_list()
nsfgrids
[11]:
['grid.457768.f',
'grid.457785.c',
'grid.457799.1',
'grid.457810.f',
'grid.457836.b',
'grid.457875.c',
'grid.457916.8',
'grid.457789.0',
'grid.457813.c',
'grid.457814.b',
'grid.457842.8',
'grid.457821.d',
'grid.457772.4',
'grid.457801.f',
'grid.457891.6',
'grid.457892.5',
'grid.457845.f',
'grid.457922.f',
'grid.457896.1',
'grid.431093.c',
'grid.457758.c',
'grid.457907.8',
'grid.473792.c',
'grid.457846.c',
'grid.457898.f']
How many grants from the NSF?¶
Let’s try a simple API query that uses the contents of nsfgrids.
The total number of results should match what you see in Dimensions.
[12]:
import json
query = f"""
search grants
where funders.id in {json.dumps(nsfgrids)}
return grants[id+title]
"""
print(query)
grants = dsl.query(query)
grants.as_dataframe()
search grants
where funders.id in ["grid.457768.f", "grid.457785.c", "grid.457799.1", "grid.457810.f", "grid.457836.b", "grid.457875.c", "grid.457916.8", "grid.457789.0", "grid.457813.c", "grid.457814.b", "grid.457842.8", "grid.457821.d", "grid.457772.4", "grid.457801.f", "grid.457891.6", "grid.457892.5", "grid.457845.f", "grid.457922.f", "grid.457896.1", "grid.431093.c", "grid.457758.c", "grid.457907.8", "grid.473792.c", "grid.457846.c", "grid.457898.f"]
return grants[id+title]
Returned Grants: 20 (total = 601237)
Time: 2.03s
[12]:
| id | title | |
|---|---|---|
| 0 | grant.9752271 | NNA Planning: Developing community frameworks ... |
| 1 | grant.9890102 | RUI: Exciton-Phonon Interactions in Solids bas... |
| 2 | grant.9982417 | CAREER: Empowering White-box Driven Analytics ... |
| 3 | grant.9982416 | CAREER: Holistic Framework for Constructing Dy... |
| 4 | grant.9982395 | CAREER: Leveraging physical properties of mode... |
| 5 | grant.9785674 | BPC-AE Collaborative Research: Researching Equ... |
| 6 | grant.9785672 | BPC-AE Collaborative Research: Researching Equ... |
| 7 | grant.9752397 | Equitable Learning to Advance Technical Education |
| 8 | grant.9995499 | CAREER: New imaging of mid-ocean ridge systems... |
| 9 | grant.9995464 | CAREER: Reconstructing Parasite Abundance in R... |
| 10 | grant.9752334 | Collaborative Research: SWIFT: Intelligent Dyn... |
| 11 | grant.9752333 | Collaborative Research: SWIFT: Intelligent Dyn... |
| 12 | grant.9995542 | CAREER: Learning Mechanisms from Single Cell M... |
| 13 | grant.9995538 | CAREER: A Transformative Approach for Teaching... |
| 14 | grant.9995527 | CAREER: Interlimb Neural Coupling to Enhance G... |
| 15 | grant.9995522 | CAREER: Fossil Amber Insight Into Macroevoluti... |
| 16 | grant.9995520 | 2022 Origins of Life GRC and GRS: Environments... |
| 17 | grant.9995519 | CAREER: Invariants and Entropy of Square Integ... |
| 18 | grant.9995488 | CAREER: Statistical Learning from a Modern Per... |
| 19 | grant.9995470 | CAREER: CAS- Climate: Making Decarbonization o... |
Note
The Dimensions Analytics API allows to carry out sophisticated research data analytics tasks like the ones described on this website. Check out also the associated Github repository for examples, the source code of these tutorials and much more.
