Retrieve Records from the Urban Institute's Education Data Portal API.

educationdata

Retrieve data from the Urban Institute’s Education Data API as a data.frame for easy analysis.

NOTE: By downloading and using this programming package, you agree to abide by the Data Policy and Terms of Use of the Education Data Portal.

Installation

You can install the released version of educationdata from CRAN with:

install.packages("educationdata")

And the development version from GitHub with:

# install.packages('devtools') # if necessary
devtools::install_github('UrbanInstitute/education-data-package-r')

Usage

library(educationdata)

df <- get_education_data(level = 'schools', 
                         source = 'ccd', 
                         topic = 'enrollment', 
                         subtopic = list('race', 'sex'),
                         filters = list(year = 2008,
                                        grade = 9:12,
                                        ncessch = '340606000122'),
                         add_labels = TRUE)

str(df)
#> 'data.frame':    96 obs. of  9 variables:
#>  $ year       : int  2008 2008 2008 2008 2008 2008 2008 2008 2008 2008 ...
#>  $ ncessch    : chr  "340606000122" "340606000122" "340606000122" "340606000122" ...
#>  $ ncessch_num: num  3.41e+11 3.41e+11 3.41e+11 3.41e+11 3.41e+11 ...
#>  $ grade      : Factor w/ 19 levels "Pre-K","Kindergarten",..: 11 11 11 11 11 11 11 11 11 11 ...
#>  $ race       : Factor w/ 14 levels "White","Black",..: 2 3 5 5 2 4 6 11 1 7 ...
#>  $ sex        : Factor w/ 7 levels "Male","Female",..: 1 1 2 1 2 2 2 1 2 1 ...
#>  $ enrollment : int  41 39 0 0 46 32 3 270 166 0 ...
#>  $ fips       : Factor w/ 79 levels "Alabama","Alaska",..: 34 34 34 34 34 34 34 34 34 34 ...
#>  $ leaid      : chr  "3406060" "3406060" "3406060" "3406060" ...

The get_education_data() function will return a data.frame from a call to the Education Data API.

get_education_data(level, source, topic, subtopic, filters, add_labels)

where:

• level (required) - API data level to query.
• source (required) - API data source to query.
• topic (required) - API data topic to query.
• subtopic (optional) - Optional list of grouping parameters for an API call.
• filters (optional) - Optional list query to filter the results from an API call.
• add_labels - Add variable labels (when applicable)? Defaults to FALSE.
• csv - Download the full csv file? Defaults to FALSE.

Available Endpoints

Main Filters

Due to the way the API is set-up, the variables listed within ‘main filters’ are the fastest way to subset an API call.

In addition to year, the other main filters for certain endpoints accept the following values:

Grade

Level of Study

Examples

Let’s build up some examples, from the following set of endpoints.

The following will return a data.frame across all years and grades:

library(educationdata)
df <- get_education_data(level = 'schools', 
                         source = 'ccd', 
                         topic = 'enrollment')

Note that this endpoint is also callable by certain subtopic variables:

• race
• sex
• race, sex

These variables can be added to the subtopic argument:

df <- get_education_data(level = 'schools', 
                         source = 'ccd', 
                         topic = 'enrollment', 
                         subtopic = list('race', 'sex'))

You may also filter the results of an API call. In this case year and grade will provide the most time-efficient subsets, and can be vectorized:

df <- get_education_data(level = 'schools', 
                         source = 'ccd', 
                         topic = 'enrollment', 
                         subtopic = list('race', 'sex'),
                         filters = list(year = 2008,
                                        grade = 9:12))

Additional variables can also be passed to filters to subset further:

df <- get_education_data(level = 'schools', 
                         source = 'ccd', 
                         topic = 'enrollment', 
                         subtopic = list('race', 'sex'),
                         filters = list(year = 2008,
                                        grade = 9:12,
                                        ncessch = '3406060001227'))

The add_labels flag will map variables to a factor from their labels in the API.

df <- get_education_data(level = 'schools', 
                         source = 'ccd', 
                         topic = 'enrollment', 
                         subtopic = list('race', 'sex'),
                         filters = list(year = 2008,
                                        grade = 9:12,
                                        ncessch = '340606000122'),
                         add_labels = TRUE)

Finally, the csv flag can be set to download the full .csv data frame. In general, the csv functionality is much faster when retrieving the full data frame (or a large subset) and much slower when retrieving a small subset of a data frame (especially ones with a lot of filters added). In this example, the full csv for 2008 must be downloaded and then subset to the 96 observations.

df <- get_education_data(level = 'schools', 
                         source = 'ccd', 
                         topic = 'enrollment', 
                         subtopic = list('race', 'sex'),
                         filters = list(year = 2008,
                                        grade = 9:12,
                                        ncessch = '340606000122'),
                         add_labels = TRUE,
                         csv = TRUE)

Summary Endpoints

You can access the summary endpoint functionality using the get_education_data_summary() function.

df <- get_education_data_summary(
    level = "schools",
    source = "ccd",
    topic = "enrollment",
    stat = "sum",
    var = "enrollment",
    by = "fips",
    filters = list(fips = 6:8, year = 2004:2005)
)

In this example, we take the schools/ccd/enrollment endpoint and retrieve the sum of enrollment by fips code, filtered to fips codes 6, 7, 8 for the years 2004 and 2005.

The syntax largely follows the original syntax of get_education_data(): with three new arguments:

• stat is the summary statistic to be retrieved. Valid statistics include: avg, sum, count, median, min, max, stddev, and variance.
• var is the variable to run the summary statistic on.
• by is the grouping variable(s) to use. This can be a single string, or a vector of multiple variables, i.e., by = c("fips", "race").

Some endpoints are further broken out by subtopic. These can be specified using the subtopic option.

df <- get_education_data_summary(
    level = "schools",
    source = "crdc",
    topic = "harassment-or-bullying",
    subtopic = "allegations",
    stat = "sum",
    var = "allegations_harass_sex",
    by = "fips"
)

Note that only some endpoints have an applicable subtopic, and this list is slightly different from the syntax of the full data API. Endpoints with subtopics for the summary endpoint functionality include:

• schools/crdc/harassment-or-bullying/allegations
• schools/crdc/harassment-or-bullying/students
• schools/crdc/restraint-and-seclusion/instances
• schools/crdc/restraint-and-seclusion/students
• college-university/ipeds/enrollment-full-time-equivalent/summaries
• college-university/ipeds/fall-enrollment/age/summaries
• college-university/ipeds/fall-enrollment/race/summaries
• college-university/ipeds/fall-enrollment/residence/summaries
• college-university/scorecard/student-characteristics/aid-applicants/summaries
• college-university/scorecard/student-characteristics/home-neighborhood/summaries

For more information on the summary endpoint functionality, see the full API documentation.