Skip to contents

Download and read a resource from a B.C. Data Catalogue record

Usage

bcdc_get_data(record, resource = NULL, verbose = TRUE, ...)

Arguments

record

either a bcdc_record object (from the result of bcdc_get_record()), a character string denoting the name or ID of a resource (or the URL) or a BC Geographic Warehouse (BCGW) name.

It is advised to use the permanent ID for a record or the BCGW name rather than the human-readable name to guard against future name changes of the record. If you use the human-readable name a warning will be issued once per session. You can silence these warnings altogether by setting an option: options("silence_named_get_data_warning" = TRUE) - which you can set in your .Rprofile file so the option persists across sessions.

resource

optional argument used when there are multiple data files within the same record. See examples.

verbose

When more than one resource is available for a record, should extra information about those resources be printed to the console? Default TRUE

...

arguments passed to other functions. Tabular data is passed to a function to handle the import based on the file extension. bcdc_read_functions() provides details on which functions handle the data import. You can then use this information to look at the help pages of those functions. See the examples for a workflow that illustrates this process. For spatial Web Feature Service data the ... arguments are passed to bcdc_query_geodata().

Value

An object of a type relevant to the resource (usually a tibble or an sf object)

Examples

# \donttest{
# Using the record and resource ID:
try(
  bcdc_get_data(record = '76b1b7a3-2112-4444-857a-afccf7b20da8',
                resource = '4d0377d9-e8a1-429b-824f-0ce8f363512c')
)
#> Error : There was an issue sending this WFS request
#> ▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
#> Request:
#>   URL: https://openmaps.gov.bc.ca/geo/pub/wfs
#>   POST fields:
#>     resultType=hits&SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&outputFormat=application%2Fjson&typeNames=WHSE_IMAGERY_AND_BASE_MAPS.GSR_AIRPORTS_SVW&SRSNAME=EPSG%3A3005
#>   Content-Type: application/x-www-form-urlencoded
#>   Accept-Encoding: gzip, deflate
#>   Accept: application/json, text/xml, application/xml, */*
#>   User-Agent: https://github.com/bcgov/bcdata
#> Response:
#>   status: HTTP/1.1 503
#>   date: Tue, 05 Jul 2022 21:16:23 GMT
#>   server: Apache
#>   x-frame-options: allow-from (null)
#>   x-control-flow-delay-ms: 60004
#>   content-type: text/html;charset=utf-8
#>   content-language: en
#>   content-length: 1201
#>   access-control-allow-origin: (null)
#>   access-control-allow-credentials: true
#>   access-control-allow-methods: POST, GET, OPTIONS, HEAD
#>   access-control-allow-headers: X-Requested-With, Referer, Origin, Content-Type, SOAPAction, Authorization, Accept
#>   access-control-max-age: 1000
#>   connection: close
#> 

try(
  bcdc_get_data('1d21922b-ec4f-42e5-8f6b-bf320a286157')
)
#> Reading the data using the read_csv function from the readr package.
#> Rows: 357 Columns: 4
#> ── Column specification ────────────────────────────────────
#> Delimiter: ","
#> chr (4): CITY, NAME, ADDRESS, POSTALCODE
#> 
#>  Use `spec()` to retrieve the full column specification for this data.
#>  Specify the column types or set `show_col_types = FALSE` to quiet this message.
#> # A tibble: 357 × 4
#>    CITY          NAME                     ADDRESS POSTALCODE
#>    <chr>         <chr>                    <chr>   <chr>     
#>  1 PRINCE GEORGE Pacific Western Brewing… 641 NO… V2K4M4    
#>  2 OSOYOOS       Bordertown Vineyards     9140 9… V0H1V2    
#>  3 CHILLIWACK    Whispering Horse Winery  43721 … V2R4C5    
#>  4 KELOWNA       Okanaganvilla Estate Wi… 3240 P… V1W4G7    
#>  5 OLIVER        Church & State Wines (B… 31120 … V0H1T0    
#>  6 KEREMEOS      Corcelettes Estate Wine… 2582 U… V0X1N4    
#>  7 DUNCAN        Blue Grouse Vineyards    4365 B… V9L6M3    
#>  8 VANCOUVER     City Side Winery         328 WE… V5Y1C8    
#>  9 WEST KELOWNA  Volcanic Hills Estate W… 2845 B… V1Z2G6    
#> 10 OLIVER        Orchard Hill Estate Cid… 3480 F… V0H1T0    
#> # … with 347 more rows

# Using a `bcdc_record` object obtained from `bcdc_get_record`:
try(
  record <- bcdc_get_record('1d21922b-ec4f-42e5-8f6b-bf320a286157')
)

try(
  bcdc_get_data(record)
)
#> Reading the data using the read_csv function from the readr package.
#> Rows: 357 Columns: 4
#> ── Column specification ────────────────────────────────────
#> Delimiter: ","
#> chr (4): CITY, NAME, ADDRESS, POSTALCODE
#> 
#>  Use `spec()` to retrieve the full column specification for this data.
#>  Specify the column types or set `show_col_types = FALSE` to quiet this message.
#> # A tibble: 357 × 4
#>    CITY          NAME                     ADDRESS POSTALCODE
#>    <chr>         <chr>                    <chr>   <chr>     
#>  1 PRINCE GEORGE Pacific Western Brewing… 641 NO… V2K4M4    
#>  2 OSOYOOS       Bordertown Vineyards     9140 9… V0H1V2    
#>  3 CHILLIWACK    Whispering Horse Winery  43721 … V2R4C5    
#>  4 KELOWNA       Okanaganvilla Estate Wi… 3240 P… V1W4G7    
#>  5 OLIVER        Church & State Wines (B… 31120 … V0H1T0    
#>  6 KEREMEOS      Corcelettes Estate Wine… 2582 U… V0X1N4    
#>  7 DUNCAN        Blue Grouse Vineyards    4365 B… V9L6M3    
#>  8 VANCOUVER     City Side Winery         328 WE… V5Y1C8    
#>  9 WEST KELOWNA  Volcanic Hills Estate W… 2845 B… V1Z2G6    
#> 10 OLIVER        Orchard Hill Estate Cid… 3480 F… V0H1T0    
#> # … with 347 more rows

# Using a BCGW name
try(
  bcdc_get_data("WHSE_IMAGERY_AND_BASE_MAPS.GSR_AIRPORTS_SVW")
)
#> Error : There was an issue sending this WFS request
#> ▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
#> Request:
#>   URL: https://openmaps.gov.bc.ca/geo/pub/wfs
#>   POST fields:
#>     resultType=hits&SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&outputFormat=application%2Fjson&typeNames=WHSE_IMAGERY_AND_BASE_MAPS.GSR_AIRPORTS_SVW&SRSNAME=EPSG%3A3005
#>   Content-Type: application/x-www-form-urlencoded
#>   Accept-Encoding: gzip, deflate
#>   Accept: application/json, text/xml, application/xml, */*
#>   User-Agent: https://github.com/bcgov/bcdata
#> Response:
#>   status: HTTP/1.1 503
#>   date: Tue, 05 Jul 2022 21:17:28 GMT
#>   server: Apache
#>   x-frame-options: allow-from (null)
#>   x-control-flow-delay-ms: 60002
#>   content-type: text/html;charset=utf-8
#>   content-language: en
#>   content-length: 1201
#>   access-control-allow-origin: (null)
#>   access-control-allow-credentials: true
#>   access-control-allow-methods: POST, GET, OPTIONS, HEAD
#>   access-control-allow-headers: X-Requested-With, Referer, Origin, Content-Type, SOAPAction, Authorization, Accept
#>   access-control-max-age: 1000
#>   connection: close
#> 

# Using sf's sql querying ability
try(
  bcdc_get_data(
    record = '30aeb5c1-4285-46c8-b60b-15b1a6f4258b',
    resource = '3d72cf36-ab53-4a2a-9988-a883d7488384',
    layer = 'BC_Boundary_Terrestrial_Line',
    query = "SELECT SHAPE_Length, geom FROM BC_Boundary_Terrestrial_Line WHERE SHAPE_Length < 100"
  )
)
#> Reading the data using the read_sf function from the sf package.
#> Warning: argument layer is ignored when query is specified
#> Simple feature collection with 6106 features and 1 field
#> Geometry type: LINESTRING
#> Dimension:     XY
#> Bounding box:  xmin: 529420.8 ymin: 367780.7 xmax: 1203949 ymax: 1157953
#> Projected CRS: NAD83 / BC Albers
#> # A tibble: 6,106 × 2
#>    SHAPE_Length                                         geom
#>           <dbl>                             <LINESTRING [m]>
#>  1         79.2 (1183654 367787.5, 1183637 367780.7, 118363…
#>  2         92.8 (1178975 369462.2, 1178962 369460.6, 117895…
#>  3         71.0 (1178340 369486.6, 1178335 369481.4, 117832…
#>  4         80.2 (1182295 369909.3, 1182289 369909.1, 118228…
#>  5         90.7 (1177992 370787.7, 1177986 370785.4, 117797…
#>  6         94.3 (1178597 370831.9, 1178591 370831.7, 117858…
#>  7         75.3 (1178008 371008.1, 1178002 371005.8, 117799…
#>  8         77.4 (1178022 371104.5, 1178017 371100.4, 117801…
#>  9         76.7 (1176492 371248.6, 1176485 371248.3, 117647…
#> 10         87.2 (1178786 371574.1, 1178778 371572.7, 117877…
#> # … with 6,096 more rows

## Example of correcting import problems

## Some initial problems reading in the data
try(
  bcdc_get_data('d7e6c8c7-052f-4f06-b178-74c02c243ea4')
)
#> Reading the data using the read_xlsx function from the readxl package.
#> 
#> This .xlsx resource contains the following sheets: 
#>  'Population'
#>  'Gender & Age Distribution'
#>  'Median, Average & Growth Rates'
#>  'Households'
#>  'Household Size'
#> Defaulting to the 'Population' sheet. See ?bcdc_get_data for examples on how to specify a sheet.
#> New names:
#>  `` -> `...2`
#>  `` -> `...3`
#>  `` -> `...4`
#>  `` -> `...5`
#>  `` -> `...6`
#>  `` -> `...7`
#>  `` -> `...8`
#>  `` -> `...9`
#>  `` -> `...10`
#>  `` -> `...11`
#> # A tibble: 901 × 11
#>    `Regional District …` ...2  ...3  ...4  ...5  ...6  ...7 
#>    <chr>                 <chr> <chr> <chr> <chr> <chr> <chr>
#>  1 RD Code               Regi… Sex   Year  Age … 15-19 20-24
#>  2 1000                  East… Male  2019  4863  1651  1838 
#>  3 1000                  East… Male  2020  4850  1607  1812 
#>  4 1000                  East… Male  2021  4850  1569  1819 
#>  5 1000                  East… Male  2022  4785  1594  1693 
#>  6 1000                  East… Male  2023  4756  1577  1622 
#>  7 1000                  East… Male  2024  4731  1574  1561 
#>  8 1000                  East… Male  2025  4673  1604  1512 
#>  9 1000                  East… Male  2026  4590  1659  1483 
#> 10 1000                  East… Male  2027  4515  1645  1510 
#> # … with 891 more rows, and 4 more variables: ...8 <chr>,
#> #   ...9 <chr>, ...10 <chr>, ...11 <chr>

## From bcdc_get_record we realize that the data is in xlsx format
try(
 bcdc_get_record('8620ce82-4943-43c4-9932-40730a0255d6')
)
#> B.C. Data Catalogue Record: New Homes
#>  Registry (2016-2021)
#> Name: new-homes-registry-2016-2021-
#>  (ID: 8620ce82-4943-43c4-9932-40730a0255d6)
#> Permalink:
#>  https://catalogue.data.gov.bc.ca/dataset/8620ce82-4943-43c4-9932-40730a0255d6
#> Licence: Access Only
#> Description: This dataset contains
#>  information from BC Housing's New Homes Registry.
#> Available Resources (1):
#>  1. New Homes Registrations 2016-2021 (xlsx)
#> Access the full 'Resources' data frame using:
#>  bcdc_tidy_resources('8620ce82-4943-43c4-9932-40730a0255d6')

## bcdc_read_functions let's us know that bcdata
## uses readxl::read_excel to import xlsx files
try(
 bcdc_read_functions()
)
#> # A tibble: 11 × 3
#>    format  package fun      
#>    <chr>   <chr>   <chr>    
#>  1 kml     sf      read_sf  
#>  2 geojson sf      read_sf  
#>  3 gpkg    sf      read_sf  
#>  4 gdb     sf      read_sf  
#>  5 fgdb    sf      read_sf  
#>  6 shp     sf      read_sf  
#>  7 csv     readr   read_csv 
#>  8 txt     readr   read_tsv 
#>  9 tsv     readr   read_tsv 
#> 10 xlsx    readxl  read_xlsx
#> 11 xls     readxl  read_xls 

## bcdata let's you know that this resource has
## multiple worksheets
try(
 bcdc_get_data('8620ce82-4943-43c4-9932-40730a0255d6')
)
#> Reading the data using the read_xlsx function from the readxl package.
#> 
#> This .xlsx resource contains the following sheets: 
#>  'Notes'
#>  'Single Detached '
#>  'Multi Unit Homes'
#>  'Purpose Built Rental'
#> Defaulting to the 'Notes' sheet. See ?bcdc_get_data for examples on how to specify a sheet.
#> # A tibble: 0 × 0

## we can control what is read in from an excel file
## using arguments from readxl::read_excel
try(
  bcdc_get_data('8620ce82-4943-43c4-9932-40730a0255d6', sheet = 'Regional Districts')
)
#> Reading the data using the read_xlsx function from the readxl package.
#> 
#> This .xlsx resource contains the following sheets: 
#>  'Notes'
#>  'Single Detached '
#>  'Multi Unit Homes'
#>  'Purpose Built Rental'
#> Error : Reading the data set failed with the following error message:
#> 
#>   Error: Sheet 'Regional Districts' not found
#> 
#> The file can be found here:
#>   '/tmp/RtmpdVoptK/bcdata_3a754ab9b760/file3a75541be347.xlsx'
#> if you would like to try to read it manually.
#> 
# }