BC DATA CATALOGUE PUBLICATION WORKFLOW

When data is published to the DataBC Program through any of its services, it must also have a metadata record in the BC Data Catalogue. DataBC has well documented Data Standards and Guidelines that are followed throughout the process for metadata creation. The workflows described below will guide you through what your reponsibilities are as a publisher to the BC Data Catalogue and how to create a metadata record in the BC Data Catalogue.

This page is written in the context that you have reviewed all information in the ‘Documentation Review’ section.

    AUDIENCE    
Data Publishers Data Custodians Data Managers Data Stewards Metadata Editors

Table of Contents

DOCUMENTATION REVIEW

The first step in the process is to review the documentation:

RETURN TO TOP

WORKFLOW DIAGRAMS

BCDC Metadata Readiness Workflow

Wondering if you are ready to create metadata? This workflow guides you through the pre-work so you will be ready to create a metadata record. BCDC Metadata Readiness Workflow

BCDC Metadata Creation Workflow

Once you are ready, use the steps in this workflow to guide you through the process of creating a metadata record. BCDC Metadata Creation Workflow

RETURN TO TOP

HOW TO BECOME A PROVIDER TO THE CATALOGUE

To become a Catalogue editor/publisher for a Sub-Organization (a Branch or Division), approval from the Data Custodian (Director or Executive Director) is required. Here are the steps to complete this:

  1. Contact the Data Custodian to request approval for publishing/editing privileges in the Sub-Org. Be sure to note the Orgnization and Sub-Organization (as it appears in the Catalogue) in your email. If more than one person requires editing access, include them in the request as well.
  2. Once approval is received from the Custodian, forward the email including ther response to DataBC.DA with the subject: “BCDC Editor Request”.
  3. A Catalogue administrator will then add you as an editor to the Sub-Org and contact you with all relavant information to get you started as an Editor in the BC Data Catalogue.

Organizations in the Broader Public Sector (BPS) may publish metadata records and data to the Catalogue.

  • For datasets licensed under Open Data, these organizations will require their own Open Government Licence. See Licenses section below for more information on this.
  • Contact DataBC.DA to discuss publishing your data to the Catalogue.

RETURN TO TOP

LICENCES

There are three licencing options for data published in the BC Data Catalogue:

  1. Access Only

    Access only is a statement of use that is restrictive as defined in the Crown Copyright Page.

    • “reproduction is not permitted without written permission.”
    • A user is then required to request permission through the IPP office form and does have a fee associated with it.

  2. Open Government Licence - British Columbia

    It is recommended to publish data under an Open Government Licence, where possible, as it is much more permissive.

    To publish a dataset under OGL-BC, follow the full steps here but in brief you will need to obtain the required authorizations as noted in the Open Data Assessment and Checklist. These include:

    Once these approvals are in place, Data can be published under OGL-BC if:

    • it is in an open format, e.g., CSV, Shape File (SHP), JSON, or additional formats noted by the OCIO’s Open Data Physical Dataset Format Standard.
    • The Catalogue record links to the appropriate Open Data Licence.

  3. Open Government - (Other)

    The Catalogue contains datasets licenced under many other Open Government Licences, from the Broader Public Sector to Federal licences.

    Federal Licences are listed as there are times when BC Government employees require data to be published for their own consumption via one or more channels.

    New licences can be added to the drop down.

RETURN TO TOP

CREATING A METADATA RECORD

Note: To use the CKAN API for creating metadata, more information is available here.

  1. Log into the BC Data Catalogue by clicking Log In on the upper right of the Catalogue page. This will take you to your dashboard.
  2. Click the Datasets button on the toolbar.
  3. Click the [Add Dataset] button.
  4. Choose the dataset type that describes the type of content you are publishing.
    • Follow the workflow link below for details on the specific fields for each dataset type.
Dataset Type  
Application Workflow
Dataset Workflow
Geographic Dataset Workflow
Web Service - API Workflow

  1. Click the Create Dataset button.

  2. Complete all mandatory fields (minimum requirement). We suggest adding content to non-mandatory fields as this will help people to better understand and use the data as intended.

METADATA REQUIREMENTS FOR DATA AND MAPS IN BC MAP HUB (ARCGIS ONLINE)

The use of the BC Map Hub has a requirement for the creation of metadata alongside data and maps. The following documentation will assist in that process:

RETURN TO TOP

MANAGING METADATA RESOURCES

Resources are the data in the metadata record and are added to the record to provide users with access to your data.

For details on how to add resources, see the workflow for your dataset type in the Creating a Metadata Record section above.

Notes:

  • To use the CKAN API for managing resources, more information is available here
  • Only files less than 150MB in size can be uploaded to the Catalogue.
  • If the file is less than 150MB limit we recommend uploading the file to the Catalogue vs. storing the data elsewhere.

REORDERING RESOURCES

The Reorder resources button allows you to control the order of resources in the metadata record.

EDITING EXISTING RESOURCES

There are two ways to edit a resource in the metadata record:

  • Using the ‘wrench’ icon > click the Resources tab
  • Using the Explore button on the main page:
    • Click the Explore drop down button
    • Click the Edit option to edit the resource
  1. Make edits that follow the workflow for your dataset type under Creating a New Metadata Record.
  2. Click the Update Resource button to save your changes.

REPLACING AN EXISTING RESOURCE

While in edit mode on a resource (see Editing Exisitng Resources section above):

  1. Click the Remove button at the end of the File URL.
  2. Re-upload or provide a new link to your data.
  3. Make any other edits needed to reflect the updated/new resource.
  4. Click Update Resource to save your changes.

DELETING AN EXISTING RESOURCE

While in edit mode on a resource (see Editing Exisitng Resources section above):

  1. Click the Delete button at the bottom of the page.

RETURN TO TOP

METADATA MANAGEMENT AND MAINTENANCE

Please review BC Data Catalogue - Roles and Responsibilities, which outlines the obligations for metadata management and maintenance. It is good practice for the Data Custodain (or designate) to complete regular metadata reviews to ensure currency and completeness for records published under their Branch/Division.

RETURN TO TOP

PUBLISHING A METADATA RECORD

All new metadata records are created in the DRAFT record state.

Once the metadata record is complete, the editor sets the Record State to Pending Publish. Here are the steps:

  1. Log into the Catalogue.
  2. Navigate to the metadata record to be edited and click the “wrench” tool icon (top right corner) to start editing the record.
  3. Change the dataset Record State(first metadata element):
    • from DRAFT
    • to PENDING PUBLISH
  4. Click Update Dataset.

Once a metadata record has been set to PENDING PUBLISH, the DataBC is notified and will then review the record for:

  • Completeness
  • Data quality
  • Licence type

If your metadata record is validated by DataBC, they will change the state to PUBLSHED. Otherwise, a DataBC team member will contact you to discuss your record.

Note on Publication timing: if there is a specific time and date that the release of these records is required, contact DataBC to work on timing that is withing regular government business hours, M-F 8:30 to 5pm.

Once your metadata record is in the Published state, it is then visible and searchable in the BC Data Catalogue for all users.

Data Custodians should regularly reivew all metadata records within ther Sub-Org for currency and completeness. This may be delegated to any of the other Roles or the Editor associated with the metadata record.

RETURN TO TOP

ARCHIVING A METADATA RECORD

When a dataset is no longer relevant, it should be archived. Here are the steps to begin this process:

  1. Log into the Catalogue.
  2. Navigate to the metadata record to be edited and click the “wrench” tool icon (top right corner) to start editing the record.
  3. Ensure the Organization and Sub-Organization are correct. If not, update these fields or contact the DataBC if you do not have editing access for the Sub-Org.
  4. Under Data Currency/Update, update the Resource Status:
    1. If the data is obsolete, a link must be added for a Replacement Record.
      • When this is another record in the catalogue:
        • Use the replacement record’s permalink
        • Add the text “DEPRECATED - “ to the beginning of the metadata Title
      • If there is no replacement, then either the main catalogue page can be used or a webpage about the data or program.
    2. If the data is going into historicalArchive, two additional fields must be completed:
      1. Retention Expiry Date
      2. Source Data Path
  5. Change the dataset Record State:
    • from PUBLISHED
    • to PENDING ARCHIVE.

Once the record has been set to PENDING ARCHIVE, DataBC will then reivew the record to ensure it is ready for archiving, this includes:

  • for BCGW specific datasets:
    • removing the resources for the WMS Get Capabilities and the Network Link KML
    • add a new resource called “BCGW Deprecated Object” in peperation for migration of the new catalogue and the move of the object_name to the resource level.
      • Copy over all the fields from the BCGW Custom Download
      • when done then delete the BCGW Custom Download resource
    • remove the values in the Preview Details section
  • confirming the correct Org and Sub-Org
  • confirming that records with a replacement dataset contain “DEPRECATED” in the title

Note: If the record is related to a BCGW dataset, it will be added to the next Notice of Intent (NOI) for archival.

Once DataBC has confirmed the record is ready for archiving, they will update the Record State to ARCHIVED.

Archived datasets:

  • are only visible to logged in Catalogue members of the Sub-Org which the dataset resides in
  • do not contain resources

RETURN TO TOP

METADATA GROUPS

Metadata Groups provide a listing of like datasets for a specific purpose and can contain records from across multiple organizations.

  • These are not intended to replace a full listing of records for an sub-organization.
  • Groups by default are public but a request them to be visible to IDIR only can be made.

To create a group, the following information is required and can be sent to DataBC.

Requirements: At any point this information needs to be updated or group members modidified, this can be done by emamiling DataBC.

  • Group Name:
  • Group Description:
  • URL: for more information
  • Image: either a file or url
  • Members: list of IDIRs who will maintain the Group
  • Group visibility: Public or IDIR

Managing Groups A member of a catalogue group can add any record published as an item in the group.

  • However when adding records from other organizations, we do recommend reaching out to inform them of this. This builds a relationship between the data consimer and data providert they understand the value their data is providing your pusposes.

How to add Records to a Group

  1. Log in to the catalogue
  2. Navigate to a metadata record to add to the group
  3. Click the Group button on the grey toolbar
  4. Select the group to add to in the drop down list
  5. Click the Add to group button

Repeat the above steps for each record

RETURN TO TOP

ORGANIZATION AND SUB-ORGANIZATION UPDATES

Organizations change names frequently and thus need continual maintenance.

If a Organization/Ministry or Sub-organization/Branch/Division/Program has changed names, the following can be following information can be sent to DataBC:

  • Old name
  • New name
  • List of data providers (Editors)
    • When an update is requested for a sub-organization we ask that the members and contacts list be reviewed.
    • DataBC can provide you with a current list to review.

RETURN TO TOP

BULK UPDATES

When there are more than 20 records that require updating, a bulk update can be requested for DataBC to complete or can be completed by a user via the API.

  • Common bulk updates:
    • Branch/Division names
    • Contact information
    • State Change
    • URL modifications for More Info and Resources

RETURN TO TOP