1. Home
  2. Publishing Resources - Overview

Publishing Resources - An Overview

This is an overview of the publishing process. Various aspects of publishing and data preparation are covered in more detail in later sections. The major steps in this process are as follows:

Decide how you want to participate in the Commons

Full Geospatial Data Resource Site (GDRS) node

The GDRS, pioneered by the Minnesota Department of Natural Resources (DNR) and extended to other state organizations, is a system of networked sites that enables organizations to share and regularly update data and applications. Under this model, an organization publishes its data and applications (resources) to its own GDRS node, and those resources are distributed to all participating networked organizations through a set of tools. When resources are shared in the GDRS, they can then be exposed to the public via the Commons; publishers authorize that exposure through the Commons "Geobroker" application. The organization supporting a full GDRS node maintains all publishing rights to its folders on the node. This method is generally preferred for state agency publishers.

Web-based (FTP) publishing

In our second model, publishers are provided an account and folder on MnGeo's "Inbound FTP site" to post their resources. This FTP site becomes a de facto "GDRS Node", allowing for synchronization of resources from all the publishers that don't have their own GDRS node. After posting a resource to the Inbound FTP site, publishers are provided the same options in the Geobroker as publishers using a full GDRS: they can choose to keep the distribution to only state agencies, or to publish the resource to the Commons. This method is generally preferred for publishers who are not state agencies; publishers such as the Metropolitan Council and Itasca County are using it. Data or app resource preparation is the same for both publishing methods.

Hosting on another organization’s GDRS

An organization which does not wish to support a full GDRS node may partner with another organization which has a GDRS and agrees to act as a host. This may be appropriate if your organization has few records to publish and does not wish to (or is unable to) maintain a full connected GDRS node. The organization publishing resources through a hosted node will still prepare its resources, but will need to work with the host organization to get the resources published. This method is generally not preferred, but is available in limited situations.

Regardless of the method, in order to create a publisher presence, an organization will need to create an organizational profile. Organizations should contact MnGeo to discuss their particular situation to decide how best to participate in the Commons. If necessary, additional publishing methods will be explored as we extend the Commons to new types of publishing partners, and this page will provide details on those methods.

The Commons team is hoping to pilot an additional method for partnering with organizations that have only a small number of resources to contribute.

Decide whether you are publishing a data resource, an app resource, or both

Most organizations have data resources (i.e., shapefiles or file geodatabases) to share, but the option to publish application ("app") resources is also available. In some cases the publisher has the option of publishing as a data resource or as an app resource (see Choosing a Resource Type).

For a data resource, decide what data formats you will publish. That will determine the directory structure of your staging area (see Resource Formats).

Document and organize your resource

Under the GDRS scenario, there are several steps to document and organize your resource for publication. Below is an overview.

Create a staging area (folder) for your data resource or app resource

This is a folder outside of your GDRS node which you use to "stage" your resource for publication. Follow these guidelines:

  • For a data resource, the staging folder will include a subdirectory for each format in which the data is provided (such as file geodatabase or kml), a “metadata” subdirectory, and a file called “dataResource.xml”, which provides key information that will register the data resource with the GDRS and with the Commons.
  • An app resource does not require a metadata subdirectory, but does require the creation of an “appResource.xml” file that will register the app resource with the GDRS and with the Commons.
  • Tools are available to help publishers prepare the resource and correctly format the data and metadata within a staging folder:
  • The Minnesota Metadata Editor (MME) is a standalone tool to create compliant metadata needed by the system. ArcCatalog metadata tools can also be used. Both methods are described under "Creating MGMG-Compliant Metadata".
  • The MGC Resource Editor aids in the creation of data resources for inclusion on a GDRS. This ArcGIS Add-in formats the staging folder correctly and automatically creates the dataResource.xml; it is being expanded to also format app resources. This tool, and a companion validator tool, can be used to make sure the data or app is ready to be published.
  • The Data Resource Validator Tool can be used to validate the metadata, directory and xml structure of a resource that was created manually.

Copy resource to GDRS or Inbound FTP

Copy the internally validated data resource or app resource to your public site. This could be:

  • Your organization's GDRS node
  • Your organization's hosted site (folder) on another organization's GDRS node (in this scenario, you likely won't have copying permission, and the host organization will need to copy it for you)
  • Your organization's folder on the Inbound FTP site

An hourly "cron job" (program) further validates the resource and, if there are no major errors, places it in the Geobroker.

Geobroker processing

The Geobroker is an application that helps you authorize your GDRS data or applications for publishing on the Commons. It comprises a database and a set of programs which manage the publishing and updating of the resources. The Geobroker will check resources hourly and will flag errors that need to be corrected. Once errors have been corrected, the publisher specifies whether the resource is to be published as “Agency” or “Commons” (see Using the Geobroker to Manage your Resources).

  • If published as “Agency”, the data or app resource will be shared with (copied to) all other GDRS nodes.
  • If published as “Commons”, then the data or app resource will additionally be zipped and placed on an FTP site for public distribution, and the resource will be represented on the Commons with a data resource record, metadata, etc.
  • Publishing occurs nightly, except on weekends.
  • Some additional open data formats are automatically created as part of the publishing process. Proprietary Esri file geodatabases are automatically converted to Open Geopackage and Shapefile formats as additional distribution formats.
  • Once a resource is published, any updates provided by the publisher on its own node will be pushed out to the Commons nightly. Updates do not occur over the weekend.