Configure Fire Incident Dashboard

The Fire Incident Dashboard solution can be configured to meet specific needs in your organization. In this topic, you will learn how to use the Record Import Tools toolbox to automatically load data from Records Management Systems (RMS) and Computer-Aided Dispatch Systems (CAD) into the layers in your Fire Incident Dashboard; and modify or remove dashboard elements unsupported by your data schema.

Load data using Record Import Tools

The layers in the solution can be updated using the Record Import Tools toolbox provided with the solution. These tools can be run from ArcGIS Pro manually or automated on a schedule using Windows Task Scheduler. If you choose not to use these tools, your data can also be manually appended by using the Append Data to layer tool in ArcGIS Online. The Record Import Tools toolbox supports the use of database tables from your computer-aided dispatch (CAD) or records management system (RMS) as well as Microsoft Excel spreadsheets (.xlsx) or comma-separated values (.csv) files.

For each dataset you import, you will follow a two-step process. First, create a configuration file for each layer, and then run the import tool using the configuration file.

The Configure Record Import tools are used to create a configuration file (.ini) that stores user-defined parameters about the dataset and target layer it is loading to. There are two versions of the Configure Record Import tool—to determine which one to use, you must first identify whether the data you want to load is using address information or x,y coordinates (longitude and latitude).

  • If your dataset already has x,y data, create a configuration file based on x,y coordinates using the Configure Record Import (XY) tool.
  • If your dataset only has addresses, create a configuration file based on addresses using the Configure Record Import (Addresses) tool.

Once you have created a configuration file based on x,y coordinates or addresses, use the Record Import Tools to update the layer or feature class.

Note:

The Record Import tool can be run manually using the script tool in the Record Import Tools toolbox, or it can be run automatically using Windows Task Scheduler.

Download the Record Import Tools toolbox

To begin this workflow, follow the instructions to download and unzip the Record Import Tools toolbox.

  1. In a browser, go to the toolbox item that was deployed to your content section of ArcGIS Online, and click download to download the zipped folder containing the toolbox.
    Note:

    If you deployed the solution on ArcGIS Enterprise 10.6.1 or later, you need to download the toolbox separately.

  2. Extract the Record Import Tools toolbox and add it to a new ArcGIS Pro project.

Create a configuration file based on x,y coordinates

The Configure Record Import (XY) tool creates a configuration file that is used when executing the Import Records tool. The configuration file stores unique information that is used by the Import Records tool to load from a table or spreadsheet into a feature class or a web layer.

To create a configuration file, complete the following steps:

  1. In ArcGIS Pro, browse to the RecordImport.tbx file, open the Configure Record Import (XY) tool, and fill in the following parameters:

    ParameterInput
    Configuration file name

    Type the name for your configuration file.

    Source Table

    Browse to the incident file containing the data to be mapped.

    Target Features

    Browse to your incident feature class or web layer.

    Portal URL

    Add your ArcGIS Online organization or ArcGIS Enterprise portal URL.

    Username

    Type your user name.

    Password

    Type your password for the user name you provided.

    Log File Directory

    Browse to a directory to store the import records log files generated by the Import Records tool.

    Summary Field

    Provide a field from the source table containing values by which the records can be summarized.

    Record Identifier Field

    Choose a field containing a unique identifier value.

    Record Date Field

    Choose a date field containing the date the record occurred or was reported.

    Delete Duplicates

    Check this parameter to ensure duplicate records are deleted.

    X Coordinate Field (Longitude)

    Choose the field from the source table that represents the x-coordinate (decimal degrees or easting).

    Y Coordinate Field (Latitude)

    Choose the field from the source table that represents the y-coordinate (decimal degrees or northing).

    Source Table Coordinate System

    Choose the coordinate system used in the source table that represents the x,y coordinates.

    Ignore Blank Coordinate System

    Check this box to ignore the coordinate pair value 0,0 from being imported into the target features.

    Schema Mapping Type

    Choose Source Table Schema Matches Table if your schema matches the target features or Use Field Mapping if they are not the same.

    Field Mapping

    Choose the fields from the source table that contain information to preserve in the target features.

    Target Field

    Choose fields from the target features that correspond to the field in the source table.

    Note:

    Source and target information for the fields specified in the Record identifier field, Record Date Field, X Coordinate Field, and Y Coordinate Field parameters are required.

    Tip:

    If the Source Table parameter contains dates formatted as strings, you need to update the Timestamp Format parameter to define the original time and date format. For more information, see Python strftime.

  2. Run the tool to build a configuration file containing these parameter values.
  3. Once you have created a configuration file based on x,y coordinates, use the Configure Record Import to update the web layer or feature class.

Create a configuration file based on addresses

The Configure Record Import (Addresses) tool creates a configuration file that is used when executing the Import Records tool. The configuration file stores unique information that is used to import records from a table or spreadsheet into a feature class or a web layer.

To create a configuration file, complete the following steps:

  1. In ArcGIS Pro, browse to the RecordImport.tbx file, open the Configure Record Import (XY) tool, and fill in the following parameters:

    ParameterInput
    Configuration file name

    Type the name for your configuration file.

    Source Table

    Browse to the incident file containing the data to be mapped.

    Target Features

    Browse to your incident feature class or web layer.

    Portal URL

    Add your ArcGIS Online organization or ArcGIS Enterprise portal URL.

    Username

    Type your user name.

    Password

    Type your password for the user name you provided.

    Log File Directory

    Browse to a directory to store the import records log files generated by the Import Records tool.

    Summary Field

    Provide a field from the source table containing values by which the records can be summarized.

    Record Identifier Field

    Choose a field containing a unique identifier value.

    Record Date Field

    Choose a date field containing the date the record occurred or was reported.

    Delete Duplicates

    Check this parameter to ensure duplicate records are deleted.

    Street Address Field

    Choose the field from the source table with the street address, for example, 123 Main Street, or full address, for example, 123 Main Street, Naperville, IL.

    City Field

    Choose the field from the source table with the name of the city, town, or village.

    State Field

    Choose the field from the source table with the state or province name.

    ZIP Field

    Choose the field from the source table with the ZIP Code or postal code.

    Address Locator

    Choose the ArcGIS World Geocoding Service, your locator service, or browse to an Address locator file.

    Schema Mapping Type

    Choose Source Table Schema Matches Table if your schema matches the target features or Use Field Mapping if they are not the same.

    Field Mapping

    Choose the fields from the source table that contain information to preserve in the target features.

    Target Field

    Choose fields from the target features that correspond to the field in the source table.

    Note:

    If the configuration file from this tool references the ArcGIS World Geocoding Routing Service in the Address Locator parameter, this configuration will consume credits when it is run in the Record Import Tools . The credit consumption rate is the same as the Geocode Addresses tool. For more information, see Geocode Addresses in the System tool chart.

    Tip:

    If the Source Table parameter contains dates formatted as strings, you need to update the Timestamp Format parameter to define the original time and date format. For more information, see Python strftime.

  2. Run the tool to build a configuration file containing these parameter values.
  3. Once you have created a configuration file based on addresses, use the Record Import Tools to update the web layer or feature class.

Run the Import Records tool

The Record Import Tools use the configuration file created from the Configure Record Import (XY) or Configure Record Import (Addresses) tools. Run the Import Records tool in ArcGIS Pro the first time you use this tool to verify it runs successfully.

To import records, complete the following steps:

  1. In ArcGIS Pro, browse to the location of the RecordImport.tbx toolset and open the Record Import Tools tool.
  2. In the Configuration file parameter, browse to the location of the configuration file created by the Configure Record Import (XY) or Configure Record Import (Addresses), and click OK.
  3. Click Run to run the tool.
  4. Review any warnings or errors in the Geoprocessing Results window and make any necessary changes. Review the Import Records log file created in the designated reports directory defined in the Configure Record Import (XY) or Configure Record Import (Addresses) tool. These reports will help you resolve any errors found in your data and configuration file.
Note:

To update your incident layer on a regular interval basis, you can use the Import Records tool (import_records.py) with Task Scheduler to automate import updates.

Automate Record Import Tools

You can use the Record Import Tools (import_records.py) with Microsoft Windows Task Scheduler to update incident layers on a regular interval. Scheduled updates ensure users have access to current information to support more dynamic decision making.

To update incident data on a schedule, complete the following steps:

  1. Open Task Scheduler.
    Note:

    Use the Windows Start menu search box to find Task Scheduler.

  2. Click Action, click Create Task, and name your task.
  3. Click the Actions tab and click New.
  4. Set Action to Start a Program.
  5. Browse to the location of your Python installation (for example, <default directory>\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\python.exe).
  6. In the Add arguments text box, type the name of the script, import_records.py, followed by the path of the configuration file in double quotation marks.
    Note:

    Ensure that you have double quotation marks around the configuration file path but not around the script name, for example, import_records.py "C:\recordimports\scripts\records.ini".

  7. In the Start in text box, type the path of the folder hosting your script and click OK.
  8. On the Trigger tab, click New and set a schedule for your task, and click OK.

Log file reports

The Record Import Tools create a series of reports designed to help you understand any errors that occurred while processing your data. Each time the tools are run, a new set of reports are created and identified by the date and time.

Note:

The content of each import records report will vary based on your data and the configuration parameters. You can find the reports in the log file directory you specified when running the Configure Record Import (XY) or Configure Record Import (Addresses) tool. The path to these reports can also be found in the Geoprocessing Results window after running the Import Records script tool.

Log file contents

The log file header includes basic envelope information about the processes that ran, including the user name that ran the tool, what datasets were used, and the locator that was used (if the data was geolocated).

The body of the log file includes reports and notes printed by the tool as the records were processed. As each stage of the tool begins, a time stamp and brief message are printed. The tool also reports the numbers of records that were successfully and unsuccessfully processed after completing many of the major processes. When records cause problems (for example, a record that was not successfully geocoded), they are removed from processing and are written out to another report. A note is printed in the log file stating the number of records that were not successfully processed, and in which report they can be found.

Note:

The tool copies the records from the source table into a temporary geodatabase table. Problem records are removed from this table—not the source table.

The configuration includes an option to print a summary of the processed data. If a summary field is provided, this summary also appears in the configuration file. For example, summarizing the data by report date or an administrative boundary can indicate days or areas that may have missing reports.

Unmatched report

The Unmatched report is only created when the data was located using the ADDRESSES location type. This report contains all records that did not meet two criteria:

  • Geocoded with a Status value of M (matched) or T(tied)
  • Geocoded with an Addr_type value of AddrPoint, StreetAddr, BldgName, Place, POI, Intersection,

    PointAddress, StreetAddress, StreetInt, or StreetAddressExt

Records may not meet these criteria for many reasons. To help troubleshoot these issues, records in this report contain all fields from the attempted geocoding (Loc_name, Shape, Status, Score, Match_type, Match_addr, Side, User_fld, Addr_type, ARC_Street, ARC_City, and ARC_ZIP) as well as all fields from the original record. Use these values to determine why records were not successfully processed and resolve the issues.

For more information, see Geocode a table of addresses.

First, identify why the record was written to this report. Look at the value in the Status column.

  • Records with M failed the second criteria: they were geocoded, but not to a scale conducive to incident mapping, likely to the city or ZIP Code level.
  • Records with T (tied) had multiple options that were viewed by the locator as equally probable. There is likely some address information missing or slightly incorrect that is creating ambiguity for the locator.
  • Records with U (unmatched) were unable to be geocoded to an acceptable probability level. This probability threshold is set in the locator properties and is often near 80 percent.

Next, resolve the issue. You can make the necessary changes to the original report, get a corrected report from the data source, or make the necessary changes directly in the Unmatched report and reprocess the records using the same workflow and this report as the input spreadsheet.

  • Confirm that the address is complete.

Whether the full address was contained in a single field (for example, FULLADDR) or spread across several fields (for example, FULLADDR, CITY, STATE, or ZIP), verify that the level of information provided matched your locator. Your local locator may be able to handle a house number and street name, but that isn't enough information to get a good result from the World Geocoding Service. Provide as much information as possible to get the best results from the locator.

  • Confirm that the address is correct.

Many locators require at least an 80 percent match. Verify that the provided values are spelled correctly, house number values are reasonable for the range of the road, and the address falls within the provided city, state, and ZIP Code.

  • Confirm that the address falls within the range of your locator.

Locators do not always cover all addresses. Private roads and new developments may not exist in the locator yet, or the record may fall outside the extents of your locator. In these cases, try another locator, or get the coordinates for the location of the record and process the record or records using the Configure Record Import (XY) and Configure Record Import (Addresses) tools.

Once the corrections are made, reprocess the report. You can use the Delete Duplicates option to ensure that records are not added to the target features.

To stop excluding results from processing, or to modify the Status or Addr_type values that are acceptable for your data, you can edit the Python script import_records.py that are found in the same directory as the ImportRecords.tbx file.

Near the top of the script is a section titled Accepted levels of geolocation. The section contains two list variables: addrOK and match_value. Both lists must contain at least one value enclosed in quotation marks. Add, remove, or modify the values as long as the following are true:

  • Each list is enclosed in a single pair of square brackets.
  • Each item is enclosed in a single pair of quotation marks.
  • Multiple items are separated with a comma.

For example, allow the tool to only process results with a status of "M" by changing the value as follows:

  • match_value = ["M","T"] to match_value = ["M"]

Change the address locator

The Record Import Tools are configured to use the ArcGIS World Geocoding Service available with all ArcGIS Online organizational accounts but can also be configured to use your own locator or geocoding service.

To use your own locator, complete the following steps:

  1. Browse to the location of the Record Import Tools and create a backup copy of the import_records.py script.
  2. Open the import_records.py script in a Python editor or a text editor.
  3. Search for the first occurrence of all_locator_fields.
  4. This value should be near the top of the script. This value and the four lines of the script must be modified to reflect the properties of the locator or geocode service you will be using with the Record Import Tools .

ParameterInput

all_locator_fields

These are the input address fields that the locator may use to identify the geography of an address.

For a locator file (*.loc), these values can typically be found in the locator properties. The Input fields section contains a list of expected input address fields. If you are using a geocoding service, the expected input address fields are found on the service REST page under the Address Fields section. List the values in the same order they appear in the locator properties.

loc_address_field

The input address field that the locator expects to contain, at minimum, the address number and street name of the address. For single-line geocoding, this field may also contain the complete address. This input address field is typically named Street or Address.

This value must also exist in the all_locator_fields list.

loc_city_field

The input address field that the locator expects to contain the city name. This input address field is typically named City. For single-line geocoding, use empty quotes for this value; otherwise, this value must also be included in the all_locator_fields list.

loc_zip_field

The input address field that the locator expects to contain the ZIP Code or postal code. This input address field is typically named ZIP or Postal. For single line-geocoding, use empty quotes for this value; otherwise, this value must also be included in the all_locator_fields list.

loc_state_field

The input address field that the locator expects to contain the state or province name or code. This input address field is typically named State, Province, or Region. For single-line geocoding, use empty quotes for this value; otherwise, this value must also be included in the all_locator_fields list.

Remove or modify dashboard elements

When loading data from your RMS or CAD system into the layers of the Fire Incident Dashboard you may find that your source data does not contain specific fields used in the filters or charts. The Fire Incident Dashboard can be re-configured to point to different fields, or deleted entirely, as the schema of your data necessitates. See configuring ArcGIS Dashboard elements for more information.