Batch Import Data (Intelligence)

Summary

Imports KML, KMZ, shapefiles, Excel worksheets, tabular text files, GeoJSON, and GPX files to feature classes stored in a single geodatabase.

Usage

• If the Input Data value is a folder, it will be recursively searched for KML (.kml) and KMZ (.kmz) files, shapefiles (.shp), Excel workbooks (.xlsx and .xls), tabular text files (.csv, .txt, and .tab), GeoJSON files (.geojson), and GPX files (.gpx). The Filter parameter can be used to limit which files are imported from folders.

• Rasters, or ground overlays contained in a .kml or .kmz file, are converted into a mosaic dataset in the Target Geodatabase parameter. The source rasters referenced by the mosaic dataset are stored in their native format in a subfolder named after the mosaic dataset. This is located at the same location as the target geodatabase. These files are named after the Target Geodatabase value and appended with _Rasters. For example, if the Target Geodatabase value is C:\Data\OperationalData.gdb, a folder named OperationalData_Rasters will be created under C:\Data.

• If you are importing files into an enterprise geodatabase, ensure that the database connection file (.sde) is specified as a UNC path in the Target Geodatabase parameter and that the path is accessible by the enterprise geodatabase.

• KML and KMZ input up to KMZ version 2.2 of the Open Geospatial Consortium KML standard is generally supported. Point locations that use the address tab (by way of geocoding) are not supported. A valid latitude and longitude location is required in the source KML.

• The standard delimiter for tabular text files with .csv and .txt extensions is a comma; for files with a .tab extension, it's a tab. To use an input table with a nonstandard delimiter, the delimiter must be specified using a schema.ini file. See Schema.ini File (Text File Driver) in the Microsoft documentation for more information.

• For input Excel worksheets and tabular text files, only point geometry is supported.

• The GeoJSON specification supports multiple geometry types. A separate feature class will be created for each geometry type in the input .geojson file using the naming convention SourceGeoJSONFile_GeometryType. For example, if the .geojson file is named World Airports.geojson and contains point, multipoint, polyline, and polygon geometry types, the feature classes will be named World Airports_Points, World Airports_Multipoint, World Airports_Polylines, and World Airports_Polygons.

• Coordinate systems have the following considerations when running this tool:

  • For shapefiles, and .kml, .kmz, .geojson, and .gpx files, the coordinate system of the output dataset will be the same as the input dataset. The KML and GPX specifications mandate the coordinate system of the input dataset to be GCS WGS1984.
  • For Excel workbooks and tabular text files, the coordinate system of the output will be GCS WGS1984.

• If any of the input coordinates for a feature are invalid (null or invalid coordinate notation), the corresponding feature will have a null geometry. The Check Geometry tool can be used to identify null geometries in the output.

• Supported coordinates for tabular text files (.csv, .txt, and .tab) and Excel worksheets are latitude and longitude in decimal degrees, Military Grid Reference System (MGRS), and United States National Grid (USNG). The source is searched for coordinate columns in the following order: latitude and longitude in separate columns, latitude and longitude in a single column, MGRS, and finally USNG.

  Note:

  For latitude and longitude values in a single column, the latitude value must be first, then longitude separated by a space, a comma, or a slash.

  The following are valid column names:

  • Latitude—latitude, lat, y_coordinate, y_coord, and y
  • Longitude—longitude, long, lon, x_coordinate, x_coord, and x
  • Latitude/longitude—latitude_longitude, lat_long, lat_lon, xy_coordinate, xy_coordinates, xy_coord, xy_coords, and xy
  • MGRS coordinate—mgrs, mgrs_coordinate, mgrs_coordinates, mgrs_coord, and mgrs_coords
  • USNG coordinate—usng, usng_coordinate, usng_coordinates, usng_coord, and usng_coords

• All worksheets in an Excel workbook will be imported.

• Symbology is applied to the output layers that are added to the map based on the symbology in the source data files as follows:

  Note:

  An active map must exist in the AllSource project at the time of tool execution to maintain the symbology. If the tool is running outside an AllSource project—in other words, from a stand-alone Python script—the symbology will not be maintained.
  • For output layers created from .kml or .kmz files, the symbology is extracted from the source .kml or .kmz file and applied to the output layers. This symbology can be overridden by providing a layer file (.lyrx or .lyr) in the same folder as the source .kml or .kmz file. The layer file must have the same base name as the source data file, appended with an underscore (_), and the geometry type (points, polylines, or polygons) of the layer to be overridden. For example, if the .kml file is named World Airports.kml and contains point, line, and polygon features and you want to override the symbology for all geometry types, the layer files would be named World Airports_Points.lyrx, World Airports_Polylines.lyrx, and World Airports_Polygons.lyrx. The geometry type values are case sensitive. If a layer file does not exist for a given geometry type, the symbology from the source .kml or .kmz file is applied to the output layer.
  • For output layers created from sources other than .kml or .kmz files, symbology is applied to the output layer if a layer file (.lyrx or .lyr) exists in the same folder as the source data file. The layer file must have the same base name as the source data file, except Excel worksheets and .geojson files. For Excel worksheets, the layer file must be named the same as the base name of the source data file, appended by an underscore (_) and the worksheet name. For example, if the Excel file is named World.xlsx and contains a worksheet named Cities (2018), the layer file must be named World_Cities (2018).lyrx or World_Cities (2018).lyr. For .geojson files, the layer file must be named the same as the base name of the source data file, appended by an underscore (_) and the output geometry type—points, multipoint, polylines, and polygons. If a layer file does not exist, a default symbol will be applied to the output layer.

Parameters

Dialog

Label	Explanation	Data Type
Input Data	The folders containing the data files or the data files to convert to geodatabase feature classes.	Folder; File
Target Geodatabase	The target geodatabase where output feature classes will be stored.	Workspace
Filter (Optional)	Applies a filter to limit which files are imported from folders. The following wildcard characters for the filter work on the full path to the input data: *—Match any character ?—Match a single character [range]—Match a single character in the range [!range]—Match any character not in the range Multiple patterns can be added to the filter by separating each pattern with the vertical bar or pipe delimiter (|). Pattern comparisons are not case sensitive, so using the *airport.shp, *AIRPORT.SHP, or *Airport.shp pattern, for example, will import the same shapefile. The following are filter examples: To import all shapefiles, use *.shp. To import all shapefiles and all .kml files, use *.shp|*.kml. To import all files that have airport in the file path or file name, use *airport*. To import all .geojson files that have airport in the file path or file name, use *airport*.geojson. To import all .kmz files that have airport appended with any two characters in the name, use *airport??.kmz. To import all files that have 1990 through 1997 in the file path or file name, use *199[0-7]*. To import all shapefiles that have the exact folder name airfacilities in their file path, use *\airfacilities\*.shp.	String
Include Sub Folders (Optional)	Specifies whether subfolders will be recursively explored. Checked—All subfolders will be explored. This is the default. Unchecked—Only the top-level folder will be explored.	Boolean
Include Ground Overlay (Optional)	Specifies whether KML or KMZ ground overlays (raster, air photos, and so on) will be included in the output. Use caution if the KMZ points to a service that serves raster imagery. The tool will attempt to translate the raster imagery at all available scales. This process may be lengthy and possibly overwhelm the service. Checked—Ground overlays will be included in the output. This is the default. Unchecked—Ground overlays will not be included in the output.	Boolean

Derived Output

Label	Explanation	Data Type
Updated Geodatabase	The updated target geodatabase.	Workspace

Python

arcpy.intelligence.BatchImportData(in_data, target_gdb, {filter}, {include_sub_folders}, {include_groundoverlay})

Name	Explanation	Data Type
in_data [in_data,...]	The folders containing the data files or the data files to convert to geodatabase feature classes.	Folder; File
target_gdb	The target geodatabase where output feature classes will be stored.	Workspace
filter (Optional)	Applies a filter to limit which files are imported from folders. The following wildcard characters for the filter work on the full path to the input data: *—Match any character ?—Match a single character [range]—Match a single character in the range [!range]—Match any character not in the range Multiple patterns can be added to the filter by separating each pattern with the vertical bar or pipe delimiter (|). Pattern comparisons are not case sensitive, so using the *airport.shp, *AIRPORT.SHP, or *Airport.shp pattern, for example, will import the same shapefile. The following are filter examples: To import all shapefiles, use *.shp. To import all shapefiles and all .kml files, use *.shp|*.kml. To import all files that have airport in the file path or file name, use *airport*. To import all .geojson files that have airport in the file path or file name, use *airport*.geojson. To import all .kmz files that have airport appended with any two characters in the name, use *airport??.kmz. To import all files that have 1990 through 1997 in the file path or file name, use *199[0-7]*. To import all shapefiles that have the exact folder name airfacilities in their file path, use *\airfacilities\*.shp.	String
include_sub_folders (Optional)	Specifies whether subfolders will be recursively explored. SUBFOLDERS—All subfolders will be explored This is the default. NO_SUBFOLDERS—Only the top-level folder will be explored.	Boolean
include_groundoverlay (Optional)	Specifies whether KML or KMZ ground overlays (raster, air photos, and so on) are included in the output. Use caution if the KMZ points to a service that serves raster imagery. The tool will attempt to translate the raster imagery at all available scales. This process may be lengthy and possibly overwhelm the service. GROUNDOVERLAY—Ground overlays will be included in the output. This is the default. NO_GROUNDOVERLAY—Ground overlays will not be included in the output.	Boolean

Derived Output

Name	Explanation	Data Type
out_geodatabase	The updated target geodatabase.	Workspace

Code sample

BatchImportData example 1 (Python window)

The following Python window script demonstrates how to use the BatchImportData function in immediate mode:

import arcpy 
arcpy.intelligence.BatchImportData("c:/data/sourcedata", "c:/data/output.gdb", "", "SUBFOLDERS", "NO_GROUNDOVERLAY")

BatchImportData example 2 (stand-alone script)

The following Python script demonstrates how to use the BatchImportData function in a stand-alone script:

# Name: BatchImportData_Example2.py
# Description: Imports all shapefiles with 'airport' in the file name or file
#              path within a directory to feature classes in a single geodatabase.
# Requirements: Standard or Advanced license

# Import system modules
import arcpy

# Set local variables
in_data = ["c:/data/sourcedata"]
target_gdb = r"\\data\operationaldata.sde" 
filter = "*airports*.shp" 
include_sub_folders = "NO_SUBFOLDERS" 

# Run BatchImportData
arcpy.intelligence.BatchImportData(in_data, target_gdb, filter, include_sub_folders)

Environments

This tool does not use any geoprocessing environments.