Migrating metadata scripts from ArcMap to ArcGIS AllSource

Python scripts that are used to manage metadata in ArcGIS Desktop must be modified before you can run them in ArcGIS AllSource. The sections below cover the significant changes between the tools offered in ArcGIS Desktop in the Conversion toolbox, and the capabilities available in arcpy.metadata.

Note:

The capabilities provided in ArcGIS Desktop by the XML Schema Validation and Validate Metadata tools are not yet available in ArcGIS AllSource. These capabilities will be added in a future release.

Export metadata

The capabilities of the ArcGIS Desktop Export Metadata tool are now available from the Metadata exportMetadata method. Scripts written for ArcGIS Desktop that used the following translators should be updated to use the appropriate metadata export option, as indicated below.

ArcGIS Desktop translator	ArcGIS AllSource metadata_export_option
ARCGIS2FGDC.xml	FGDC_CSDGM
ARCGIS2ISO19139.xml	ISO19139
ARCGIS2ISO19139gml321.xml	ISO19139_GML32

The following sample demonstrates how to export an item's metadata to an FGDC CSDGM-format metadata .xml file using ArcGIS Desktop:

import arcpy
import os

dir = arcpy.GetInstallInfo("desktop")["InstallDir"]
translator = os.path.join(dir, "Metadata/Translator/ARCGIS2FGDC.xml")
source_item = "C:/data/data.gdb/roads"
output_file = "C:/data/roads_csdgm.xml"
arcpy.ExportMetadata_conversion(source_item,translator, output_file)

The following sample demonstrates how to export an item's metadata to an FGDC CSDGM-format metadata .xml file using ArcGIS AllSource:

import arcpy
from arcpy import metadata as md

src_item_md = md.Metadata("C:/data/data.gdb/roads")
output_file = "C:/data/roads_csdgm.xml"
src_item_md.exportMetadata(output_file, 'FGDC_CSDGM')

Import and copy metadata

The capabilities of the ArcGIS Desktop Import Metadata tool are now available from the Metadata importMetadata method. Scripts written for ArcGIS Desktop that used the following import type settings should be updated to use the appropriate import option, as indicated below:

ArcGIS Desktop Import_Type	ArcGIS AllSource import_option
FROM_ARCGIS	ARCGIS_METADATA
FROM_FGDC	FGDC_CSDGM
FROM_ISO_19139	ISO19139_UNKNOWN

The following sample demonstrates how to import the contents of an FGDC CSDGM-format metadata .xml file that describes some data to an item that represents the same data using ArcGIS Desktop:

import arcpy

csdgm_file = "C:/data/roads_csdgm.xml"
target_item = "C:/data/data.gdb/streams"
arcpy.ImportMetadata_conversion(csdgm_file, "FROM_FGDC", target_item)

The following sample demonstrates how to import the contents of an FGDC CSDGM-format metadata .xml file that describes some data to an item that represents the same data using ArcGIS AllSource:

import arcpy
from arcpy import metadata as md

src_csdgm_file = "C:/data/roads_csdgm.xml"
tgt_item_md = md.Metadata("C:/data/data.gdb/streams")
tgt_item_md.importMetadata(src_csdgm_file, "FGDC_CSDGM")
tgt_item_md.save()

Save changes to an item's metadata

The capabilities of the ArcGIS Desktop Metadata Importer tool are now available from the Metadata copy method. This method should be used to transfer content in the ArcGIS metadata format from one item to another, such as from a metadata template to a feature class.

The following sample demonstrates how to copy content from an ArcGIS metadata template to another item using ArcGIS Desktop:

import arcpy

src_template = "C:/data/arcgis_template.xml"
target_fc = "C:/data/data.gdb/streams"
arcpy.MetadataImporter_conversion(src_template, target_fc)

The following sample demonstrates how to copy content from an ArcGIS metadata template to another item using ArcGIS AllSource:

import arcpy
from arcpy import metadata as md

src_template_md = md.Metadata("C:/data/arcgis_template.xml")
tgt_item_md = md.Metadata("C:/data/data.gdb/streams")
tgt_item_md.copy(src_template_md)
tgt_item_md.save()

Synchronize metadata content

The capabilities of the ArcGIS Desktop Synchronize Metadata tool are now available from the Metadata synchronize method.

The following sample demonstrates how to update the properties of the item that are stored in its metadata using ArcGIS Desktop:

import arcpy

source_item = "C:/data/data.gdb/roads"
arcpy.conversion.MetadataImporter(source_item, "ALWAYS")

The following sample demonstrates how to copy content from an ArcGIS metadata template to another item using ArcGIS AllSource:

import arcpy
from arcpy import metadata as md

src_item_md = md.Metadata("C:/data/data.gdb/roads")
src_item_md.synchronize("ALWAYS")

Upgrade metadata

The capabilities of the ArcGIS Desktop Upgrade Metadata tool are now available from the Metadata upgrade method.

The following sample demonstrates how to update an archived item's metadata to the ArcGIS metadata format using ArcGIS Desktop:

import arcpy

source_item = r'C:\Data\Old.gdb\Buildings'
arcpy.conversion.UpgradeMetadata(source_item, "FGDC_TO_ARCGIS")

The following sample demonstrates how to update an archived item's metadata to the ArcGIS metadata format using ArcGIS AllSource:

import arcpy
from arcpy import metadata as md

archive_item_md = md.Metadata(r'C:\Data\Old.gdb\Buildings')
archive_item_md.upgrade('FGDC_CSDGM')

Save a copy of an item's metadata, or filter its content

The capabilities of the ArcGIS Desktop XSLT Transformation tool are now available from the Metadata saveAsXML and saveAsUsingCustomXSLT methods.

Some common operations for managing metadata were supported in ArcGIS Desktop through XSLT stylesheets that were provided with the software. These stylesheets were used with the XSLT Transformation tool to make the desired changes to an item's metadata, save copies of an item's metadata, or generate different types of output. Many of these operations continue to be available in ArcGIS AllSource as described below. The capabilities are now built into ArcGIS AllSource instead of being provided as XSLT stylesheet resources.

Save a copy of an item's metadata

The capabilities that were provided in ArcGIS Desktop in the XSLT stylesheets listed below are now available in ArcGIS AllSource from the Metadata saveAsXML method. Instead of specifying an XSLT stylesheet file, specify the corresponding save as XML option, as indicated below.

ArcGIS Desktop XSLT stylesheet	ArcGIS AllSource metadata_save_as_xml_option
exact copy of.xslt	EXACT_COPY
remove local storage info strict.xslt	REMOVE_ALL_SENSITIVE_INFO
remove local storage info.xslt	REMOVE_MACHINE_NAMES
generate metadata template.xslt	TEMPLATE

The following sample demonstrates how to save a copy of an item's metadata to an ArcGIS metadata format .xml file and also a version of the same item's metadata that is filtered to remove any machine names that may be present using ArcGIS Desktop:

import arcpy
import os

source_item = "C:/data/data.gdb/roads"
dir = arcpy.GetInstallInfo("desktop")["InstallDir"]

# Save an exact copy of the item's ArcGIS metadata
xslt_copy = os.path.join(dir, "Metadata/Stylesheets/gpTools/exact copy of.xslt")
output_copy = "C:/data/roads_copy.xml"
arcpy.conversion.XSLTransform(source_item, xslt_copy, output_copy)

# Save a copy where metadata content is filtered to remove machine names
filter = dir + "Metadata/Stylesheets/gpTools/remove local storage info.xslt"
output_filtered = "C:/data/roads_filtered.xml"
arcpy.conversion.XSLTransform(source_item, filter, output_filtered)

The following sample demonstrates how to save a copy of an item's metadata to an ArcGIS metadata format .xml file, and also a version of the same item's metadata that is filtered to remove any machine names that may be present using ArcGIS AllSource:

import arcpy
from arcpy import metadata as md

src_item_md = md.Metadata("C:/data/data.gdb/roads")

# Save an exact copy of the item's ArcGIS metadata
output_copy = "C:/data/roads_copy.xml"
src_item_md.saveAsXML(output_copy)

# Save a copy where metadata content is filtered to remove machine names
output_filtered = "C:/data/roads_filtered.xml'
src_item_md.saveAsXML(output_filtered, 'REMOVE_MACHINE_NAMES')

Remove content from an item's metadata

The capabilities that were provided in ArcGIS Desktop in the XSLT stylesheets listed below are now available in ArcGIS AllSource from the Metadata deleteContent method. Instead of specifying an XSLT stylesheet file, specify the corresponding option for deleting content, as indicated below.

ArcGIS Desktop XSLT stylesheet	ArcGIS AllSource metadata_delete_option
remove geoprocessing history.xslt	GPHISTORY
remove thumbnail.xslt	THUMBNAIL

The following sample demonstrates how to delete the geoprocessing history recorded in an item's metadata to an ArcGIS metadata format .xml file, and also a version of the same item's metadata that is filtered to remove any machine names that may be present using ArcGIS Desktop:

import arcpy
import os

source_item = "C:/data/data.gdb/roads"
dir = arcpy.GetInstallInfo("desktop")["InstallDir"]

# Save copy of the item's ArcGIS metadata that is filtered to remove geoprocessing history
xslt_no_gp_history = os.path.join(dir, "Metadata/Stylesheets/gpTools/remove_gp_history.xslt")
output_no_gp_history = "C:/data/roads_noHistory.xml"
arcpy.XSLTransform_conversion(source_item, xslt_no_gp_history, output_no_gp_history)

# Filter the output metadata content further to remove thumbnails as well
xslt_no_thumbs = dir + "Metadata/Stylesheets/gpTools/remove thumbnail.xslt"
output_nogp_nothumbs = "C:/data/roads_noHistory_noThumbnails.xml"
arcpy.conversion.XSLTransform(output_no_gp_history, xslt_no_thumbs, output_nogp_nothumbs)

The following sample demonstrates how to delete the geoprocessing history and any thumbnails from an item's metadata using ArcGIS AllSource:

import arcpy
from arcpy import metadata as md

# Get the target item's Metadata object
poles_path = r'C:\Data\LocalArea.gdb\UtilityPoles'
tgt_item_md = md.Metadata(poles_path)

# Delete all geoprocessing history and any thumbnails from the item's metadata
if not tgt_item_md.isReadOnly:
    tgt_item_md.deleteContent('GPHISTORY')
    tgt_item_md.deleteContent('THUMBNAIL')
    tgt_item_md.save()

Custom XSLT

Custom XSLT stylesheets can be created to generate .html files that can be used on an organization's website, for example, to share metadata describing maps and datasets that are available. Custom XSLT stylesheets can also be used to modify an item's ArcGIS metadata content, for example, to update contact information or access and use constraints.

In addition to the XSLT stylesheets described above, additional XSLT stylesheets were provided with ArcGIS Desktop to perform operations such as removing metadata elements from an item's ArcGIS metadata that were used before version 10.0 and have not been used since.

The following sample demonstrates how to save a copy of an item's metadata to an ArcGIS metadata format .xml file using a custom XSLT stylesheet that removes some of the item's original metadata content as part of the process:

import arcpy
import os

source_item = "C:/data/data.gdb/roads"
dir = arcpy.GetInstallInfo("desktop")["InstallDir"]

# Save copy of the item's ArcGIS metadata that is filtered to remove geoprocessing history
xslt_no_fgdcCsdgm = os.path.join(dir, "Metadata/Stylesheets/gpTools/remove pre94 metadata elements.xslt")
output_no_fgdcCsdgm = "C:/data/roads_copy.xml"
arcpy.conversion.XSLTransform(source_item, xslt_no_fgdcCsdgm, output_no_fgdcCsdgm)

The custom XSLT stylesheet used in the above example is provided with ArcGIS Desktop to use with the XSLT Transformation tool. It removes elements from the item's metadata that were used by the metadata editors provided before version 10.0 and that have not been in use since.

The following sample demonstrates how to save a copy of an item's metadata to an ArcGIS metadata format .xml file using a custom XSLT stylesheet that filters out content using ArcGIS AllSource:

import arcpy
import os

from arcpy import metadata as md
src_item_md = md.Metadata("C:/data/data.gdb/roads")

# Save copy of the item's ArcGIS metadata that is filtered to remove
# unused ESRI-ISO and FGDC CSDGM elements
output_file = "C:/data/roads_no_fgdcCsdgm.xml"
dir = arcpy.GetInstallInfo("desktop")["InstallDir"]
xslt_no_fgdcCsdgm = os.path.join(dir, "Metadata/Stylesheets/gpTools/remove pre94 metadata elements.xslt")
src_item_md.saveAsUsingCustomXSLT(output_file, xslt_no_fgdcCsdgm)

The XSLT stylesheet and the script above will run successfully in ArcGIS AllSource if you have ArcGIS Desktop installed. You can substitute your own custom XSLT in its place to make your own modifications to an item's metadata as it is saved to a stand-alone .xml file.

Note:

The following operations are all accomplished using XSLT stylesheets that process an item's ArcGIS metadata. The Microsoft .NET Framework XSLT processor is used to perform the transformation in all of the following scenarios:

• ArcGIS Desktop metadata tool XSLT Transformation, and some of the options used with the Import Metadata and Export Metadata tools
• The Save As, Import, Export, and Upgrade options that are available in ArcGIS AllSource on the Catalog tab on the ribbon in the Metadata group when the catalog view is active
• The ArcGIS AllSource .NET SDK methods SaveMetadataAsXML, SaveMetadataAsUsingCustomXSLT, and SaveMetadataAsHTML, ExportMetadata, ImportMetadata, and UpgradeMetadata.

However, the Metadata class saveAsXML, saveAsUsingCustomXSLT, importMetadata, exportMetadata, and upgrade methods are all accomplished using the Microsoft Core XML Services (MSXML) XSLT processor. As a result, some XSLT stylesheets that run successfully when the .NET Framework runs the transformation may not run successfully when the same XSLT stylesheet is used with the arcpy.metadata module, and vice versa.

Some XSLT stylesheets provided with ArcGIS Desktop whose capabilities are not available yet in ArcGIS AllSource will work successfully with the Metadata class saveAsWithCustomXSLT method. However, some stylesheets, such as the ones that generate .html files for display, will not work because the required .NET Framework XSLT functions or custom ArcGIS Desktop XSLT functions are not available when the MSXML XSLT processor is used to perform the transformations.

Edit metadata for many ArcGIS items

The ArcPy Metadata object has properties that allow you to access and modify the basic information that is displayed on the item details page when an item is published to ArcGIS Online. However, often the metadata content you need to add or update is more detailed. Performing these operations can be accomplished with Python packages such as xml.etree.ElementTree, PyXML, and lxml.

The ArcMap help provided an example of how to accomplish these updates using ElementTree. The example below is an updated version of that script that shows how to perform the same metadata updates in ArcGIS AllSource:

Batch update metadata for a list of datasets. This script is designed to be run as a script tool.

import os
import sys
import xml.etree.ElementTree as ET
import arcpy
from arcpy import metadata as md

# Script arguments...
Source_Metadata = arcpy.GetParameter(0)

# Local variables
#    new purpose text
newPurpose = "This is new text for an existing Purpose metadata element."
newCredits = "This is text for a new Credits metadata element."


# function to update purpose and credits in metadata
def update_metadata(root):
    num_elements = 0

    # modify purpose element's text
    # there is only supposed to be one purpose element in metadata
    # replace purpose element text if element exists
    # if element doesn't exist, do nothing
    purposeEls = root.findall(".//idPurp")
    for element in purposeEls:
        if element.text is not None:
            element.text = newPurpose
            num_elements += 1

    # add credits element to dataIdInfo parent, if the parent exists
    # ISO allows many dataIdInfo groups; ArcGIS supports only one, so get the 1st
    # ISO allows many idCredit elements, and many are supported in ArcGIS
    # append a new idCredit element with appropriate text, existing elements remain
    dataIdInfoEls = root.findall("./dataIdInfo[1]")
    for element in dataIdInfoEls:
        if element:
            newCreditEl = ET.SubElement(element, "idCredit")
            newCreditEl.text = newCredits
            num_elements += 1

    return num_elements


# get and save item metadata
for item in Source_Metadata:
    arcpy.AddMessage("Item: {0}".format(item))
    
    # get the item's metadata xml
    item_md = md.Metadata(item)
    metadata_xml_string = item_md.xml
    
    # create an ElementTree object from the metadata XML string
    root = ET.fromstring(metadata_xml_string)

    # call the update_metadata function to modify the item's metadata
    changes = update_metadata(root)
    
    if changes > 0:
        # get modified XML
        updated_xml_string = ET.tostring(root, encoding="utf-8")
        
        # import result back into metadata 
        arcpy.AddMessage("Saving updated metadata with the item...")
        item_md.xml = updated_xml_string
        item_md.save()
    else:
        arcpy.AddMessage("No changes to save")


arcpy.AddMessage('Finished updating metadata for all source metadata items')

ArcGIS Desktop tools that will not be available in ArcGIS AllSource

Some tools that were available in ArcGIS Desktop will not be migrated to ArcGIS AllSource.

USGS MP Metadata Translator

The USGS MP Metadata Translator tool provides limited access to the capabilities provided by the USGS's metadata parser utility, mp. For example, it only accepts .xml files as input. You cannot perform operations such as converting a structured text file to an FGDC CSDGM-format .xml file. However, when the same utility is accessed directly from Python, all of mp's capabilities are available. Additionally, the version of mp that is provided with ArcGIS Desktop is out of date. Download the most recent version of the USGS's metadata parser utility, mp, from the website, and run it in your Python script using os.system as illustrated below:

Use the USGS metadata parser utility, mp, to convert a structured text file to an FGDC CSDGM-format XML file.

import os

csdgm_xml_file = "D:/Metadata_Pro/arcpy/Map_current_csdgm.xml"
csdgm_txt_file = "D:/Metadata_Pro/arcpy/Output/csdgm_to_text.txt"
runCmd = "D:/Metadata_Pro/arcpy/mp.exe " + csdgm_txt_file + " -x " + csdgm_xml_file
print(runCmd)
os.system(runCmd)

Esri Metadata Translator

The Esri Metadata Translator tool used code to convert metadata from one format to another. This conversion mechanism would produce messages that were difficult to understand, such as Validation failed: em:Metadata($a) --> em:contact($a, $b), em:CI_ResponsibleParty($b) and Error [InvalidForSome]: <MD_Metadata> for-some clause: em:contact(v:Metadata, v:Object) <contact>.

The transformations used in ArcGIS AllSource to import and export metadata will not produce messages of this type. In ArcGIS AllSource, all transformations between the ArcGIS metadata XML format and standards-based XML formats are accomplished using XSLT transformations instead. Use the appropriate Metadata class methods to import and export metadata.

Export Metadata Multiple and Validate Metadata Multiple

The Export Metadata Multiple and Validate Metadata Multiple tools were provided to support buttons on the ArcCatalog Metadata toolbar. These capabilities are supported in ArcGIS AllSource without the need for these tools.

In an ArcGIS AllSource Python script, you can export (and in the future validate) metadata for many items by accessing datasets and files using the arcpy list functions, and accessing maps and layers using the arcpy.mp list functions. Use the appropriate Metadata class method to manage metadata for the items returned by those lists.

Metadata Publisher

The Metadata Publisher tool was provided in ArcGIS Desktop to publish metadata to an ArcIMS Metadata Service. ArcIMS has been retired.