Create geoprocessing modules

Geoprocessing toolboxes can be distributed as Python packages. The process for building and distributing a toolbox involves creating a Python package and extending the package structure with an esri folder to include your own toolboxes. Once the package is installed in a Python environment, the toolbox will appear as and be accessible in all the ways that a system toolbox in ArcGIS AllSource would, accessible through the Geoprocessing pane and from arcpy. In addition, a custom geoprocessing module can take advantage of the well-established methodology that ArcGIS AllSource system toolboxes have for message distribution, language-based help, and responding to localized settings.

Create a Python package

For illustrative purposes, we will create a Python package named foo that contains the module bar.py.

bar.py

Sample code for the bar Python module.

import os 
 
def hello(): 
    return f'Hello {os.getenv("username")}'

For the package to be built and distributed, a specific directory structure must exist. A directory named foo must be created to store the bar module. Since distribution requires that the directory storing the bar module be within a parent directory, a directory named src is created to house the foo directory and the bar module. The directory structure will be as follows:

 src
└───foo
     └  bar.py

For the bar module to initialize and run specific code once it has been imported, it requires an __init__.py file. By using an __init__.py file, the foo directory is treated as a package by Python.

__init__.py

Sample code for __init__.py in foo.

from foo import bar

The directory structure will now be the following:

src
└───foo
     ├  __init__.py 
     └  bar.py

Note:

Throughout the process, additional __pycache__ folders may appear in the directories. These folders are caches of compiled versions of modules generated by Python to speed up the loading of modules. To simplify the folder structure diagrams, they are not shown here.

With the files and directory structure, the bar module can be imported through import foo, and the foo.bar.hello() function can be run.

Note:

To import a module in Python, the module must be in the current directory, a directory listed in sys.path, or a directory listed with the PYTHONPATH environment variable.

Extend a Python package with geoprocessing tools

To extend a Python package with custom or Python toolboxes, the toolboxes and accompanying files are organized into a module in an esri folder. The standard directory structure for a Python package extended with custom geoprocessing functionality is the following:

Src
└──foo
    ├  __init__.py 
    ├  bar.py
    └──esri
        ├──arcpy
        ├──help
        │   └──gp
        │       ├──messages
        │       └──toolboxes
        └──toolboxes

Custom toolboxes are placed in the esri\toolboxes directory along with any supporting .py files. The toolbox and tool metadata files (.xml) are stored in the esri\help\gp folder.

• The naming convention for a toolbox's metadata file is the toolbox alias followed by _toolbox.
• The naming convention for a tool's metadata file is the tool name followed by the toolbox alias, separated by an underscore.

Any geoprocessing message .xml files are stored in the esri\help\gp\messages directory. The message files are used in the Python toolboxes for messages that need to be internationalized.

The process of extending geoprocessing through Python packages can be demonstrated through the creation of a Python toolbox named SamplePythonToolbox.

Learn more about creating a Python toolbox

SamplePythonToolbox.pyt

Sample code to create a Python toolbox.

# -*- coding: utf-8 -*-

import arcpy
import os
import foo

class Toolbox:
    def __init__(self):
        """Define the toolbox (the name of the toolbox is the name of the
        .pyt file)."""
        self.label = "Sample Python Toolbox"
        self.alias = "SamplePythonToolbox"

        # List of tool classes associated with this toolbox
        self.tools = [SampleTool]

class SampleTool:
    def __init__(self):
        """Define the tool (tool name is the name of the class)."""
        self.label = "Sample Tool"
        self.description = ""

    def getParameterInfo(self):
        """Define parameter definitions"""
        parameters=[arcpy.Parameter(displayName='Msg', 
                                  name='msg',
                                  datatype='GPString',
                                  parameterType='Derived',
                                  direction='Output')
                                  ]
        return parameters

    def isLicensed(self):
        """Set whether tool is licensed to execute."""
        return True

    def updateParameters(self, parameters):
        """Modify the values and properties of parameters before internal
        validation is performed.  This method is called whenever a parameter
        has been changed."""
        return

    def updateMessages(self, parameters):
        """Modify the messages created by internal validation for each tool
        parameter.  This method is called after internal validation."""
        return

    def execute(self, parameters, messages):
        """The source code of the tool."""
        result = foo.bar.hello()
        messages.AddMessage(f"{result}, welcome to the sample tool")
        parameters[0].value = result
        return

Note:

To test the toolbox at this stage, temporarily place it into the root src folder. Since foo is being imported in the toolbox, the tool will likely not run properly unless it is in the same directory as the foo package.

In the SamplePythonToolbox.pyt toolbox, the foo package is imported and the execute method of the SampleTool class calls the hello()  function from the bar module.

This approach is an effective way of distributing custom Python code as a package and exposing its functionality through ArcGIS AllSource geoprocessing tools. Once the SamplePythonToolbox.pyt toolbox has been created, the parameter help can be configured for the toolbox through the metadata. To edit a tool's metadata, from the Catalog pane, right-click the tool in the toolbox and choose Edit Metadata.

After the metadata is updated for the tool and toolbox, the arcpy.gp.createtoolboxsupportfiles function can be run to generate the supporting .xml files found in the esri folder structure described above. This command also produces an arcpy wrapper, SamplePythonToolbox.py, named after the tool, which allows the toolbox to be accessible from arcpy. If the file isn't there already, place the SamplePythonToolbox.pyt into the root src folder, then run python -c "import arcpy; arcpy.gp.createtoolboxsupportfiles(r'<path to toolbox>')" from the Python Command Prompt.

Note:

Ensure that the toolbox has an alias before running arcpy.gp.createtoolboxsupportfiles. The alias is needed to generate the help files.

In the resulting esri folder, create a toolboxes folder and move SamplePythonToolbox.pyt into it; the accompanying metadata (.pyt.xml) files can be discarded. The esri folder should now look like the following:

esri
 ├──arcpy
 │   └  SamplePythonToolbox.py
 ├──help
 │   └──gp
 │       ├  SamplePythonToolbox_toolbox.xml
 │       ├  SampleTool_SamplePythonToolbox.xml
 │       ├──messages 
 │       └──toolboxes
 │           └  SamplePythonToolbox.xml
 └──toolboxes
     └  SamplePythonToolbox.pyt

Note:

The messages folder is not generated by arcpy.gp.createtoolboxsupportfiles; it must be created manually to make the error messages used in the script tools or Python toolboxes localizable, but can be omitted if there is no plan for internationalization.

Move the esri folder into the src\foo directory. The directory structure for the distribution will now be as follows:

Src
└──foo
    ├  __init__.py 
    ├  bar.py
    └──esri
        ├──arcpy
        │   └  SamplePythonToolbox.py
        ├──help
        │   └──gp
        │       ├  SamplePythonToolbox_toolbox.xml
        │       ├  SampleTool_SamplePythonToolbox.xml
        │       ├──messages
        │       └──toolboxes
        │           └  SamplePythonToolbox.xml
        └──toolboxes
            └  SamplePythonToolbox.pyt

For English language distributions, no additional steps are required. For more information on extending geoprocessing for distribution in languages other than English, see Internationalization of geoprocessing modules. To learn how to distribute your module, see Distribute a geoprocessing module.