Train Text Transformation Model (GeoAI)

Summary

Trains a text transformation model to transform, translate, or summarize text.

Learn more about how Text Transformation works

Usage

• This tool requires deep learning frameworks be installed. To set up your machine to use deep learning frameworks in AllSource, see Install deep learning frameworks for ArcGIS.

• This tool can also be used to fine-tune an existing trained model.

• To run this tool using GPU, set the Processor Type environment to GPU. If you have more than one GPU, specify the GPU ID environment instead.

• The input to the tool is a table or a feature class containing training data, with a text field containing the input text and a label field containing the transformed text.

• For information about requirements for running this tool and issues you may encounter, see Deep Learning frequently asked questions.

Parameters

Dialog

Label	Explanation	Data Type
Input Table	A feature class or table containing a text field with the input text for the model and a label field containing the target transformed text.	Feature Layer; Table View
Text Field	A text field in the input feature class or table that contains the input text that will be transformed by the model.	Field
Label Field	A text field in the input feature class or table that contains the target transformed text for training the model.	Field
Output Model	The output folder location that will store the trained model.	Folder
Pretrained Model File (Optional)	A pretrained model that will be used to fine-tune the new model. The input can be an Esri model definition file (.emd) or a deep learning package file (.dlpk). A pretrained model that performs a similar task can be fine-tuned to fit the training data. The pretrained model must have been trained with the same model type and backbone model that will be used to train the new model.	File
Max Epochs (Optional)	The maximum number of epochs for which the model will be trained. A maximum epoch value of 1 means the dataset will be passed forward and backward through the neural network one time. The default value is 5.	Long
Model Backbone (Optional)	Specifies the preconfigured neural network that will be used as the architecture for training the new model. t5-small—The new model will be trained using the T5 neural network. T5 is a unified framework that converts every language problem into a text-to-text format. t5-small is the small variant of T5. t5-base—The new model will be trained using the T5 neural network. T5 is a unified framework that converts every language problem into a text-to-text format. t5-base is the medium variant of T5. t5-large—The new model will be trained using the T5 neural network. T5 is a unified framework that converts every language problem into a text-to-text format. t5-large is the large variant of T5.	String
Batch Size (Optional)	The number of training samples that will be processed at one time. The default value is 2. Increasing the batch size can improve tool performance; however, as the batch size increases, more memory is used. If an out of memory error occurs, use a smaller batch size.	Double
Model Arguments (Optional)	Additional arguments for initializing the model, such as seq_len for the maximum sequence length of the training data, that will be considered for training the model. See keyword arguments in the SequenceToSequence documentation for the list of supported models arguments that can be used.	Value Table
Learning Rate (Optional)	The step size indicating how much the model weights will be adjusted during the training process. If no value is specified, an optimal learning rate will be deduced automatically.	Double
Validation Percentage (Optional)	The percentage of training samples that will be used for validating the model. The default value is 10.	Double
Stop when model stops improving (Optional)	Specifies whether model training will stop when the model is no longer improving or until the Max Epochs parameter value is reached. Checked—The model training will stop when the model is no longer improving, regardless of the Max Epochs parameter value specified. This is the default. Unchecked—The model training will continue until the Max Epochs parameter value is reached.	Boolean
Make model backbone trainable (Optional)	Specifies whether the backbone layers in the pretrained model will be frozen, so that the weights and biases remain as originally designed. Checked—The backbone layers will not be frozen, and the weights and biases of the Model Backbone parameter value can be altered to fit the training samples. This takes more time to process but typically produces better results. This is the default. Unchecked—The backbone layers will be frozen, and the predefined weights and biases of the Model Backbone parameter value will not be altered during training.	Boolean
Remove HTML Tags (Optional)	Specifies whether HTML tags will be removed from the input text. Checked—The HTML tags in the input text will be removed. This is the default. Unchecked—The HTML tags in the input text will not be removed.	Boolean
Remove URLs (Optional)	Specifies whether URLs will removed from the input text. Checked—The URLs in the input text will be removed. This is the default. Unchecked—The URLs in the input text will not be removed.	Boolean

Python

arcpy.geoai.TrainTextTransformationModel(in_table, text_field, label_field, out_model, {pretrained_model_file}, {max_epochs}, {model_backbone}, {batch_size}, {model_arguments}, {learning_rate}, {validation_percentage}, {stop_training}, {make_trainable}, {remove_html_tags}, {remove_urls})

Name	Explanation	Data Type
in_table	A feature class or table containing a text field with the input text for the model and a label field containing the target transformed text.	Feature Layer; Table View
text_field	A text field in the input feature class or table that contains the input text that will be transformed by the model.	Field
label_field	A text field in the input feature class or table that contains the target transformed text for training the model.	Field
out_model	The output folder location that will store the trained model.	Folder
pretrained_model_file (Optional)	A pretrained model that will be used to fine-tune the new model. The input can be an Esri model definition file (.emd) or a deep learning package file (.dlpk). A pretrained model that performs a similar task can be fine-tuned to fit the training data. The pretrained model must have been trained with the same model type and backbone model that will be used to train the new model.	File
max_epochs (Optional)	The maximum number of epochs for which the model will be trained. A maximum epoch value of 1 means the dataset will be passed forward and backward through the neural network one time. The default value is 5.	Long
model_backbone (Optional)	Specifies the preconfigured neural network that will be used as the architecture for training the new model. t5-small—The new model will be trained using the T5 neural network. T5 is a unified framework that converts every language problem into a text-to-text format. t5-small is the small variant of T5. t5-base—The new model will be trained using the T5 neural network. T5 is a unified framework that converts every language problem into a text-to-text format. t5-base is the medium variant of T5. t5-large—The new model will be trained using the T5 neural network. T5 is a unified framework that converts every language problem into a text-to-text format. t5-large is the large variant of T5.	String
batch_size (Optional)	The number of training samples that will be processed at one time. The default value is 2. Increasing the batch size can improve tool performance; however, as the batch size increases, more memory is used. If an out of memory error occurs, use a smaller batch size.	Double
model_arguments [model_arguments,...] (Optional)	Additional arguments for initializing the model, such as seq_len for the maximum sequence length of the training data, that will be considered for training the model. See keyword arguments in the SequenceToSequence documentation for the list of supported models arguments that can be used.	Value Table
learning_rate (Optional)	The step size indicating how much the model weights will be adjusted during the training process. If no value is specified, an optimal learning rate will be deduced automatically.	Double
validation_percentage (Optional)	The percentage of training samples that will be used for validating the model. The default value is 10.	Double
stop_training (Optional)	Specifies whether model training will stop when the model is no longer improving or until the max_epochs parameter value is reached. STOP_TRAINING—The model training will stop when the model is no longer improving, regardless of the max_epochs parameter value specified. This is the default. CONTINUE_TRAINING—The model training will continue until the max_epochs parameter value is reached.	Boolean
make_trainable (Optional)	Specifies whether the backbone layers in the pretrained model will be frozen, so that the weights and biases remain as originally designed. TRAIN_MODEL_BACKBONE—The backbone layers will not be frozen, and the weights and biases of the model_backbone parameter value can be altered to fit the training samples. This takes more time to process but typically produces better results. This is the default. FREEZE_MODEL_BACKBONE—The backbone layers will be frozen, and the predefined weights and biases of the model_backbone parameter value will not be altered during training.	Boolean
remove_html_tags (Optional)	Specifies whether HTML tags will be removed from the input text. REMOVE_HTML_TAGS—The HTML tags in the input text will be removed. This is the default. DO_NOT_REMOVE_HTML_TAGS—The HTML tags in the input text will not be removed.	Boolean
remove_urls (Optional)	Specifies whether URLs will be removed from the input text. REMOVE_URLS—The URLs in the input text will be removed. This is the default. DO_NOT_REMOVE_URLS—The URLs in the input text will not be removed.	Boolean

Code sample

TrainTextTransformationModel (Python window)

The following Python window script demonstrates how to use the TrainTextTransformationModel function.

# Name: TrainTextTransformation.py
# Description: Train a sequence-to-sequence model to translate text from English to German.  
#
# Requirements: ArcGIS Pro Advanced license

# Import system modules
import arcpy
import os

# Set local variables
in_table = "training_data.csv"
out_folder = "c\\texttransformer"

# Run Train Text Transformation Model
arcpy.geoai.TrainTextTransformationModel(in_table, out_folder, max_epochs=2,
         text_field="input", label_field="target", batch_size=16)

Environments

Processor Type, GPU ID