The February 2024 update for ArcGIS Data Pipelines includes new input formats, tools, and general enhancements. Additionally, Data Pipelines is no longer in beta and is now available for general use. For more information, see the ArcGIS Data Pipelines is Now Available blog article.
Inputs
The following new inputs and enhancements are available:
- You can use JSON files from Amazon S3, Microsoft Azure Storage, or Public URL. JSON files are not supported for upload to your content or for use as a File input.
- The method to create a data store item has been improved. Previously there was a button in the ArcGIS Data Pipelines editor toolbar for creating data stores. Now you can browse to Inputs, choose the type you want to use, and you will be presented with an option to create a data store or use an existing data store.
Tools
The following new tools and enhancements are available:
- Map fields—A tool to quickly update the schema for a dataset by mapping it to the schema of an existing feature layer, or another dataset in your data pipeline.
- Unnest field—A tool that returns values stored in your array, map, or struct fields as new fields or rows.
- Calculate field—You can access complex field types (array, map, struct) in the profile variables list in the Arcade editor. In addition, there is an output field type option of Date only.
- Select fields—You can access nested fields from struct field types in the field pick list and return them as new fields.
Outputs
The following outputs enhancements are available:
- When writing output feature layers, the default length for creating string fields is now 4,000. To create string fields with a different length, use the String field length parameter. To learn more about setting output string lengths, see Feature layer output.
- Improved errors and warnings when writing to feature layers.
General enhancements
The following general enhancements are available:
- A guided tour is available when you first enter the editor. Use the guided tour to become familiar with core features and concepts in ArcGIS Data Pipelines. You can skip or complete the guided tour. Once skipped or completed, you will not be presented with the dialog box the next time you enter the application. To find the guided tour again, use the Guided tour button on the editor toolbar.
- A Connection details dialog box is available in the editor for viewing information such as the time the compute resource started, and the duration for which the compute resource has been active.
- When you save data pipeline items, they will be saved with a custom thumbnail that represents a preview of your data pipeline workflow. If you set your own thumbnail on the item, it will not be overwritten.
- The method to relabel the elements in the canvas has changed. Previously each tool pane had a label parameter. Now you can click the element in the canvas, and choose the Rename option to give the element a new label.
- Improved errors, warnings, and validation in the editor and for scheduled runs.