The June 2024 update for ArcGIS Data Pipelines includes new input formats, enhanced tools, higher scheduling frequencies, and more. For more information, see the What's New in Data Pipelines (June 2024) blog article.
Inputs
The following new input enhancements are available:
- You can use GeoParquet files from Amazon S3, Microsoft Azure Storage, or Public URL. GeoParquet files are not supported for upload to your content or for use as a File input.
- You can use file geodatabases from Amazon S3, Microsoft Azure Storage, or files from ArcGIS Online content. Only feature class and table datasets are supported.
Tools
The following tool enhancements are available:
- The Calculate field tool now supports Arcade geometry functions to create and modify geometry field values.
- The Filter by attribute tool now supports in the last and not in the last operations for date and date only fields. For example, you can find records that occur in the last three days to extract a subset of the most recent data.
Scheduling
The following scheduling enhancements are available:
- You can now schedule a data pipeline to run at more frequent intervals, such as every 15, 30, or 45 minutes, by using the new repeat type option of Minute.
- Use the new Maximum run duration parameter to specify how long a task can run before it is automatically canceled.
Other enhancements
The following general enhancements are available:
- The connection details dialog box now has an option to disconnect all open data pipeline editors. This allows you to actively disconnect the editor and manage credit usage. To learn more about connections in Data Pipelines, see Compute resources.
- Preview and run messages can now be filtered based on their severity so you can more easily access the messages of interest.