Manage multidimensional raster data

Introduction

Earth science data—such as those related to the atmosphere, oceanography, weather, and the environment—are often modeled over multiple time steps and/or at various depths or heights. These multidimensional datasets, whether captured by satellites or generated through numerical models, are typically stored in formats such as netCDF, GRIB, or HDF. In recent years, the Zarr format has gained popularity for storing multidimensional raster data due to its efficiency and optimization for cloud-based access.

Multidimensional raster formats can store one or multiple variables, and each variable may have multiple dimensions. Many organizations produce such data and maintain extensive collections of Earth science variables. ArcGIS, including ArcGIS Pro, ArcGIS Enterprise, and ArcGIS Online, allows you to manage and share your multidimensional data. This document summarizes the key capabilities and best practices for working with your multidimensional data and making it available on the web.

ArcGIS data models for working with multidimensional raster

There are three main approaches to working with multidimensional formats in ArcGIS:

• Direct access to a single multidimensional dataset
• Using a mosaic dataset
• Creating a Cloud Raster Format (CRF)

These methods allow you to work with one or more multidimensional datasets stored in formats such as netCDF, HDF, GRIB, or Zarr, as well as other raster formats, along with additional information that defines dimensions.

These multidimensional formats are open and inherently flexible, which has led to many variations in how the data is stored. ArcGIS supports the following conventions, and producing data according to these conventions ensures your data is natively supported in ArcGIS:

• netCDF following CF-1 convention
• HDF4 (HDF-EOS2) and HDF5 (HDFEOS5)
• GRIB written with the NCEP library
• Zarr version 2 and version 3

Direct access to a single multidimensional file

You can access a multidimensional raster from any supported multidimensional format natively in ArcGIS Pro.

1. Use the Add Multidimensional Data dialog box (click Add Data > Multidimensional Data on the Map tab) to create a multidimensional raster layer.
2. In the Contents pane, select the multidimensional layer created, then use the tools on the Multidimensional tab to explore and manage variables and dimensions, visualize changes over time and/or at other dimensions and perform multidimensional analysis. For detailed instructions, refer to the Multidimensional Analysis Workflow tutorial.

Use a mosaic dataset to manage multiple datasets

For a collection of multidimensional datasets, you can use a mosaic dataset to organize and aggregate them into a single virtual multidimensional raster. Refer to Create a multidimensional mosaic dataset for steps to create a multidimensional mosaic dataset from netCDF files.

When adding netCDF, HDF, or GRIB files to a mosaic dataset, use the corresponding multidimensional raster types. These raster types automatically build the multidimensional information.

You can also create a multidimensional mosaic dataset from a collection of TIFF files or other formats. In this case, you must add a field in the footprint table and assign time values or values of other dimensions to each raster. Use the Build Multidimensional Info tool to enable multidimensional support. Refer to the step-by-step workflow to create a multidimensional mosaic dataset from a collection of TIFF images.

A mosaic dataset can be modified as needed—you can add or remove slices or variables using the Add Rasters To Mosaic Dataset and Remove Rasters From Mosaic Dataset tools. You can also modify a mosaic dataset by importing the footprints or boundary.

Mosaic datasets are often used in the following scenarios:

• Limited storage space—Mosaic datasets manage only metadata by storing references to data files. They do not duplicate the source data.
• Flexible updates—Mosaic datasets support flexible updates, allowing you to insert, remove, or delete slices as needed.
• Short time window with frequent updates—Certain types of data such as weather forecast data are characterized by short temporal windows, frequent updates, and identical variable names. Some may have identical file names. A mosaic dataset is recommended when managing such data as a live service. You can prepare a table with the required multidimensional fields and create the mosaic datasets using the Table Raster Type. This method is efficient because it does not require opening each file during data creation. When new data arrives, you only need to update the time values in the table and then update or re-create the mosaic dataset. This workflow is especially advantageous when data files are stored in the cloud.
• Sparse or discontinuous data—For scene-by-scene data captured at different locations and times such as satellite imagery, a standard mosaic dataset (without multidimensional info) is often a good option. You can use the Time slider to work with this type of data. If needed, you can create a multidimensional raster with a time interval using the Process Raster Collection function.

Use a multidimensional Cloud Raster Format (CRF) for optimized performance

CRF is a raster data format optimized for cloud storage and distributed computing. It is designed for optimized multidimensional raster management and analysis. It can store multiple variables with multiple dimensions in one dataset, and store a transposed of the data, which accelerates analysis along the time or other dimensions. CRF supports LERC compression, which allows you to compress your data with controlled data accuracy.

To convert a mosaic dataset or a multidimensional raster to a multidimensional CRF, use the Copy Raster tool. You can also use tools in the Multidimensional toolbox to manage your multidimensional CRF datasets.

You can modify a CRF by inserting or removing variables, or appending slices, using the Manage Multidimensional Raster tool. To extend a CRF spatially, use the Mosaic Raster tool.

To enhance performance for multidimensional analysis, you can build a transpose for the CRF.

Note:

When building a transpose, a temporary file of the same size as the CRF will be written to your local temporary directory. Ensure that you have adequate disk space before proceeding.

CRF is particularly suitable for the following scenarios:

• Fast slice display performance—CRF uses a tiled structure to optimize data access and visualization.
• Multidimensional analysis on long time series—Tasks such as temporal profiling, trend analysis, and time series change detection benefit significantly from having a transpose, which optimizes performance along the time dimension.

Note:

Updating a CRF is not as flexible as a mosaic dataset, and building a transpose requires additional time and disk space.

Frequently asked questions

1. Can I define a default variable on my multidimensional raster?

   Yes, you can define a default variable in the dataset by navigating to Catalog Pane → Raster Dataset Properties → Multidimensional Info tab → Default Variable. This variable will be used by default when the layer is added in ArcGIS Pro or published to the web.

2. Can I predefine symbology for variables?

   Yes, you can define how the variables are to be displayed. The symbology defined for each variable is saved within the layer.

   • Refer to Register symbology for a raster layer for defining symbology for variables in ArcGIS Pro.
   • To create a multidimensional imagery layer with styled variables, use user experience in the web viewer.
3. When should I use raster function templates?

   Raster function templates are useful when you need to modify variable values, such as changing units or performing simple calculations, without re-creating the datasets. You can attach a raster function template to either a mosaic dataset or a CRF, enabling dynamic, server-side processing.

   Note:

   • Server-side processing with templates is supported for image services or dynamic imagery layers, not for tiled imagery layers.
   • Since raster function templates modify pixel values, they should not be used to define display symbology. Instead, style the variables and save the symbology within the layer or layer item.
4. Can I publish a Zarr?

   Yes, Zarr is supported as a raster dataset. You can publish a Zarr dataset just like any other raster dataset using ArcGIS Image Server or ArcGIS Enterprise. However, publishing Zarr datasets is only supported by the By Reference option—publishing value option is not supported.

5. Should I create a single dataset or multiple datasets?

   While mosaic datasets and CRFs can manage multiple variables and diverse data, organizing your datasets and imagery layers based on the data characteristics and intended uses offers clear advantages. The best practice is to organize by project or by inherent properties of the data. Here are some guidelines to help you decide:

   • Different temporal resolutions—For the same variable with different temporal resolutions—such as daily temperature versus monthly temperature—it's best to manage them as separate datasets. This avoids complications, because ArcGIS normalizes all time values into a standard time format (StdTime), rather than mixing the same variable of different temporal resolutions in one dataset.
   • Different bands—When creating CRFs, it's recommended that you group data with the same number of bands. Mixing datasets with different band counts can increase storage complexity and affect analysis and visualization.
   • Different spatial resolutions—Mosaic datasets and CRFs automatically resample data when they are in differing spatial resolutions. While this can be convenient, downsampling may reduce data accuracy, and upsampling can unnecessarily increase storage usage. To preserve data fidelity and efficiency, create separate datasets that maintain original spatial resolutions.
   • Wind or ocean data—For variables such as wind or ocean currents, which include both magnitude and direction, or U and V components, it is best to store them in a dedicated dataset. These are commonly visualized using the Vector Field symbology in ArcGIS Pro or using the Flow renderer in web maps. Managing these variables separately simplifies styling and use. For more details, refer to the Visualize raster using a vector field blog.

Publish multidimensional imagery layers using ArcGIS Enterprise

Once you've created the data models, you can share the CRFs, mosaic datasets, or Zarr datasets as web imagery layers using ArcGIS Enterprise. Refer to Share a web imagery layer for publishing via ArcGIS Online.

Note:

Zarr can only be published with the Reference register data option.

To publish a live imagery layer, do the following:

• When sharing the imagery layer, ensure the Has live data option is checked: Share As Web Layer > Configuration > Image Service Properties > Has live data.
• Update the underlying mosaic dataset or CRF using tools described earlier, or the corresponding Python API ( manage_multidimensional_raster ), or REST API ( Manage Multidimensional Raster endpoint).
• Call the Refresh Service operation to update service extent and multidimensional info.

Published imagery layers can be accessed via the Multidimensional tab in ArcGIS Pro or Multidimensional filter in Map Viewer.

Publishing multidimensional imagery layers in ArcGIS Online

ArcGIS Online supports two types of imagery layers for multidimensional data: tiled imagery layers and dynamic imagery layers. Refer to Imagery layers to learn about the differences between the two layers, and choose the one that best fits your requirements.

Typically, tiled imagery layers are preferred for their fast display performance and public sharing capability. Dynamic imagery layers can only be shared within your organization.

You can create imagery layers using the web experience, or using the ArcGIS Pro user experience. In many cases, preparing or testing data models using the advanced tools in ArcGIS Pro is a good approach. Once ready, upload the resulting CRFs or mosaic datasets to ArcGIS Online for publishing.