The CLIP Zero-Shot Classifier pretrained model available on ArcGIS Living Atlas of the World is a deep learning model that is used to classify images.
This deep learning model, powered by CLIP (Contrastive Language-Image Pretraining), is used to classify an image into one of the several classes specified by the user. It uses CLIP's pretrained knowledge of image and text relationships to classify images based on your provided class names. This model works well on oriented imagery and works on single input image or a folder full of images. In addition to assigning a class to the image, it also provides a probability score for the classification.
License requirements
To complete this workflow, the following are the license requirements:
- ArcGIS Desktop—ArcGIS Image Analyst extension for ArcGIS Pro
- ArcGIS Enterprise—ArcGIS Image Server with raster analytics configured
- ArcGIS Online—ArcGIS Pro or Professional Plus user type.
Model overview
This model has the following characteristics:
- Input—8-bit RGB oriented images.
- Output—Feature class with information about classification of the image.
- Compute—This workflow is compute-intensive, and a GPU with a minimum CUDA compute capability of 6.0 is recommended.
- Applicable geographies—This model is expected to work well globally.
- Architecture—The implementation is based on the OpenAIs CLIP with ViT-B-32 transformer architecture.
- Accuracy metrics—Refer to the original CLIP model card .
Access and download the model
Download the CLIP Zero-Shot Classifier pretrained model from ArcGIS Living Atlas of the World. Alternatively, access the model directly from ArcGIS Pro, or consume it in ArcGIS Image for ArcGIS Online.
- Browse to ArcGIS Living Atlas of the World.
- Sign in with your ArcGIS Online credentials.
- Search for CLIP Zero-ShotClassifier and open the item_page from the search results.
- Click the Download button to download the model.
You can use the downloaded .dlpk file directly in ArcGIS Pro.
Release notes
The following are the release notes:
| Date | Description |
|---|---|
| June 2024 | First release of CLIP Zero-Shot Classifier |