Use the model

You can use this model to identify or redact common objects in images captured with the Survey123 field app.

The model must be configured for a survey using Survey123 Connect.

Note:

While it is not recommended to use this model in production surveys, it can be useful for demonstration purposes and to get started with smart assistants in Survey123. You are responsible for the use of this model. When using Survey123, it is your responsibility to review and manually correct its outputs.

Link the model to a survey

For the following steps, it is assumed you are working with an existing survey. To create a survey for use with this model, see Create and publish a survey.

1. Open the survey in Survey123 Connect.
2. Click the Linked Content tab.

   

3. Click Link content and choose Deep learning package.

   If Deep learning package is not in the list of options, tap and hold the title of the Link Content pop-up, and additional options will appear in the list.

   

4. Click the Filter button and disable all filters.
5. Type common object detection in the search bar.

   

6. Select the Common Object Detection model and click OK.

   

7. Click the download button to download the model to your computer.

   Note:

   You must download the model to test it in Survey123 Connect.

   

The deep learning package is saved to the survey's media folder.

Note:

You can also download the model from ArcGIS Living Atlas of the World and copy it manually to the survey's media folder. For more information, see Access and download the model.

Add redaction to a survey

You can use the model to redact objects in images. The model must be linked to the survey.

To add the model to an image question in the survey and redact objects, complete the following steps:

1. Open the XLSForm for the survey in Survey123 Connect.
2. Add an image question.
3. In the bind::esri:parameters column, type redaction=CommonObjectDetection.

   You can specify additional parameters to control the camera preview, minimum detection score, bounding boxes, and labels in the camera preview. Parameters are separated by the ampersand character (&).

4. Set the cameraPreview parameter to true and the class parameter to person to limit redaction to only that class.

   The value in the bind::esri:parameters column should look like the following:

   

5. Publish the survey.

The survey is ready for use in the Survey123 field app.

Redact objects in the field app

To use the model to redact objects in images, complete the following steps:

1. Open the Survey123 field app.
2. Download and open the survey.
3. For the image question, take a photo with the camera or add an image from your device.

   The image opens in the redaction editor with bounding boxes around the detected objects.

4. Add, modify, or remove the bounding boxes as necessary.

   

5. Save the image.

   

6. Submit the survey.

The image is submitted with the survey with regions redacted using the effect specified by the survey author (pixelate, blur, or blockout).

In addition to redacting images, you can use this model with smart attributes and smart annotation to identify objects in the Survey123 field app. For more information, see Smart assistants.