The HTTP Poller data source in ArcGIS Velocity sends an HTTP request to the specified URL. This ingests data that can be retrieved in a web browser by browsing to the same URL.
Example
A GIS analyst wants to load district polygons stored in a .csv file on a web server into Velocity.
Usage notes
- The HTTP endpoint must be externally accessible on the internet.
- The size of the HTTP response or file being polled cannot exceed 16 MB. If the data or file being polled exceeds 16 MB, features will not be ingested.
- After configuring source connection parameters, configure the input data to define the schema and the key parameters.
- Both the URL parameters and Custom headers parameters can use analytic global variables for scheduling recurring big data analysis. When analytics run, these global variables are converted into epoch time (milliseconds since January 1, 1970 UTC). The following outlines the available analytic variables and the time they represent.
- $analytic.AnalyticLastScheduledStartTime—The time the last run of the analytic was scheduled to start
- $analytic.AnalyticScheduledStartTime—The time the current run of the analytic was scheduled to start
- $analytic.AnalyticLastEndTime—The time the last run of the analytic ended
- $analytic.AnalyticStartTime—The time the current run of the analytic started (the difference between this and $analytic.AnalyticScheduledStartTime is the startup delay).
- If using OAuth authentication:
- The HTTP Poller source supports authentication using OAuth 2.0 for HTTP endpoints that follow the Client Credentials or Password grant types. These OAuth grant types follow a pattern that includes the configuration of an HTTP request to fetch an access token. These are in addition to the data request that is typically configured as part of any HTTP Poller source type.
- During configuration of the access token request, you are prompted to identify the access token from the response from the OAuth authorization server.
- When returning to the configuration of the data request, you must use the $accessToken global variable to define where the access token will be used. This is often in an Authorization custom header with the Bearer $accessToken value. $accessToken can also be used in the POST body, URL parameters, and in the URL itself (for example, https://<<company.domain>>/$accessToken/<<method>>?<<url params>>).
Parameters
| Parameter | Description | Data type |
|---|---|---|
URL | The full URL to the externally accessible HTTP endpoint. This parameter supports the $accessToken global variable. | String |
HTTP method | The HTTP method used for the HTTP endpoint, either GET or POST. If POST is chosen, you can specify a message body. | String |
POST body | The message body to be included when sending a POST HTTP request. This provides a selection between x-www-form-urlencoded and other generic text content types. It also sets the Content-type header. This parameter only applies when the HTTP Method parameter is set to POST. This parameter supports analytic and $accessToken global variables. | String |
Authentication | The type of authentication used to access the HTTP endpoint. Authentication options are None, Basic, Certificate, or OAuth. | String |
Username | The user name for accessing the HTTP endpoint. This parameter only applies when the Authentication parameter is set to Basic. | String |
Password | The password for accessing the HTTP endpoint. This property only applies when the Authentication parameter is set to Basic. | Password |
Certificate URL | The URL location of the client certificate for accessing the HTTP endpoint. This parameter only applies when the Authentication parameter is set to Certificate. | String |
Password | The client certificate password if one is set. This parameter only applies when the Authentication parameter is set to Certificate. | Password |
Custom headers | Optional custom headers to be included in the HTTP request that is sent to the specified HTTP endpoint. For example, custom headers can be used to specify a third-party API key or authorization header. This parameter supports analytic and $accessToken global variables. | Key-Value Pair |
URL parameters | Optional URL parameters to be included in the URL of the HTTP request that is sent to the specified HTTP endpoint. While parameters can also be set in the URL parameter, the URL parameters entered here will override any matching parameter supplied for the URL parameter. Parameters defined here will always be URL encoded. This parameter supports analytic and $accessToken global variables. | Key-Value Pair |
Considerations and limitations
- The HTTP endpoint must be externally accessible on the internet.
- Consider the following when
authoring a source with OAuth:
- Multiple requests to an authorization server and the associated data endpoint occur during authoring. Some APIs implement rate limiting for certain requests and this may affect authoring the feed.
- For security reasons, access token POST body must be reconfigured when editing the authentication parameters.