Developer Documentation
Platform Overview
Authentication
API Services
Overview Accounts Accounts: Associations Accounts: Metadata Accounts: Profile Appstore: Users Broker Distributions Broker Tours Consumers Consumers: Linked Agents Contacts Contacts: Activity Contacts: Export Contacts: Tags Contacts: Portal Accounts Developers: Identities Developers: Keys Developers: Authorizations Developers: Billing Summary Developers: Change History Developers: Domains Developers: News Feed Webhooks Developers: Roles Developers: Syndications Developers: Templates Developers: Usage Detail Developers: Usage Summary Devices Flexmls: Email Links Flexmls: Listing Meta Origins Flexmls: Listing Meta Translations Flexmls: Listing Meta Field List Translations Flexmls: Listing Reports Flexmls: Mapping Layers Flexmls: Mapping Shapegen IDX IDX Links Listing Carts Listing Carts: Portal/VOW Carts Incomplete Listings Incomplete Listings: Documents Incomplete Listings: Documents Metadata Incomplete Listings: Document Uploads Incomplete Listings: Floor Plans Incomplete Listings: FloPlans Incomplete Listings: Photos Incomplete Listings: Photos Metadata Incomplete Listings: Photo Uploads Incomplete Listings: Rooms Incomplete Listings: Tickets Incomplete Listings: Units Incomplete Listings: Videos Incomplete Listings: Videos Metadata Incomplete Listings: Virtual Tours Incomplete Listings: Virtual Tours Metadata Listings Listings: Clusters Listings: Documents Listings: Documents Metadata Listings: Floor Plans Listings: FloPlans Listings: Historical Listings: History Listings: Notes Listings: Search Parameters Listings: Open Houses Listings: Photos Listings: Photos Metadata Listings: Photo Uploads Listings: Document Uploads Listings: Rental Calendar Listings: Rooms Listings: Rules Listings: Tour of Homes Listings: Tickets Listings: Units Listings: Validation Listings: Videos Listings: Videos Metadata Listings: Virtual Tours Listings: Virtual Tours Metadata Listing Meta: Custom Fields Listing Meta: Custom Field Groups Listing Meta: Field Order Listing Meta: Field Relations Listing Meta: Property Types Listing Meta: Rooms Listing Meta: Standard Fields Listing Meta: Units Registered Listings Market Statistics News Feed News Feed: Curation News Feed: Events News Feed: Metadata News Feed: Restrictions News Feed: Schedule News Feed: Settings News Feed: Templates Open Houses Overlays Overlays: Shapes Portals Preferences Saved Searches Saved Searches: Provided Saved Searches: Restrictions Saved Searches: Tags Search Templates: Quick Searches Search Templates: Views Search Templates: Sorts Shared Links System Info System Info: Languages System Info: Search Templates
Supporting Documentation
Examples
RESO Web API
RETS
FloPlan
Terms of Use

RESO Web API Replication

If you'd like to replicate data but do not have an API key with replication permission, or if you find a problem with this outline, please contact api-support@sparkplatform.com.

  1. Replication Overview
  2. Step 1. Initial Download and Pagination
  3. Step 2. Updating Data
  4. Step 3. Purging Stale Data
  5. Optimizations
 

Replication Overview

Developers who wish to replicate data need to use an API key with a special replication access. API keys with permission to replicate must hit a separate RESO Web API replication endpoint: https://replication.sparkapi.com/Reso/OData. Requests from API keys with replication access will fail if they are made to the regular RESO Web API endpoint.

These keys are also granted a few additional privileges to make replicating data easier:

 

Replication API keys are otherwise identical to regular API keys, with their access to resources and data governed by the API key's role.

The following guidelines outline the process for replication using our RESO Web API. While the Property resource is used in these examples, the same general guidelines also apply to replicating data from other resources, such as the Member and Office resources.

There are three separate parts of the replication process: the initial data download, periodic requests for recently modified records, and purging stale data.

 

Step 1. Initial Download and Pagination

To kick off the initial download. you'll need to make a request to the Property endpoint. If you don't include a $filter parameter with the request, this will pull results from all the listings that are accessible with your API key, and you'll be able to paginate through the results.

https://replication.sparkapi.com/Reso/OData/Property?$top=1000&$count=true
 

The $top parameter determines the number of records you'll get back per page. The maximum value is 1000, but if you encounter timeouts or errors, try using a lower $top value. You may also wish to include a $count=true parameter, which will include an @odata.count field in the response that indicates the number of records matching your request.

Paginating through the full result set can be accomplished one of two ways:

  1. By using the @odata.nextLink values from the API response to make your next request (preferred)
  2. By incrementing the $skip parameter with each request

 

Using the @odata.nextLink

When a RESO Web API response doesn't return all matching results on a single page, the response will include an @odata.nextLink. That contains a RESO Web API request to return the next page of results, which is accomplished with a $skiptoken parameter set to the ListingKey of the last record on the current page.

"@odata.nextLink": "https://replication.sparkapi.com/Reso/OData/Property?%24skiptoken=20000612235426456684000000&%24top=1000",
 

Once you've finished processing the current page of results, you can use that value for your next API request. Once you've reached the last page of records, the @odata.nextLink won't be returned in the RESO Web API response.

Using the @odata.nextLink is the recommended method for replicating data. By default, results are ordered by ListingKey, so using the @odata.nextLinks ensures the order of the results is consistent as you paginate through them.

 

Incrementing the $skip parameter (use with caution)

An alternative method of paginating through results is using the $skip parameter. This specifies a number of records to skip before returning results. Use this method with caution; see below.

If this is your first request...

https://replication.sparkapi.com/Reso/OData/Property?$top=1000
 

...then your second request should include a $skip=1000 parameter...

https://replication.sparkapi.com/Reso/OData/Property?$top=1000&$skip=1000
 

...and so on. Increment your $skip value by whatever value you are using for the $top parameter. Note that this differs from the $skiptoken parameter, which takes a ListingKey as a value and skips all records ahead of that listing.

A warning about using the $skip parameter

Paginating using the $skip parameter can cause you to replicate the same listing more than once unless your process controls for it. For example, if one new listing is entered between when you pull the first and the second pages of results, the last record of the first page will now show up as the first record on the second page, i.e. it was the 1,000th listing when the first API request was made, but now it's the 1,001st listing when the second API request is made. Given this, your process must account for this possibility.

Step 2. Updating Data

Keeping your local data updated involves making periodic requests for recently updated listings, which is done by using the ModificationTimestamp field with the $filter parameter.

We recommend polling for updated records no more than once every 10 minutes but at least once every hour. This helps avoid unnecessary requests while keeping your data up-to-date.

To request listings that have been modified within a given window, use a $filter that includes both an lower bound - ModificationTimestamp gt <timestamp> - and an upper bound: ModificationTimestamp lt <timestamp>. Here's an example request:

https://replication.sparkapi.com/Reso/OData/Property?$top=1000&$filter=(ModificationTimestamp gt 2021-04-12T08:10:00Z and ModificationTimestamp lt 2021-04-12T08:20:00Z)
 

Note that the parentheses are required if you include additional filter terms.

Use the more recent timestamp from the previous request as the lower bound for the next request.

The same pagination methods outlined above should also be used to paginate over the updated records; the @odata.nextLink method is recommended.

If you use Media, Room, Unit and Open House data, we recommend updating these for each property as well.

Server-side caching

We use server-side caching, which stores for a short time (10min) indexes of listings that match a given filter. If you make the same request twice in a row, one minute apart, the cached index of matching listings dictates which listings will be returned in the second response. Caching is done on a per-API key basis; you'll never pull someone else's cached results list.

To avoid errors caused by caching, always use two timestamps to specify a start and end of your polling window. Polling only based on one ModificationTimestamp - e.g. _filter=ModificationTimestamp gt <timestamp> - can cause errors. For example, if the first request fails, and you successfully retry it a minute later, you will not receive any listings that were modified between your first and second requests.

 

Step 3. Purging Stale Data

Purging stale data is the final part of the replication process. It's common to lose access to listings that were previously accessible with your API key. This most frequently happens when a listing changes status, and with IDX API keys, though it can happen with API keys of any role. Consequently, it's necessary to have a process to check whether listings in your local data can still be accessed.

This process should be run no less than once daily, and MLS-specific rules may require more frequent updates. Failure to do so can result in continuing to display listings that are no longer active or eligible for display.

We recommend using a $select=ListingKey parameter to limit the data you get back, then paginate through the listings results as outlined above in step 1 (again, the @odata.nextLink method is recommended). Compare all ListingKey values returned with the values in your local data, and purge any records that do not match.

https://replication.sparkapi.com/Reso/OData/Property?$top=1000&$select=ListingKey

 

Optimizations

The following are recommended optimizations to ensure efficient data replication.

Custom fields: By default, you'll receive back the standard listing fields from the Property endpoint. Custom fields are MLS-specific fields, often locally oriented, and they are not included in the default RESO Web API response. You can include them by using an $expand=CustomFields parameter with your request. Only request custom fields back if you need this local data, as this significantly increases the size of the payload. See this guidance on how to use the metadata to translate custom field names from their encoded values to human-readable ones.

The $select parameter: Use the $select parameter with your requests to limit the fields you receive back to only those which you actually use. Doing so can cut out large amounts of data you don't need.

Media, Room, Unit and Open House data: If you use Media, Room, Unit or Open House data, you can request it be included along with the standard response from the Property resource, by including any of the Media, Room, Unit, and Open House expansions with your request:

https://replication.sparkapi.com/Reso/OData/Property?$top=1000&$expand=Media,Unit,Room,OpenHouse

Alternatively, you can retrieve this data with separate requests to property-specific endpoints (e.g. /Property/<ListingKey>/Media). If you do so, we strongly encourage you to make use of the PhotosChangeTimestamp, VideosChangeTimestamp and DocumentsChangeTimestamp fields to see if any media needs to be updated prior to polling for that data.