Table of Contents
Coursedog executes a complex system of integration functions every night to sync the Coursedog system with your SIS.
We both GET data out of the SIS and POST updates back through REST API endpoints.
Every school for which an SIS integration has been set up will have their integration functions executed on a nightly basis.
Start time for nightly merges can vary and is impacted by daylight savings, but generally speaking they begin sometime between 1:30-2:30 a.m. ET.
When a nightly merge ends depends on a variety of factors, including the time it starts, whether or not the institution or SIS is undergoing maintenance/backup, whether or not your institution consists of multiple schools, and whether your nightly integration queue is not progressing due to issues in other instances.
Merges are executed in a specific order as defined by the dependencies between different elements (sections before courses, etc.). When a single institution has multiple schools, execution functions are run in batches, so the order in which the function starts is not necessarily the order in which it is completed.
PATH: Admin > Merge Settings
Actual settings can vary by SIS and institution. What you see below is a general overview; please work with your Customer Success representative to confirm the correct settings for you.
The first setting is "Merge Types" which lists all entities that are being merged. You can enable (set to “Yes”) or disable (set to “No”) the integration for each of these data objects.
Nightly Integration Schedule
Users can define the "nightly integration schedule".
When set to “Yes”, merges will run nightly for all enabled entities.
When set to “No”, nightly merges will NOT be run.
The "type-specific" settings define the default source of truth for the selected entity, along with field-specific exceptions.
The “Source of truth” defines the logic for handling merge conflicts.
Click through all of the different object types/entities at top to configure source-of-truth settings for each.
Source of Truth Options
If there is a conflict, stop the integration for that object and return an error.
Resolve as Coursedog
If there is a conflict, take the value from the Coursedog system as the source of truth.
Resolve as Institution
If there is a conflict, take the value from the SIS as the source of truth.
Take all values from Coursedog (this is the same as not having the integration on at all).
Take all values from SIS.
Additionally, for each object, there can be field-specific exceptions.
The main purpose of these exceptions is that the Coursedog system often contains much more functionality than an SIS, and so there can be fields that exist in Coursedog that don't exist in the SIS.
To ensure these fields don't get wiped out in the nightly merge, we can add these exceptions to the “field” input and then set the source of truth for these exceptions as "Always Coursedog".
In other words, if the default source of truth for “Rooms” is “Always Institution” – but there are some room fields that have a different source of truth – you can add those fields as “Field-specific exceptions”.
Every night when the integration is run, the following occurs:
We first GET all the school data and run it through our internal data verifier. If there is a verification error, we return an error and stop the merge.
We then GET all of our data.
We GET a copy of the previous night's data (stored in our data backups) which represents the last successful merge.
We execute a three-way merge because we use three data sources to execute the merge. The three sources being: the institution’s data, Coursedog’s data, and yesterday’s version of the data.
After executing the merge, if any merge conflicts exist, we store the merge conflicts as errors and stop the merge for all objects that had a conflict. For all other objects, they continue to the next phase.
If the institution has POST APIs set up, the POST APIs are executed.
After the POST APIs, we GET the institution’s data again because the SIS sometimes adds IDs or other fields to the data, and we want to make sure we have those updated IDs.
If there were no issues with the POST APIs, we take the new version of all objects and store it in our database.
We send out a merge report email.
Reasons For Errors
The following issues can lead to errors with a nightly merge:
SIS servers down
N2N API middleware down
GET endpoint returning multiple objects with the same ID
POST API Error
Identifying Errors - Merge Reports
You can review merge errors in multiple places: via email notifications; the admin panel; on the Scheduling reports dashboard; and in the section editor. You can learn more about managing integration errors here.