Table of Contents
Overview
Merge Order
Merge Settings
Integration Pipeline
Reasons For Errors
Identifying Errors - Merge Reports
Related Articles
Overview
Coursedog executes a complex system of integration functions every night to sync the Coursedog system with your SIS.
We both GET data out of the SIS and POST updates back through REST API endpoints.
Every school for which an SIS integration has been set up will have their integration functions executed on a nightly basis.
Start time for nightly merges can vary and is impacted by daylight savings, but generally speaking they begin sometime between 1:30-2:30 a.m. ET.
When a nightly merge ends depends on a variety of factors, including the time it starts, whether or not the institution or SIS is undergoing maintenance/backup, whether or not your institution consists of multiple schools, and whether your nightly integration queue is not progressing due to issues in other instances.
Merge Order
Merges are executed in a specific order as defined by the dependencies between different elements (sections before courses, etc.). When a single institution has multiple schools, execution functions are run in batches, so the order in which the function starts is not necessarily the order in which it is completed.
Merge Settings
Overview | Merge Types | Nightly Integration Schedule | Type-Specific Settings
Overview
PATH: Admin > Merge Settings
Actual settings can vary by SIS and institution. What you see below is a general overview; please work with your Customer Success representative to confirm the correct settings for you.
Merge Types
The first setting is "Merge Types" which lists all entities that are being merged. You can enable (set to “Yes”) or disable (set to “No”) the integration for each of these data objects.
Nightly Integration Schedule
Users can define the "nightly integration schedule".
When set to “Yes”, merges will run nightly for all enabled entities.
When set to “No”, nightly merges will NOT be run.
Type-Specific Settings
Overview
The "type-specific" settings define the default source of truth for the selected entity, along with field-specific exceptions.
The “Source of truth” defines the logic for handling merge conflicts.
Click through all of the different object types/entities at top to configure source-of-truth settings for each.
Source of Truth Options
Manual Resolution
If there is a conflict, stop the integration for that object and return an error.
Resolve as Coursedog
If there is a conflict, take the value from the Coursedog system as the source of truth.
Resolve as Institution
If there is a conflict, take the value from the SIS as the source of truth.
Always Coursedog
Take all values from Coursedog (this is the same as not having the integration on at all).
Always Institution
Take all values from SIS.
Field-Specific Exceptions
Additionally, for each object, there can be field-specific exceptions.
The main purpose of these exceptions is that the Coursedog system often contains much more functionality than an SIS, and so there can be fields that exist in Coursedog that don't exist in the SIS.
To ensure these fields don't get wiped out in the nightly merge, we can add these exceptions to the “field” input and then set the source of truth for these exceptions as "Always Coursedog".
In other words, if the default source of truth for “Rooms” is “Always Institution” – but there are some room fields that have a different source of truth – you can add those fields as “Field-specific exceptions”.
Integration Pipeline
Every night when the integration is run, the following occurs:
We first GET all the school data and run it through our internal data verifier. If there is a verification error, we return an error and stop the merge.
We then GET all of our data.
We GET a copy of the previous night's data (stored in our data backups) which represents the last successful merge.
We execute a three-way merge because we use three data sources to execute the merge. The three sources being: the institution’s data, Coursedog’s data, and yesterday’s version of the data.
After executing the merge, if any merge conflicts exist, we store the merge conflicts as errors and stop the merge for all objects that had a conflict. For all other objects, they continue to the next phase.
If the institution has POST APIs set up, the POST APIs are executed.
After the POST APIs, we GET the institution’s data again because the SIS sometimes adds IDs or other fields to the data, and we want to make sure we have those updated IDs.
If there were no issues with the POST APIs, we take the new version of all objects and store it in our database.
We send out a merge report email.
Reasons For Errors
The following issues can lead to errors with a nightly merge:
SIS servers down
N2N API middleware down
GET endpoint returning multiple objects with the same ID
Verification Error
Merge Conflict
POST API Error
Identifying Errors - Merge Reports
You can review merge errors in multiple places: via email notifications; the admin panel; on the Scheduling reports dashboard; and in the section editor. You can learn more about managing integration errors here.