Coursedog

Submit a Ticket My Tickets
Welcome
Login  Sign up

How Merges Work in Coursedog

Table of Contents

Overview of Integration Merge Pipeline
How Merges Work
Merge Setting Configurability
Examples of Merges
Field-Specific Exceptions 
Term-Specific Integration Objects
Integration Pipeline Steps
Integration Dry Run

Integration Schedule - Real Time
Merge Reports
Related Articles

Overview of Integration Merge Pipeline

All merge types result from a specific REST API endpoint that Coursedog can access. Each integration merge follows a series of very specific steps. These steps are:

  1. We GET all the school data using a REST API

  2. We GET all Coursedog data

  3. We verify all school data is formatted correctly

  4. We verify all Coursedog data is formatted correctly

  5. We merge Coursedog data with SIS data (see notes below on how merging works)

    • If any merge conflicts exist, we store the merge conflicts as errors and stop the merge for all objects that had a conflict. 

    • For all other objects, they continue to the next phase.

  6. If a school has POST APIs set up, updates are sent from Coursedog to SIS

  7. We store results of the merge in Coursedog

  8. We save a merge report to be viewed in Coursedog

 

How Merges Work

When Coursedog merges our data with SIS data, we first get the following data sources:

 

  1. Coursedog Data

  2. SIS Data

  3. Output of the last successful merge (which we will call Original Data)

 

We use these three data sources to perform a three-way merge, a process which is best explained with an example. Let's say we have the following data:

{

original: 1,

sis: 1,

coursedog: 2

}

The three-way merge will compare the Coursedog value to the SIS value and to the original. It detects that:

 

  1. Coursedog is different from SIS

  2. Coursedog is different from original

  3. SIS and original equal each other

 

As a result, it will accept that a change was made on the Coursedog side, and store the change as an update.

 

However, let's say we have the following data:

{

original: 1,

sis: 3,

coursedog: 2

}

The three-way merge detects that:

  1. Coursedog is different from the SIS.

  2. Coursedog is different from the original.

  3. SIS is different from the original.

 

The only difference in this example and the previous one is that the SIS value has also changed. So even though the Coursedog value changed as well, our system now does not know which change is supposed to be accepted. Is the value supposed to be 3 or is it supposed to be 2? The System does not know, so it will return this as a merge conflict because both the SIS data and Coursedog data changed, and neither match with the original.

 

 

 

Merge Setting Configurability

For each merge type, custom settings can be configured to define the behavior of the merge and the logic for specific fields.

 

The default source of truth is used to determine the behavior of a merge, and how to handle merge conflicts.

  1. Manual Resolution – If there is a conflict, stop the integration for that object and return an error.

  2. Resolve as Coursedog – If there is a conflict, take the value from the Coursedog system as the source of truth. This option should only be used during bi-directional integration.

  3. Resolve as Institution – If there is a conflict, take the value from the SIS as the source of truth. This option should only be used during bi-directional integration.

  4. Always Coursedog –Take all values from Coursedog.

Always Institution – Take all values from SIS.

 

Examples of Merges

Merges Set to Resolve on a Nightly Bi-Directional Merge  

Day 1: Data is retrieved from the SIS and brought into Coursedog.

 

Day 2: Coursedog should be the source of truth for Sections. Updates are made in Coursedog.

  • Section data is retrieved from the SIS

  • A difference is detected (because we made changes in Coursedog)

  • Because the source of truth is Coursedog, we keep the Coursedog change and drop the SIS value

  • Because we are bi-directional, we send the Coursedog update to the SIS

  • We perform a backup of the current state of Coursedog and SIS data

 

Day 3: Coursedog should be the source of truth for Sections. Section updates are made in SIS.

  • Section data is retrieved from the SIS

  • A difference is detected (because we made changes in the SIS)

  • Because the source of truth is Coursedog, we keep the Coursedog change and drop the SIS change

  • Because we are bi-directional, we send the Coursedog update to the SIS (again)

  • We perform a backup of the current state of Coursedog and SIS data

 

Merges Set to Resolve on a One-Way Sync

Day 1: Data is retrieved from the SIS and brought into Coursedog.

 

Day 2: Coursedog should be the source of truth for Sections. Section updates are made in Coursedog.

  • Section data is retrieved from the SIS

  • A difference is detected (because we made changes in Coursedog)

  • Because the source of truth is Coursedog, we keep the Coursedog change and drop the SIS value

  • Because we are one-way, we do not send the Coursedog update to the SIS

  • We perform a backup of the current state of Coursedog and SIS data

 

Day 3: Coursedog should be the source of truth for Sections. Section updates are made in SIS.

  • Section data is retrieved from the SIS

  • A difference is detected (because we did not push our changes to the SIS, so the SIS still has the old value)

  • No conflict is detected (because we did not change our record today)

  • Because no conflict was detected, but the SIS data is different than our data (and therefore newer) we drop the Coursedog change and merge in the SIS change

  • Because we are one-way, we don’t send the Coursedog update to the SIS

  • We perform a backup of the current state of Coursedog and SIS data

 

Day 4: Coursedog appears to be missing changes made locally for the past 2 days.

 

Field-Specific Exceptions 

Field-specific exceptions are used to define custom logic for custom fields. This is most often used to identify data fields that don't exist in the SIS, and so as a result must be marked as "Always Coursedog"; otherwise, these data fields will be erased during any merge.

 

 

Term-Specific Integration Objects

For integration objects that are term specific (Sections, Courses, Relationships), you can specify the allowed terms for the nightly integration. If this field is populated, the nightly integrations will only execute for the specified terms. If no terms are specified, integrations will be run for the current active term and all future terms.

 

 

 

Integration Pipeline Steps

Each of the integration pipeline steps can be turned on/off at any point. However, the only step that we would suggest turning off at some points would be the "Should Coursedog send updates to the SIS?" step (see “Integration Dry Run” below to learn more about why you might want to sometimes set this step to “NO”).

 

 

Integration Dry Run

If you’d like to run a merge that captures all the POSTs that would be triggered without actually sending anything to your SIS, you will need to make the following changes to your settings under “Integration steps to execute”: 

  1. Set “Should Coursedog send updates to the SIS?” to “NO”. When this is set to “NO”, a sub-toggle will appear that will ask “Should updates be stored in merge report?” 

  2. Set “Should updates be stored in merge report” to “YES”. Updates will then be generated and included in the merge report but not sent to the SIS.

  3. Set “Should Coursedog save merged data” to “NO”.

Use Cases

An integration dry run is helpful in the following use cases:  

  1. You are adding a term to integration scope and want to figure out if anything would be sent back.

  2. You previously locked/removed a term from integration, are re-enabling it, and want to determine the scope of the first merge from Coursedog to the SIS to see how many changes were made in Coursedog before proceeding.

  3. You made a change to a template, merge settings, or other configuration and want to determine the impact to the integration without sending changes to the SIS.

 

Integration Schedule - Real Time

Overview

  • We have the ability to turn on a real-time integration for some of our integrations. This means when a user saves changes to an existing section, the real-time sync process will be initiated. 

  • Only Coursedog administrators can edit these settings.

  • When real-time syncs are turned on, you can specify how often the data should be synced in aggregate. This is present because a true "real-time" sync might overload the school's system, so Coursedog will actually queue up updates, and then execute them in bulk after a preset delay, which is defined by the "How Often Should Changed Data By Synced" setting - which defaults to 30 minutes.

  • You can learn more about suggested settings for real-time merges – and how they work – here

 

Merge Reports

Overview

After each merge, a merge report will be stored, which will show all steps of the merge that executed, along with any errors that resulted. Merge reports are found in the Admin Panel.

 

 

Each merge report will provide information on each step that was executed.

 

Merge Errors Overview

Within each step, you can view details about the step along with any errors:

 

If there are any merge conflicts, those errors will exist in the "Merge SIS And Coursedog Data" step. A sample merge error is as follows:

{

"COM1075131409": [

   {

     "kind": "E",

     "lhs": 4,

     "rhs": 0,

     "path": [

       "maxEnrollment"

     ]

   }

 ]

}

 

Merge Error Causes

There are four fields for a merge conflict to explain what caused the issue:

  1. Kind – The options are N,D,E

    1. N - New (a field was added to the SIS data)

    2. D - Deleted (a field was deleted from the SIS data)

    3. E - Edited (a field was edited in both SIS and Coursedog data)

  2. LHS – This stands for Left Hand Side, which is the Coursedog data.

  3. RHS – This stands for Right Hand Side, which is the SIS data.

  4. Path – This refers to the path to access the data in the data object.

  

Related Articles

Did you find it helpful? Yes No

Send feedback
Sorry we couldn't be helpful. Help us improve this article with your feedback.