CoreView imports data from your Microsoft 365 tenant at different times: full and delta import.
Ways to import data from Microsoft
Microsoft offers various methods to import data from their environment:
Microsoft Graph
The Microsoft Graph REST API is a comprehensive endpoint that enables developers to access and interact with a diverse range of Microsoft services, including Microsoft 365, Azure Active Directory, OneDrive, SharePoint, Outlook, Teams, and others, through RESTful API calls. The Graph API offers the capability for real-time updates, allowing developers to receive notifications for changes happening in the data.
Microsoft Services API
Microsoft also provides APIs that can be accessed directly, without using PowerShell. These APIs are typically referred to by specific names based on the service or functionality they provide, such as Outlook API, Microsoft Teams API, SharePoint API, and more. They enable developers to programmatically interact with the corresponding services. These APIs are commonly RESTful APIs, accessible through HTTPS requests.
PowerShell with service accounts
Microsoft provides PowerShell modules and cmdlets that allow programmatic access to perform administrative tasks and operations within Microsoft 365.
How the data import works
CoreView retrieves various data from Microsoft 365 via API, such as user information, groups, and licenses from various providers: Azure AD, Teams, SharePoint, and more.
Throughout the day, we perform delta imports to update specific properties whenever changes occur. Essentially, we continuously request information from Microsoft to check for any updates, and in case changes have indeed occurred, Microsoft notifies CoreView accordingly.
However, to ensure complete alignment between Microsoft and CoreView, a full import takes place once a day. This comprehensive import guarantees that all data within the tenant remains synchronized and accurate.
Full and delta imports, have their own unique characteristics and timetable:
Delta import
Thanks to Microsoft Graph, Microsoft has implemented the mechanism of “delta changes” where only values related to specific properties are updated during an import in near real-time. For example:
| Property | Values |
|---|---|
| userPrincipalName | user1@domain.com, user2@domain.com, user3@domain.com, … |
However, there are other properties that do not adhere to this mechanism, and their values are only updated during a full import.
- A delta import is only performed if values associated with properties that adhere to the “delta changes” mechanism have been changed or updated since the last successful delta import.
- This import is performed every 5-10 minutes. providing a near real-time update. For example, if you create a new group from the Microsoft 365 Admin portal, that group will be available in CoreView after 5-10 minutes.
- We cannot guarantee the duration it takes to complete. The time it takes for a full import may be subject to any data throttling Microsoft may have in place at that time.
- This import utilizes only Microsoft Graph.
If the tenant registered into CoreView platform has not been granted access to CoreView API Integration application, delta import is not available.
List of properties that adhere to delta changes
The following Microsoft 365 properties are those whose values are updated when a partial import is performed. A partial import will only be performed if values associated with any of the following properties changed/updated since the last successful partial import.
Azure AD
Properties whose values are updated for online users when a partial import is performed.
accountEnabled
assignedPlans
businessPhones
city *
companyName *
country *
createdDateTime
customAttributes
department *
displayName
employeeId
givenName
id
imAddresses
jobTitle
mail
mailNickname
mobilePhoneonPremisesLastSyncDateTime
onPremisesSyncEnabled
postalCode *
preferredLanguage
proxyAddresses
recipientType (retrieved from Exchange)
recipientTypeDetails (retrieved from Exchange)
state *
streetAddress
surname
usageLocation *
userPrincipalName
provisionedPlans
officeLocation
onPremisesDistinguishedName
CustomAttributes (onPremisesExtensionAttributes)
onPremisesImmutableId
onPremisesSamAccountName
Licenses
Newly acquired licenses are updated during partial import so that if a new license is added, it will become available in near real time.
SkuId
SkuPartNumber
ConsumedUnits
PrepaidUnits
ServicePlans
Groups
Properties whose values are updated for groups and members when a partial import is performed.
Description
DisplayName
Mail
MailEnabled
MailNickName
OnPremisesLastSyncDateTime
OnPremisesSecurityIdentifier
OnPremisesSyncEnabled
OnPremisesDomainName
OnPremisesNetBiosName
OnPremisesSamAccountName
ProxyAddresses
SecurityEnabled
Visibility
CreatedDateTime
RenewedDateTime
ResourceProvisioningOptions
Members
Owners **
For more information and details please check Microsoft documentation here: https://docs.microsoft.com/en-us/graph/api/resources/group?view=graph-rest-1.0
* Properties available for use as a License Pool Filter Criteria , which is recalculated every 30 minutes.
** In case only the owner is modified, then this change won't be captured by the partial import but imported with the full import. In addition to the owners, any other properties can be changed (add a member and remove it). This way the partial import will import the change.
Some properties of entities or functionalities (users, groups, etc.) may adhere to the delta changes mechanism, while others may not. For example, when it comes to user information, some properties adhere to this mechanism while others do not.
Full import
- The complete data is imported from all the providers, including SharePoint, Teams App, Teams Voice, Exchange, and more.
- It utilizes Microsoft Graph, Microsoft Services API, and PowerShell with service accounts.
- Performed once a day, approximately every twenty-four hours. In other words, the data is updated at least once within a twenty-four hour period.
- This import is typically performed after regular working hours, taking into account Microsoft's data management practices. We strive to schedule the import during nighttime across different data centers, but the exact timing is ultimately determined by Microsoft data management practices.. Therefore, we cannot provide a specific time guarantee for when the import will run.
- We cannot guarantee the duration it takes to complete. The time it takes for a full import may be subject to any data throttling Microsoft may have in place at that time.
- The import process requires a considerable amount of time, which is why it cannot run more than once a day.
- There is no way to force a full import.
During the full import, a data that has been imported during a delta import via Microsoft Graph will not be overridden by the data retrieved with different kind of APIs (with or without PowerShell).
Questions
Why can I see changes on CoreView but not in a specific environment, like Azure AD?
All the data on the Microsoft 365 Admin portal is exposed through the Graph API. Therefore, it is possible that if, for example, you create a new group on the Microsoft 365 portal, it will be visible in CoreView but not yet in AzureAD.
A property can exist on different providers. How can I determine the source of a specific property?
We know that the same property can exist on different providers. Based on experience and testing, we know that importing a specific property from one provider is preferable over another.
How does the import for Endpoint Management work?
Endpoint management only supports full imports at the moment. Unfortunately, Microsoft does not currently offer the capability for partial imports within Endpoint management.