
Data Preparation for data import
on 09-13-2024 12:00 AM by SnapApp by BlueVector AI
319
Data preparation is a crucial step before importing data into a new system or database. It ensures data quality, consistency, and compatibility, preventing errors and issues during the import process. There are some steps and conditions for data preparation
- Data Formatting
To successfully import data into SnapApp, it is essential to structure the data in supported formats. SnapApp currently supports CSV and JSON formats for data imports. Follow these guidelines to prepare your data in the appropriate format:
CSV Preparation
- Ensure the first row contains the header, representing field names (e.g., Name, Email, Role).
- Data should be clean, with no missing values or extra spaces.
- Ensure consistent data types (e.g., numbers, dates) across rows for each field.
JSON Preparation
- Ensure that data is well-formed, with proper use of curly braces {} for objects and square brackets [] for arrays.
- All keys must be strings enclosed in double quotes.
- Ensure consistent data types for each key across records.
-
Data Validation
-
Verify Data Completeness: Ensure that all required data fields are present and populated, with no missing or incomplete records.
- Validate Data Integrity: Confirm that the data is consistent and accurate, adhering to predefined rules and relationships between fields.
- Assess Data Quality: Apply validation rules and constraints to identify and resolve any errors or anomalies, ensuring the data meets quality standards before import.
-
Field Mapping: Verify that the columns in your data map correctly to the target fields in SnapApp.
-
Creation of Key Column
A key column is used to store a unique identifier, such as a Case ID. Since each case can have multiple steps, it’s essential to also store a unique Step ID. While two different cases may have the same Step ID, the combination of Case ID and Step ID must be unique for each step. Additionally, to properly link the Case Step to its corresponding Case, SnapApp should include a ref field in the Case Step object that points to the Case object.
-
Data Integration
-
Data Fusion: Combine data from multiple heterogeneous sources, ensuring consistency and accuracy.
-
Data Reconciliation: Address and resolve conflicts or inconsistencies that may arise when merging data from different systems or formats.
-
Data Transformation
Convert data elements to the correct data types (e.g., text, numeric, date) as required by SnapApp’s fields. Additionally, verify that column names correspond to SnapApp’s expectations, regardless of case sensitivity.
-
Data Cleaning and Standardization
-
Data Standardization: Ensure all data is converted to a consistent format (e.g., standardizing date formats).
- Duplicate Removal: Identify and remove redundant records to maintain data integrity.
- Owner and Creator Fields: Replace names in the “Owner” and “Created By” columns with the corresponding user IDs for accuracy and consistency.
By following this steps data can be prepared for data import.
Thank you for following these steps to configure your SnapApp components effectively If you have any questions or need further assistance, please don’t hesitate to reach out to our support team. We’re here to help you make the most out of your SnapApp experience.
For support, email us at snapapp@bluevector.ai