ETL Overview
The ETL capability allows you to import data into Efimis. The process starts with you populating a defined set of tables in the ETL SQL schema. After the tables are filled with data, you provide a connection string to your Microsoft SQL Server database. This connection string grants the ETL system access to your database, enabling the data to be imported into the staging environment. Once the data is validated, you can promote it to the production environment.
1. Populating the ETL SQL schema
Section titled “1. Populating the ETL SQL schema”Our ETL SQL schema defines the tables and columns that the system recognises and expects to receive data in. These tables serve as the template for the data import process.
The schema consists of various tables, each with predefined columns that specify the data types and relationships. You will populate these tables with the relevant data from your external systems. Once the tables are filled, the data is ready for import.
Download the schema: Efimis-1.0.sql
See the full Schema reference.
2. Importing data into the staging environment
Section titled “2. Importing data into the staging environment”Endpoint: POST /etl/import
Submit an MSSQL connection string to a database populated with data based on the ETL SQL schema. The ETL system will access the provided database and import the data into the staging environment for validation.
Request payload:
{ "connectionString": "your-mssql-connection-string-here"}Response:
- Status code: 202 Accepted
{ "message": "Import request queued successfully.", "requestId": "unique-request-id"}The system connects to the provided database and begins importing the data into a provisioned tenant in the staging environment.
3. Promoting to production
Section titled “3. Promoting to production”Endpoint: POST /etl/promote-to-production
Once the import is successfully completed and validated in the staging environment, this endpoint promotes the data to the production environment.
Request payload:
{ "requestId": "unique-request-id"}Response:
{ "message": "Promotion to production started.", "requestId": "unique-request-id"}The status of this process can be monitored via GET /etl/import-status/{requestId}.