Batch Management

Batch ingestion in Dataleading refers to an ingestion job that reads a finite amount of data from your source and terminates when all rows have been loaded into Dataleading.

Batch use cases

The following are a few common use cases for batch ingestion in Dataleading:

• Load data into a table for the first time, such as when you need to migrate data from another database.
• Append new data into an existing table.
• Backfill older data after initializing streaming ingestion.

Batch ingestion sources

Dataleading supports several batch ingestion sources:

• Files: Upload files to the Dataleading staging area and ingest from them using the UI or API.
• Amazon S3: Read files from Amazon S3 buckets to ingest data into Dataleading using the UI or API.
• Azure Blob Storage: Read files from Azure Blob Storage containers to ingest data into Dataleading using the UI or API.