Importing Validated Data
There are five methods for importing validated data once it has been processed by OneSchema.
For files with fewer than 50k rows:
Type | Use | importConfig value |
---|---|---|
Frontend passthrough | For importing data via your frontend code. Typically the easiest to implement. | { type: "local" } |
Webhook JSON | For receiving batches of data via a webhook endpoint rather than requesting it via API. | { type: "webhook", key: webhookKey } |
For all file sizes:
Type | Use | importConfig value |
---|---|---|
Paginated JSON GET endpoint | For very large files and if you'd like to request the data in batches of a certain size. | { type: "local", metadataOnly: true } |
S3 URL download endpoint | For fetching entire CSV, Excel, or JSON files via our S3 download links. Also good for very large files. | { type: "local", metadataOnly: true } |
File upload to signed URL | For saving files to a given URL | { type: "file-upload", url: <YOUR_URL> } |
Frontend passthrough
Importing via your code can be useful if there's already a process in place to import data received via your frontend. This method is not recommended for files exceeding 50k rows.
To import via your code, do not specify a webhook key in your configuration for OneSchema. Then the results will be passed through as a parameter for the success handler of your implementation.
JS SDK
Data will be passed via a success event.
React SDK
Data will be passed via an onSuccess event.
Paginated JSON GET Endpoint
The Get Rows endpoint allows data to be requested with a starting row index and row count. This is best for when you'd like to request batched data from your backend.
S3 URL
The Export a CSV endpoint (also available in Excel and JSON flavors) provides a URL to OneSchema's S3 bucket. This is best for when you'd like to request the entire data file from your backend.
Importing via webhook
Importing via webhook allows you to process a large number of rows through pagination. OneSchema will send requests to an endpoint you host with metadata and up to 1,000 rows of processed data at a time. The data will come over in a JSON array, with each object representing a row of data, and each row containing key value pairs. See the schema of webhook payload metadata for more information about the format of the data. This method is not recommended for files exceeding 50k rows.
To configure a webhook to be used with OneSchema go to the webhooks tab on your developer dashboard. Then follow the instructions found on our Importer Webhooks documentation page.
Detailed information can be found on the Importer Webhook documentation page.
File upload to signed URL
Importing via file upload will send the result of an import session to a URL passed in to the SDK. For more information, see File Upload URL.
Updated 2 days ago