Skip to main content

Append Data

Append data endpoint allows you to upload your historical data to the platform. You can upload data into an empty collection, or update an existing collection with new data. If you do not have a collection yet, create one following the instructions in the Create Collection section.

Each time new data is uploaded, it is automatically merged with the existing dataset (if any!) of that type.

tip

Sales Forecasting is a data-driven tool: to get the most accurate predictions, it is important to keep the historical data in your collection up-to-date. Do not forget to upload your data periodically!

The data to be uploaded must be in csv format, with specific columns depending on the type of data you are uploading. For more information on the schema of the data, please refer to the Data Schemas section.

At each append request, the platform validates the data to ensure they conform to the schema. If the data do not conform, the append request is rejected with the validation errors that caused the non-conformity.

The procedure to upload data to the platform is as follows:

  1. send a POST request to the append data endpoint with the csv file containing the data to be uploaded;
  2. retrieve the request id from the response;
  3. poll the platform periodically to check the status of the append request to ensure that the data upload is successful.

1. Send a POST request​

To append data, you need to send a POST request to the append data endpoint:

/api/collection/{collectionId}/data/{dataType}/append

This endpoint requires the following parameters:

  • collectionId: the identifier of the collection into which you want to upload the data;
  • dataType: the type of data you want to append. Available values are: `deals`.

In the request body, you need to specify as a form-data field:

  • file, the csv file containing the data to be uploaded;
  • separator, the separator of the csv file (e.g. ",").

The request must also include an header with the authorization token.

An example of the request to append deals data to the collection my-collection-id is shown below. In the example, the csv file containing the data we want to upload is located at my-data-path/my-deals-data.csv and has a structure similar to the following:

id,name,description,creationDatetime,addDatetime,closedDatetime,expectedClosureDatetime,status,stage,currentStageStartDatetime,value,numberOfSentEmails,numberOfEmailOpens,numberOfReceivedEmails,numberOfMeetings,lastInteractionDatetime,accountId,accountAverageResponseTime,companyId,companyNumberOfEmployees,companyRevenue,companyIndustry,companyType,companyDescription,companyTechnologies,companyCity,companyCountry,companyIsCustomer,contactId,contactNumberOfConversions,contactEmailDomainType,contactEmailOptedOut,contactAverageResponseTime,lastUpdateDatetime,lastSyncedAt
1,Customer A - annual supply,Partnership agreement,2024-05-01T08:00:00,2024-05-01T08:00:00,2024-06-10T09:15:30,2024-06-10T09:15:30,closed_won,Negotiation,2024-06-10T09:15:30,1000.00,5,20,5,10,2024-06-18T12:30:45,x,5,123,50,1000060.50,Technology,private,Leading AI solutions...,AI;Machine Learning;Web,Oslo,Norway,true,456,5,company,true,5,2024-06-18T12:30:45,2024-06-18T12:30:45
2,Customer B - software license,Subscription renewal,2024-04-15T10:30:00,2024-04-15T10:30:00,2024-05-20T14:45:00,2024-05-25T00:00:00,closed_lost,Proposal,2024-05-10T12:15:00,5000.00,10,35,8,6,2024-05-18T16:00:30,y,8,234,200,5000000.00,Finance,public,Financial solutions provider,Cloud;Big Data,New York,USA,true,789,3,company,false,4,2024-05-20T14:45:00,2024-06-18T12:30:45
3,Customer C - consulting services,Strategic partnership,2024-03-20T09:00:00,2024-03-21T08:00:00,,2024-07-01T12:00:00,open,Discovery,2024-06-01T10:00:00,25000.00,2,10,3,2,2024-06-15T15:45:20,z,6,345,1000,25000000.00,Consulting,private,Top consulting firm,Analytics;Cloud,London,UK,false,567,7,free_provider,true,7,2024-06-15T15:45:20,2024-06-18T12:30:45
4,Customer D - IT support,Long-term contract,2024-02-10T14:45:00,2024-02-11T09:30:00,2024-03-30T18:20:00,2024-04-01T00:00:00,closed_won,Final Review,2024-03-20T16:45:00,15000.00,7,25,4,5,2024-03-30T18:20:00,w,4,678,500,100000000.00,IT Services,private,Global IT solutions,DevOps;Security;Cloud,Berlin,Germany,true,890,12,company,false,6,2024-03-30T18:20:00,2024-06-18T12:30:45
5,Customer E - hardware procurement,Procurement agreement,2024-01-05T11:20:00,2024-01-06T14:00:00,,2024-08-15T10:00:00,open,Qualification,2024-06-10T09:00:00,7500.00,3,15,6,3,2024-06-12T11:30:00,v,9,910,120,7500000.00,Manufacturing,public,Leader in hardware manufacturing,Robotics;Automation;IoT,Tokyo,Japan,false,321,2,company,true,8,2024-06-12T11:30:00,2024-06-18T12:30:45
curl --location'https://nostradamus.u-hopper.com/api/collection/{my-collection-id}/data/deals/append' \
--header 'Authorization: Token my-secret-token' \
--form 'file=@"my-data-path/my-deals-data.csv"' \
--form 'separator=","'

2. Retrieve the append request id​

If the append request is successful (status code 202), the response will follow the UploadRequest schema defined in the OpenApi documentation. The id field in the response will contain the identifier of the append request you just sent, and the status field will be set to queued. You will need the identifier to check the status of the append request.

If the request is unsuccessful, the response will contain the error message.

An example of the response is shown below:

{
    "id": "my-upload-request-id",
    "collectionId": "my-collection-id",
    "dataType": "deals",
    "separator": ",",
    "dumpDatetime": "2024-08-20T12:53:24.134179Z",
    "status": "queued",
    "message": null,
    "validationErrors": null
}

3. Poll the platform to check the status of the append request​

To retrieve the status of an append request, you can send a GET request to the following endpoint, replacing my-upload-request-id with the id of the append request:

/api/collection/{collectionId}/data/deals/append/{my-upload-request-id}

Here is an example of how to check the status of the append request with id my-upload-request-id:

curl --location 'https://nostradamus.u-hopper.com/api/collection/{my-collection-id}/data/deals/append/{my-upload-request-id}' \
--header 'Authorization: Token my-secret-token'

The response (if the request succeeded, i.e. status code 200) will follow the UploadRequest schema defined in the OpenApi documentation. The status field in the response will indicate the status of the request.

The status can be one of the following values: queued, in_progress, completed, error. In particular:

  • a status queued indicates that the request has been received by the platform and is waiting to be processed;
  • a status in_progress indicates that the request is being processed;
  • a status completed indicates that the validation process is successfully terminated and your data upload has been accepted by the platform;
  • a status error indicates that the validation process terminated with errors; the failure motivation will appear in the field validationErrors of the response.

An example of the response is shown below:

{
    "id": "my-upload-request-id",
    "collectionId": "my-collection-id",
    "dataType": "deals",
    "separator": ",",
    "dumpDatetime": "2024-08-20T12:53:24.134179Z",
    "status": "completed",
    "message": null,
    "validationErrors": null
}

You may need to poll the platform periodically to check the status of your append request. This ensures that you receive timely updates on the progress and outcome of your request.

warning

Ensure that your upload request is successful (i.e. status: completed) before submitting an invocation request. Otherwise, the predictions or analyses you request will either be inaccurate (if the collection already contains data) or the invocation request will fail (if the collection does not have any data uploaded correctly).