Data API

One of the core design principles of the PDA has been its ability to store any structured data. In v2 of the API this ability was overhauled to simplify data input and minimise setup required before data can be saved.
Therefore, plain Data APIs consume any JSON:
individual JSON values are saved as-is in a data record
a data record is a natural grouping of data, expected to relate to the way data is grouped at the source (e.g. all data together with metadata of a calendar event)
records posted as an array is saved into separate records
records can be further grouped to form complex data relationships (e.g. calendar events linked to a separate set of event attendees, each stored as a record), either at the time of saving the data or afterwards by referring to individual records by their IDs
deleting of data records is supported by accepting a list of previously saved record IDs
The v2 API loosely replaces the notion of data source and table from the previous API version with namespace and endpoint, where:
Each application that is authorised to write any data into a PDA, can do so only within one or more namespacesdesignated for it, such as Rumpel-only writing to the rumpel namespace. Applications are not authorised to write into each other's namespaces.
endpoints are defined by the application developer as required. Data is grouped under a combination of a namespaceand an endpoint where the two together identify a set of data accessible via a URL path /api/v2.6/data/namespace/endpoint.
An example would be /api/v2.6/data/rumpel/locations for all locations saved into a PDA by the Rumpel app.
Data can thus be stored and retrieved using permitted credentials for the individual endpoints.
You will need to acquire an access_token before calling any data APIs. Please refer to the guide on HAT login for details on authentication.
Data Input
Data input can happen in one of two ways:
1.Saving one or more homogeneous data records into a single endpoint
2.Saving multiple data records that can be potentially heterogeneous, related, or both
Saving homogeneous data records
A data entry request for a single endpoint (/hat/locations), with multiple records inserted in one go. The data provided in the example is merely an illustration – the PDA will accept any well-formed JSON data. Note the request is authenticated with an ACCESS_TOKEN obtained previously and the token has been issued for the namespace of hat specifically.
{
  "request": {
    "url": "https://{{page.hat}}/api/v2.6/data/hat/locations",
    "method": "POST",
    "header": [
      {
        "key": "Content-Type",
        "value": "application/json",
        "description": ""
      },
      {
        "key": "X-Auth-Token",
        "value": "ACCESS_TOKEN",
        "description": ""
      }
    ],
    "body": {
      "mode": "raw",
      "raw": [
        {
          "id": 85998,
          "name": "record 1",
          "lastUpdated": "2017-04-23T16:11:17.000Z",
          "data": {
            "locations": {
              "latitude": "51.671358277138",
              "longitude": "0.101014673709963",
              "accuracy": "10.0",
              "timestamp": "2017-04-23T16:11:17+0000"
            }
          }
        },
        {
          "id": 85997,
          "name": "record 3",
          "lastUpdated": "2017-04-23T16:18:04.000Z",
          "data": {
            "locations": {
              "latitude": "51.6658257133844",
              "longitude": "0.080477950927866",
              "accuracy": "1414.0",
              "timestamp": "2017-04-23T16:18:04+0000"
            }
          }
        },
        {
          "id": 85996,
          "name": "record 2",
          "lastUpdated": "2017-04-23T16:12:58.000Z",
          "data": {
            "locations": {
              "latitude": "51.674001392439",
              "longitude": "0.100905202634514",
              "accuracy": "1414.0",
              "timestamp": "2017-04-23T16:12:58+0000"
            }
          }
        }
      ]
    },
    "description": "Storing a ist of data records (locations in the example) in one go"
  }
}
If the data gets saved, the response includes the full data, wrapped in the Record data structure:
endpoint – field that includes the specific endpoint the record was assigned to
recordId – UUID of the data record
data – the data JSON structure exactly as submitted
{
  "response": [
    {
      "name": "Data saved",
      "status": "Created",
      "code": 201,
      "header": [
        {
          "key": "Content-Type",
          "value": "application/json",
          "name": "Content-Type",
          "description": "The mime type of this content"
        }
      ],
      "body": [
        {
          "endpoint": "locations",
          "recordId": "f8c011d1-98f0-4e5f-89a6-64b949724eba",
          "data": {
            "id": 185998,
            "data": {
              "locations": {
                "accuracy": "10.0",
                "latitude": "51.671358277138",
                "longitude": "0.101014673709963",
                "timestamp": "2017-04-23T16:11:17+0000"
              }
            },
            "name": "record 1",
            "lastUpdated": "2017-04-23T16:11:17.000Z"
          },
          "links": []
        },
        {
          "endpoint": "locations",
          "recordId": "0a79bef4-9a8e-4d19-937d-9a759fc2981b",
          "data": {
            "id": 185997,
            "data": {
              "locations": {
                "accuracy": "1414.0",
                "latitude": "51.6658257133844",
                "longitude": "0.080477950927866",
                "timestamp": "2017-04-23T16:18:04+0000"
              }
            },
            "name": "record 3",
            "lastUpdated": "2017-04-23T16:18:04.000Z"
          },
          "links": []
        },
        {
          "endpoint": "locations",
          "recordId": "b6ebcdc3-235e-4e8c-834c-eec08beb55cd",
          "data": {
            "id": 185996,
            "data": {
              "locations": {
                "accuracy": "1414.0",
                "latitude": "51.674001392439",
                "longitude": "0.100905202634514",
                "timestamp": "2017-04-23T16:12:58+0000"
              }
            },
            "name": "record 2",
            "lastUpdated": "2017-04-23T16:12:58.000Z"
          },
          "links": []
        }
      ]
    }
  ]
}
If there are any problems with your JSON (e.g. it is malformed), you will receive a Bad Request response code with the details of the error in the body of the response. If, for example, such data (as decided by its hash) already exists in the PDA, none of the data in the request will be saved and you will receive a message indicating so:
{
  "response": [
    {
      "name": "Error: duplicate data received",
      "status": "Bad Request",
      "code": 400,
      "header": [
        {
          "key": "Content-Type",
          "value": "application/json",
          "name": "Content-Type",
          "description": ""
        }
      ],
      "body": {
        "message": "Duplicate Data",
        "cause": "Could not insert data — Duplicate data"
      }
    }
  ]
}
Note: all data entry operations are transactional, i.e. insertion of all data provided in the request succeeds or the whole request fails and no data gets inserted.
To illustrate the PDA's acceptance of any valid JSON data, here is another example, saving very different profile data in the same namespace, but a different endpoint:
{
  "request": {
    "url": "https://postman.hubat.net/api/v2.6/data/rumpel/profile",
    "method": "POST",
    "header": [
      {
        "key": "Content-Type",
        "value": "application/json",
        "description": ""
      },
      {
        "key": "X-Auth-Token",
        "value": "ACCESS_TOKEN",
        "description": ""
      }
    ],
    "body": {
      "mode": "raw",
      "raw": {
        "profile": {
          "website": {
            "link": "https://example.com",
            "private": "false"
          },
          "nick": {
            "private": "true",
            "name": ""
          },
          "primary_email": {
            "value": "[email protected]",
            "private": "false"
          },
          "private": "false",
          "youtube": {
            "link": "",
            "private": "true"
          },
          "address_global": {
            "city": "London",
            "county": "",
            "country": "UK",
            "private": "true"
          },
          "age": {
            "group": "",
            "private": "true"
          },
          "personal": {
            "first_name": "",
            "private": "false",
            "preferred_name": "Test",
            "last_name": "User",
            "middle_name": "",
            "title": ""
          },
          "blog": {
            "link": "",
            "private": "false"
          },
          "facebook": {
            "link": "",
            "private": "false"
          },
          "address_details": {
            "no": "",
            "street": "",
            "private": "false",
            "postcode": ""
          },
          "emergency_contact": {
            "first_name": "",
            "private": "true",
            "relationship": "",
            "last_name": "",
            "mobile": ""
          },
          "alternative_email": {
            "private": "true",
            "value": ""
          },
          "fb_profile_photo": {
            "private": "false"
          },
          "twitter": {
            "link": "",
            "private": "false"
          },
          "about": {
            "body": "A short bio about me shown on my PDA",
            "private": "false",
            "title": "Me the Test User"
          },
          "mobile": {
            "no": "",
            "private": "true"
          },
          "gender": {
            "type": "",
            "private": "true"
          }
        }
      }
    },
    "description": "Using Rumpel profile as an exmaple, can save the whole object as a single API call"
  }
}
Saving related data records
Heterogeneous data records can be inserted simultaneously using the batch-data API endpoint. One example where such data insertion is useful is PDA notes, where a text-based note is saved with an attached "nudge" – a time-based reminder connected to the note. It is important to keep notes and nudges logically separate, because both can be used independently, e.g. a nudge can be attached to certain locations, photos, posts...
{
  "request": {
    "url": "https://postman.hubat.net/api/v2.6/data-batch",
    "method": "POST",
    "header": [
      {
        "key": "Content-Type",
        "value": "application/json",
        "description": ""
      },
      {
        "key": "X-Auth-Token",
        "value": "ACCESS_TOKEN",
        "description": ""
      }
    ],
    "body": {
      "mode": "raw",
      "raw": [
        {
          "endpoint": "rumpel/notable",
          "data": {
            "id": 84995,
            "lastUpdated": "2017-04-23T14:21:51+01:00",
            "data": {
              "notablesv1": {
                "authorv1": {
                  "phata": "postman.hubat.net"
                },
                "created_time": "2017-04-10T14:19:59:+01:00",
                "shared": "true",
                "shared_on": "twitter",
                "message": "Showcasing the new HAT APIs",
                "public_until": "2017-05-11T14:21:54+01:00",
                "updated_time": "2017-04-23T14:21:58+01:00",
                "kind": "note"
              }
            }
          },
          "links": [
            {
              "endpoint": "rumpel/nudge",
              "data": {
                "type": "time",
                "nudge": "Share APIs with the world",
                "time": "2017-04-30T14:22:52+01:00"
              }
            }
          ]
        }
      ]
    },
    "description": "Sending multiple records to the HAT simultaneously, potentially for separate 'endpoints' as well as linked, all in one call"
  }
}
The response will include the main data record with the note as well as a separate data record with the nudge, linked to the main one.
Data records do not necessarily need to be linked up when being inserted. This can be done later by referring to the records using their IDs:
{
  "request": {
    "url": "https://postman.hubat.net/api/v2.6/data-link?records=6dcff611-45cb-49a9-82ca-318b9e5a3c17",
    "method": "POST",
    "header": [
      {
        "key": "Content-Type",
        "value": "application/json",
        "description": ""
      },
      {
        "key": "X-Auth-Token",
        "value": "ACCESS_TOKEN",
        "description": ""
      }
    ],
    "body": {
      "mode": "raw",
      "raw": "{}"
    },
    "description": "Links up data records for a notion of related data, e.g. a note associated with a specific nudge"
  }
}
Data Retrieval
In this guide we only look at retrieving simple data that is only available to the PDA user and hence, usable only by certified dashboard applications. For other kinds of data access, jump to the section on consented data exchange where you will find further pointers.
In this simple case, data from the PDA is retrieved using a GET request to the /api/v2.6/data/namespace/endpoint.
The endpoint takes the following parameters:
Parameter
Default
Meaning
take
1000
Limit to how many records the requester will take
skip
0
How many records to skip – used together with take to implement data paging
orderBy
​
The field in the JSON structure according to which data should be ordered. If, for example, you wish to order data by time, we recommend including UNIX timestamp in the data as a number
recordId
​
ID of the record when looking up data for one specific item
{
  "request": {
    "url": "https://postman.hubat.net/api/v2.6/data/hat/locations?take=3",
    "method": "GET",
    "header": [
        {
            "key": "Content-Type",
            "value": "application/json",
            "description": ""
        },
        {
            "key": "X-Auth-Token",
            "value": "ACCESS_TOKEN",
            "description": ""
        }
    ],
    "body": {
        "mode": "raw",
        "raw": ""
    },
    "description": "getting the 3 most recent data records"
  }
}
Which returns top 3 results sorted by default ordering:
{
  "response": [
    {
      "id": "0ff8df0b-fd8f-45c1-a5d6-2f64d44804e0",
      "name": "Sample data output",
      "status": "OK",
      "code": 200,
      "header": [
        {
          "key": "Content-Type",
          "value": "application/json",
          "name": "Content-Type",
          "description": "The mime type of this content"
        }
      ],
      "body": [
        {
          "endpoint": "rumpel/locations",
          "recordId": "e965e022-6613-476a-a0cd-1f587a41b148",
          "data": {
            "id": 85998,
            "data": {
              "locations": {
                "accuracy": "10.0",
                "latitude": "51.671358277138",
                "longitude": "0.101014673709963",
                "timestamp": "2017-04-23T16:11:17+0000"
              }
            },
            "name": "record 1",
            "lastUpdated": "2017-04-23T16:11:17.000Z"
          }
        },
        {
          "endpoint": "rumpel/locations",
          "recordId": "fcf1a26b-e49f-4457-915b-156e14140f38",
          "data": {
            "id": 85996,
            "data": {
              "locations": {
                "accuracy": "1414.0",
                "latitude": "51.674001392439",
                "longitude": "0.100905202634514",
                "timestamp": "2017-04-23T16:12:58+0000"
              }
            },
            "name": "record 2",
            "lastUpdated": "2017-04-23T16:12:58.000Z"
          }
        },
        {
          "endpoint": "rumpel/locations",
          "recordId": "8f7afa92-39e2-48ab-8028-f5aebaa9918e",
          "data": {
            "id": 85997,
            "data": {
              "locations": {
                "accuracy": "1414.0",
                "latitude": "51.6658257133844",
                "longitude": "0.080477950927866",
                "timestamp": "2017-04-23T16:18:04+0000"
              }
            },
            "name": "record 3",
            "lastUpdated": "2017-04-23T16:18:04.000Z"
          }
        }
      ]
    }
  ]
}
Data Modification
Updating
Data can be updated by sending a PUT request to the /api/v2.6/data endpoint with the complete Data Record structure in the request body:
{
  "request": {
    "url": "https://postman.hubat.net/api/v2.6/data",
    "method": "PUT",
    "header": [
      {
        "key": "Content-Type",
        "value": "application/json",
        "description": ""
      },
      {
        "key": "X-Auth-Token",
        "value": "ACCESS_TOKEN",
        "description": ""
      }
    ],
    "body": {
      "mode": "raw",
      "raw": [
        {
          "endpoint": "hat/locations",
          "recordId": "e965e022-6613-476a-a0cd-1f587a41b148",
          "data": {
            "id": 85998,
            "data": {
              "locations": {
                "accuracy": "1000.0",
                "latitude": "51.671358277138",
                "longitude": "0.101014673709963",
                "timestamp": "2017-04-23T16:11:17+0000"
              }
            },
            "name": "record 1",
            "lastUpdated": "2017-04-23T16:11:17.000Z"
          }
        }
      ]
    },
    "description": "Deleting a record happens via a request providing the record's ID"
  }
}
The above request is expected to effectively update only the accuracy field of the record already stored, however in practice the provided data replaces the whole record already stored in the PDA.
Deleting
Data is deleted by sending a DELETE request to the /api/v2.6/data endpoint with IDs of the records to be deleted in query parameters:
{
  "request": {
    "url": "https://postman.hubat.net/api/v2.6/data?records=3474369e-2317-4ea2-9bc8-198700a1f9cb&records=6dcff611-45cb-49a9-82ca-318b9e5a3c17",
    "method": "DELETE",
    "header": [
      {
        "key": "Content-Type",
        "value": "application/json",
        "description": ""
      },
      {
        "key": "X-Auth-Token",
        "value": "ACCESS_TOKEN",
        "description": ""
      }
    ],
    "body": {
      "mode": "raw",
      "raw": ""
    },
    "description": "Deleting a record happens via a request providing the record's ID"
  }
}
If you are deleting multiple records in one go, just add a records parameter for each record ID. The response will contain an error if you are not authorised to delete any of the provided records or if any of the records do not exist – no records will be deleted in this case.
Consented Data Exchange
This guide has covered the basics of PDA Data I/O, namely inserting new data and retrieving it as-is. The points we have not covered include:
Consistently handling heterogeneous data – using data combinators and bundles​
Managing data access by parties other than the PDA user – using data debits​
Last modified 5mo ago
Parameter	Default	Meaning
take	1000	Limit to how many records the requester will take
skip	0	How many records to skip – used together with `take` to implement data paging
orderBy		The field in the JSON structure according to which data should be ordered. If, for example, you wish to order data by time, we recommend including UNIX timestamp in the data as a number
recordId		ID of the record when looking up data for one specific item