The Data Store API allows an application to manage a data store that can be shared with other applications and provides a mechanism to allow multiple applications to concurrently synchronize data from the data store into the application-local cache.

Introduction

The Data Store API allows an application to create and maintain a data store that can be shared with other applications. To make multiple applications keep their local storages up-to-date with the data store, this API supports a mechanism to allow an application to concurrently synchronize data from the data store into the application-local storage which can be indexed, grouped or sorted in whatever format the application needs in order to support its UI.

Motivation

The Contacts Manager API and the Messaging API were originally designed to support richer querying, like filtering, sorting and grouping data. However, they both have the severe shortcoming that the consumers are forced to live with the limitations of what querying capabilities those APIs can have.

That's why we are constantly having to revise these APIs because it turns out that the querying capabilities aren't matching what our applications need, which is not a workable long-term solution. Also, it's not even a workable short-term solution for third-party applications since we cannot revise the APIs to support the capabilities that every third-party application developer needs.

Therefore, we've been figuring out such a generic Data Store API to allow applications to synchronize and save data into their own application-local storages which can be managed in various formats than we could think of and bake into APIs, thus supporting richer querying capabilities that the application really needs.

Note that the Data Store API is still in charge of maintaining a central storage to keep data which can be added, retrieved or deleted by applications through a bunch of methods provided by the Data Store API. This implies the application doesn't need to duplicate the whole data in its own local storage. Instead, it can simply synchronize the data that is actively required to construct the needed indexes for its own querying purpose.

More than IndexedDB API

Similar to the IndexedDB API, the Data Store API provides some database-like methods to add, retrieve and delete data records but it doesn't expose methods for building indexes or searching data. Instead, it provides a synchronizing mechanism for applications to keep their local storages up-to-date and accordingly update the local indexes needed for filtering data.

The other difference from the IndexedDB API is the Data Store API provides a central data storage which can be concurrently accessed and modified by multiple applications. Also, it provides a permission model to allow different applications to have different types of priviledges to change the data store or listen to the change of the data store.

Examples

If an application has the privilege to modify the content of the data store, it can use a bunch of basic methods defined in the DataStore to manage the data records in the data store, which is shown as the following example. 

          // Retrieve a list of data stores named as 'fb-contacts'.
          navigator.getDataStores('fb-contacts').then(function(stores) {
            if (!stores.length) {
              return;
            }

            // Check if the application is allowed to modify the data store.
            if (stores[0].readOnly) {
              return;
            }

            // Retrieve an object.
            stores[0].get(42).then(function(obj) {
              // Update the object
              obj.name = 'foo';
              stores[0].update(42, obj).then(function(id) {
                // The object has been updated.
              }, function(error) {
                // The object fails to be updated.
              });
            });
      
            // Delete an object.
            stores[0].remove(23).then(function(success) {
              if (success) {
                // The object has been deleted.
              } else {
                // The object fails to be deleted.
              }
            });
      
            // Add a new object.
            stores[0].insert({ name: "bar" }).then(function(id) {
              // The object has been added.
            }, function(error) {
              // The object fails to be added.
            });
          });

An application can call the sync() method defined in the DataStore to keep its local storage synchronized with the data store, which creates a DataStoreCursor to retrieve the change history starting from a certain revision kept in the application to the current revision of the data store, which is shown as the following example. 

          var appLocalRevisionId = "revision_id_kept_by_app";

          // Retrieve a list of data stores named as 'fb-contacts'.
          navagiator.getDataStores('fb-contacts').then(functions(stores) {
            if (!stores.length) {
              return;
            }

            // Check if the application's local storage is out-of-date.
            if (appLocalRevisionId == stores[0].revisionId) {
              dump("The app's local storage is already in sync.\n");
              return;
            }
   
            var cursor = stores[0].sync(appLocalRevisionId);
   
            function cursorResolve(task) {
              switch (task.operation) {
                case 'done':
                  // All the data are in sync. Update the local revision ID.
                  dump("The current revision ID: " + task.revisionId + "\n");
                  appLocalRevisionId = task.revisionId;
                  return;

                case 'clear':
                  // All the data have to be deleted in the local storage.
                  break;

                case 'add':
                  // A new object has to be added in the local storage.
                  dump("Add ID: " + task.id + " data: " + task.data + "\n");
                  break;

                case 'update':
                  // An object has to be updated in the local storage.
                  dump("Update ID: " + task.id + " data: " + task.data + "\n");
                  break;

                case 'remove':
                  // An object has to be deleted in the local storage.
                  dump("Remove ID: " + task.id + " data: " + task.data + "\n");
                  break;
              }

              cursor.next().then(cursorResolve);
            }
   
            // Start to sync.
            cursor.next().then(cursorResolve);
          });

This specification defines conformance criteria that apply to a single product: the user agent that implements the interfaces that it contains.

Implementations that use ECMAScript to implement the APIs defined in this specification MUST implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [[!WEBIDL]], as this specification uses that specification and terminology.

Terminology

The EventHandler interface represents a callback used for event handlers as defined in [[!HTML5]].

The concepts queue a task and fire an event are defined in [[!HTML5]].

The terms event handler and event handler event types are defined in [[!HTML5]].

The Promise interface, the concepts of a resolver, a resolver's fulfill algorithm and a resolver's reject algorithm are defined in [[DOM4]].

Security and privacy considerations

Application Manifest

For the application that provides the data store, its manifest MUST be claimed to own the data store by datastores-owned, where the JSON object can contain multiple properties representing different names of data stores respectively. Each data store can use readonly to specify whether the data store can be read by other applications and description to describe the purpose.

As shown as the following example, if a Facebook applicaton wants to provide a read-only fb-contacts data store, its manifest MUST be claimed to own the data store by setting the attribute readonly to true. 

          {
            datastores-owned: {
              "fb-contacts": {
                "readonly": true,
                "description": "own the Facebook contacts data store"
              }
            }
          }

For the application that wants to access the data store, its manifest MUST be claimed to access the data store by datastores-access, where the JSON object can contain multiple properties representing different names of data stores respectively. Each data store can use access to specify the application's accessibility and description to describe the purpose.

As shown as the following example, if the application wants to read or modify (e.g., add, update, remove, clear... etc) the content of the fb-contacts data store, its manifest MUST be claimed to access the fb-contacts data store by setting the attribute access to readwrite. 

          {
            datastores-access: {
              "fb-contacts": {
                "access": "readwrite",
                "description": "access (read and write) the Facebook contacts data stores"
              }
            }
          }

As shown as the following example, if the application simply wants to read the content of the fb-contacts data store without the need of modifying it, its manifest MUST be claimed to access the fb-contacts data store by setting the attribute access to readonly. 

          {
            datastores-access: {
              "fb-contacts": {
                "access": "readonly",
                "description": "access (read only) the Facebook contacts data store"
              }
            }
          }

Permission Model

The Data Store API can be exposed to trusted or untrusted contents.

Need to define the permission model of how the privileged and the third-party applications can be allowed to use the Data Store API to access the data store.

Navigator Interface

Promise getDataStores ()
This method makes a request to retrieve the data stores by the name parameter. It returns a new Promise that will be used to notify the caller about the result of the operation, which is an array of DataStore elements to access the data stores which have the same name equal to the name.
DOMString name
Specifies the name of the data store.

Steps

The getDataStores method when invoked MUST run the following steps:

  1. Let promise be a new Promise object and resolver be its associated resolver.
  2. Return promise to the caller.
  3. Make a request to the system to retrieve the data store(s) with the name equal to the name parameter passed in the request, where the caller application has claimed the data store access by datastores-access in its manifest.
  4. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  5. When the request has been successfully completed:
    1. Let dataStores be the array of the retrieved DataStore elements.
    2. Invoke resolver's fulfill algorithm with dataStores as the value argument.

DataStore Interface

The DataStore interface represents a bunch of properties of the data store and a set of operations that can be used to manage and synchronize the content of the data store.

readonly attribute DOMString name
MUST return the name of the data store. Note that different data stores can share the same name as long as they have the same database schema/format.
readonly attribute DOMString owner
MUST return the owner of the data store, which can be the manifest URL of the owner application.
readonly attribute boolean readOnly
MUST return whether the content of the data store can be changed or not by the caller.
readonly attribute DOMString revisionId
MUST return the current revision of the data store, which can be a UUID string.
Promise get ()
This method makes a request to retrieve the data record(s) by the id parameter. It returns a new Promise that will be used to notify the caller about the result of the operation, which is an arbitrary object to represent the data record if id is a single value, or a set of data records if id is an array of values.
DataStoreKey... id
Identifies the data record(s) that is requested to be retrieved.
Promise update ()
This method makes a request to update the existing data record by the id and the data parameters. It returns a new Promise that will be used to notify the caller about the result of the operation.
DataStoreKey id
Identifies the data record that is requested to be updated.
any data
Specifies the content of the data record to update.
Promise insert ()
This method makes a request to add a new data record by the data parameter. It returns a new Promise that will be used to notify the caller about the result of the operation, which is an identifier to access the data record that is added.
any data
Specifies the content of the data record to add.
Promise remove ()
This method makes a request to delete the data record(s) by the id parameter. It returns a new Promise that will be used to notify the caller about the result of the operation, which is a boolean value to indicate whether the data record(s) is successfully deleted or not.
DataStoreKey... id
Identifies the data record(s) that is requested to be deleted.
Promise clear ()
This method makes a request to clear all the data records in the data store. It returns a new Promise that will be used to notify the caller about the result of the operation.
Promise getLength ()
This method makes a request to retrieve the total number of data records saved in the data store. It returns a new Promise that will be used to notify the caller about the result of the operation, which is a numeric value to indicate the total number of data records saved in the data store.
DataStoreCursor sync ()
This method makes a request to retrieve the change history between a particular revision and the current revision of the data store by the revisionId parameter. It returns a new DataStoreCursor that will be used to iteratively access a set of DataStoreTask elements.
optional DOMString revisionId
Identifies the revision of the data store that the application currently keeps. If this parameter is absent or cannot be identified, this method will return a DataStoreCursor iterating through all the existing data records currently saved in the data store.
attribute EventHandler onchange
Handles the change event of type DataStoreChangeEvent, fired when a data record is added, updated or deleted in the data store. Note that if some data change in the data store when the DataStoreCursor is still synchronizing data and the cursor's close method has not yet been called, all the change events will not be dispatched to the application's onchange event handler. Instead, all the changes will be managed by the cursor's next method as additional operations.

Steps

The get method when invoked MUST run the following steps:

  1. Let promise be a new Promise object and resolver be its associated resolver.
  2. Return promise to the caller.
  3. Make a request to the data store to retrieve the data record(s) with the identifier equal to the id parameter passed in the request.
  4. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  5. When the request has been successfully completed:
    1. Let dataRecord be the retrieved data record(s).
    2. Invoke resolver's fulfill algorithm with dataRecord as the value argument.

The update method when invoked MUST run the following steps:

  1. Let promise be a new Promise object and resolver be its associated resolver.
  2. Return promise to the caller.
  3. If the readOnly attribute of the current data store is true to the caller, invoke resolver's reject algorithm without assigning a value to the value argument.
  4. Make a request to the data store to retrieve the data record with the identifier equal to the id parameter passed in the request.
  5. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  6. When the request has been successfully completed:
    1. Let dataRecord be the retrieved data record.
    2. Replace dataRecord by the data parameter passed in the request.
    3. Make a request to the data store to save dataRecord back to the data store.
    4. If an error occurs invoke resolver's reject algorithm with error as the value argument.
    5. When the request has been successfully completed:
      1. Invoke resolver's fulfill algorithm without assigning a value to the value argument.

The insert method when invoked MUST run the following steps:

  1. Let promise be a new Promise object and resolver be its associated resolver.
  2. Return promise to the caller.
  3. If the readOnly attribute of the current data store is true to the caller, invoke resolver's reject algorithm without assigning a value to the value argument.
  4. Make a request to the data store to add the data parameter passed in the request.
  5. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  6. When the request has been successfully completed:
    1. Let id be the generated identifier of the added data record.
    2. Invoke resolver's fulfill algorithm with id as the value argument.

The remove method when invoked MUST run the following steps:

  1. Let promise be a new Promise object and resolver be its associated resolver.
  2. Return promise to the caller.
  3. If the readOnly attribute of the current data store is true to the caller, invoke resolver's reject algorithm with false as the value argument.
  4. Make a request to the data store to retrieve the data record(s) with the identifier equal to the id parameter passed in the request.
  5. If an error occurs invoke resolver's reject algorithm with false as the value argument.
  6. When the request has been successfully completed:
    1. Make a request to the data store to delete the data record(s) with the identifier equal to the id parameter passed in the request.
    2. If an error occurs invoke resolver's reject algorithm with false as the value argument.
    3. When the request has been successfully completed:
      1. Invoke resolver's fulfill algorithm with true as the value argument.

The clear method when invoked MUST run the following steps:

  1. Let promise be a new Promise object and resolver be its associated resolver.
  2. Return promise to the caller.
  3. If the readOnly attribute of the current data store is true to the caller, invoke resolver's reject algorithm without assigning a value to the value argument.
  4. Make a request to the data store to clear all the data records.
  5. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  6. When the request has been successfully completed:
    1. Invoke resolver's fulfill algorithm without assigning a value to the value argument.

The getLength method when invoked MUST run the following steps:

  1. Let promise be a new Promise object and resolver be its associated resolver.
  2. Return promise to the caller.
  3. Make a request to the data store to get the total number of data records.
  4. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  5. When the request has been successfully completed:
    1. Let length be the total number of data records saved in the data store.
    2. Invoke resolver's fulfill algorithm with length as the value argument.

The sync method when invoked MUST run the following steps:

  1. Make a request to the system to retrieve a cursor that can iterate through the change history of the current data store.
  2. When the request has been successfully completed:
    1. Let dataStoreCursor be a new instance of DataStoreCursor.
    2. Set the store of dataStoreCursor to the current data store.
    3. Return dataStoreCursor.

Event handlers

The following are the event handlers (and their corresponding event types) that MUST be supported as attributes by the DataStore object.

Event handler Event name Event type Short description
onchange change DataStoreChangeEvent Handles the information of the data record changed in the data store.

DataStoreCursor Interface

The DataStoreCursor interface allows the application to iterate through a list of DataStoreTask elements that represents the change history of the data store.

readonly attribute DataStore store
MUST return the data store that is currently iterated by the cursor.
Promise next ()
This method makes a request to retrieve the information of the next operation that changes a data record in the data store. It returns a new Promise that will be used to notify the caller about the result of the operation, which is a DataStoreTask to represent the information of the change operation.
void close ()
This method makes a request to terminate the cursor iterating through the change history of the data store. Note that this method has to be explicitly called when the cursor completes its tasks. Otherwise, if some data changes in the data store when the cursor is still synchronizing data, all the changes will be managed by the cursor's next method as additional operations, which means when the cursor completes its tasks, the application will be in synchronization with the current revision of the data store.

Steps

The next method when invoked MUST run the following steps:

  1. Let promise be a new Promise object and resolver be its associated resolver.
  2. Return promise to the caller.
  3. Make a request to the system to retrieve the information of the next operation that changes a data record in the data store.
  4. If an error occurs invoke resolver's reject algorithm with error as the value argument.
  5. When the request has been successfully completed:
    1. Let dataStoreTask be a new instance of DataStoreTask:
      1. Set the revisionId of dataStoreTask to the revision of the data store, which changes a data record.
      2. Set the id of dataStoreTask to the identifier of the changed data record in the data store.
      3. Set the operation of dataStoreTask to the type of operation that changes the data record in the data store.
      4. Set the data of dataStoreTask to the changed data record in the data store.
    2. Invoke resolver's fulfill algorithm with dataStoreTask as the value argument.

The close method when invoked MUST run the following steps:

  1. Make a request to the system to terminate the current cursor iterating through the change history of the data store.

DataStoreTask Dictionary

The DataStoreTask dictionary contains the information related to a data record changed in the data store.

DOMString revisionId
MUST return the identifier of the revision of the data store, which changes a data record.
DataStoreOperation operation
MUST return the type of operation that changes the data record in the data store.
DataStoreKey? id
MUST return the identifier of the changed data record in the data store. MUST return null if the operation is clear or done.
any data
MUST return an arbitrary object to represent the changed data record in the data store. MUST return null if the operation is clear or done.

DataStoreChangeEvent Interface

The DataStoreChangeEvent interface represents the event related to a date record changed in the data store.

readonly attribute DOMString revisionId
MUST return the identifier of the revision of the data store, which changes a data record.
readonly attribute DataStoreOperation operation
MUST return the type of operation that changes the data record in the data store.
readonly attribute DataStoreKey? id
MUST return the identifier of the changed data record in the data store. MUST return null if the operation is clear or done.
readonly attribute DOMString owner
MUST return the manifest URL of the application which changes the data record in the data store. Note that the owner here is not the owner application of the data store. Instead, it's the application making the change in the data store.

Enumerations

The attribute operation in a DataStoreTask or a DataStoreChangeEvent can have the following values:

add
The data record is added in the data store.
update
The data record is updated in the data store.
remove
The data record is deleted in the data store.
clear
All the data records are deleted in the data store.
done
No additional opreations in the data store.

Acknowledgements

The editors would like to express their gratitude to the Mozilla Firefox OS Team and specially to Jonas Sicking, Mounir Lamouri, Ehsan Akhgari, Thinker Lee and Hsin-Yi Tsai for their technical guidance, as well as to Andrea Marchesini for his implementation work and support. Also, huge thanks to Zoltan Kis (Intel) and Christophe Dumez (Samsung) for their suggestions and contributions to this specification.