Retrieve

The /retrieve endpoint is used for downloading records from one or more Magma models.

The POST to retrieve should like this:

{
  "project_name": "labors",
  "model_name": "monster",
  "record_names": [ "Lernean Hydra" ],
  "attribute_names": "all"
}

In response Magma will send a JSON payload, with template and documents sections. The former is a hash of all of the templates for the requested model_name; the latter is a hash containing each requested record.

There are some special values which will return alternative downloads:

{ "model_name" : "all" }                         # get all models, most useful for getting templates. 
{ "record_names" : "all" }                       # get any matching record
{ "attribute_names" : "identifier" }             # only get record identifiers (names), no other data.
{ "attribute_names" : [ "some", "attributes" ] } # only get the named attributes (plus the identifier)

You may also request pages:

{ "page_size" : 20, "page" : 1 }

If you just want data and no templates, you might prefer TSV format:

{ "format" : "tsv"` }

Filtering queries

A final keyword is available, filter. This allows you to select a subset records according to their attribute values. For example, we might try:

{
  "project_name" : "labors",
  "model_name" : "prize",
  "record_names" : "all",
  "attribute_names" : "all",
  "filter" : "worth>5 name~L"
}

Here we have filtered "prize" records on two attributes, the 'worth' and the 'name' attribute. The former must be > 5, the latter must match the regular expression /L/. Date/time attributes are in ISO8601 format (2000-01-01T10:10)

Getting data

If you are just getting started on your project, you probably want to begin by posting this query to /retrieve:

{ "project_name": "my_project", "model_name": "all",
  "record_names":[], "attribute_names": "all" }

This will return a set of JSON templates that describe the data models for your project, which you can use for informing your subsequent requests (for example getting model_names and attribute_names).

Getting a data table

Next, you might get a TSV of records by posting this query to /retrieve:

{ "project_name": "my_project", "model_name" : "my_model",
  "record_names": "all", "attribute_names": "all", "format": "tsv" }

You might be interested in filtering on the 'updated_at' attribute, which all records have by default:

{ "project_name": "my_project", "model_name" : "my_model",
  "record_names": "all", "attribute_names": "all",
  "filter": "updated_at>2010-01-01", "format": "tsv" }

Getting files

Some Magma records contain links to files (file or image attributes). You can retrieve these links with a query:

{ "project_name": "my_project", "model_name" : "my_model",
  "record_names": "all", "attribute_names": [ "my_file_attribute" ] }

With the links in hand you may use curl, wget, etc., to download the files.

You can also use this magma_get script to do bulk download of all the files for a given model/attribute. The script requires you to set the environment variable TOKEN with your Janus token.