Manifests

A "manifest" is a request for data. In this case we request data from Timur by writing a script (manifest); Timur fills this manifest and returns a consignment of data.

Manifest basics

The basic form of a Manifest is a set of variable/expression pairs:

@var1 = expression1
@var2 = expression2
@var3 = expression3

Each "expression" is a valid expression in Timur's scripting language (TimurLang).

Let's take a look at an example manifest for some clarity:

@immune_fraction_by_sample = table(
  [ 'sample', [ 'patient', '::has', 'experiment' ] ],
  [
    sample_name: [ 'sample_name' ],
    experiment_name: [ 'patient', 'experiment', 'name' ],
    cd45_count: [ 'population',
      [ 'name', '::equals', 'CD45+' ],
      [ 'stain', '::equals', 'sort' ],
      '::first', 'count' ],
    live_count: [ 'population',
      [ 'name', '::equals', 'Live' ],
      [ 'stain', '::equals', 'sort' ],
      '::first', 'count' ]
  ]
)
@category = @immune_fraction_by_sample$experiment_name 
@height = @immune_fraction_by_sample$cd45_count / @immune_fraction_by_sample$live_count

This manifest computes three values: first, it uses the table() function to retrieve a Matrix of data and stores it as the variable @immune_fraction_by_sample. Next, it extracts one of the columns of this saved Matrix as a Vector and saves it to another variable @category. Finally, it calculates the pairwise division of two column Vectors from that same Matrix and saves the result to a third variable @height.

When the manifest is run, the resulting consignment will return three values for @immune_fraction_by_sample, @category and @height - viz., a Matrix, a Vector and a Vector respectively. After that? You may want to download each of these values as a TSV, or you may want to connect your manifest to a Plot to visualize the results.

Data Types

Here are the basic data types in TimurLang:

• Number
• String
• Boolean
• DateTime
• Matrix
• Vector

The Vector type is a list of items with optional labels. This is a valid Vector:

[ 1, 2, 3, 4 ]

So is this:

[ ant: 1, bear: 2, cat: 3 ]

You may also have Vectors of Vectors:

[ pies: [ apple: 2, pumpkin: 3 ], fizzy_drinks: [ soda: 1, ginger_ale: 2 ] ]

You can do math on Vectors of the same length, the result of which is defined pairwise. E.g., this is valid if @v1 and @v2 are both 3 elements long:

@v1 * @v2 + [ 1, 2, 3]

The Matrix type currently has no constructor - you can only return it as output from a function call. However, once you do you can reference named columns as a Vector using the $ operator:

@m1$column1

Functions

You can call functions on data to do further calculation or querying.

question([ 'sample', '::all', '::identifier' ])

See TimurLang for a full function reference.

Querying data

Most of the time your manifest will start off with some data queries and follow up with some calculation. Queries ask for data from the Magma Query API. You can read up on how to construct a Magma Query on the Magma wiki.

Timur provides two function interfaces to the Magma Query API.

The first is a "raw" interface which merely asks Magma a "question" and returns a Vector result (usually):

@vector = question([ 'sample', '::all', '::identifier' ])

The second generates a Matrix of data by splitting the question into a "row part" and various "column parts", e.g.

@row_part = [ 'sample', [ 'patient', 'experiment', 'name', '::equals', 'Colorectal' ], '::all' ]
@column_parts = [ name: [ 'sample_name' ], date_of_digest: [ 'date_of_digest' ] ]
@matrix = table( concat( @row_part, [  @column_parts ] ) )

The row_part is a Vector specifying the first half of a question, which maps the entity we are interested in (in this example, all Colorectal samples). The column_parts is a Vector of Vectors, with labels as column names, specifying the second half of a question for each column, which reduces the mapped entity to a specific value (in this example, the sample_name and date_of_digest).

Macros

It's cumbersome typing out the same query over and over again. To make your life easier and your scripts cleaner, you can use macros.

Here is an example of how to construct a table query using macros:

@indication = {'sample', ['patient', 'experiment', 'name', '::equals', '%1' ], '::all'}
@population_fraction = {['population', [ 'name', '::equals', '%1' ], [ 'stain', '::equals', '%2' ], '::first', 'count' ]}
@colorectal_table = table([ @indication('Colorectal'), [
    cd45_count: @population_fraction('CD45+', 'sort'),
    live_count: @population_fraction('Live', 'sort')
  ]
])

We define a macro using curly braces {}. Inside we can put any expression we like. We invoke the macro using the variable reference: @variable_name( ... args ... ). All of the arguments must be String expressions. Each of the arguments is substituted for the corresponding argument symbol (%1 for the first argument, %2 for the second, etc.) and then evaluated.