How to verify an application is indeed running inside an SGX enclave

[lokesh2019]

Hello,

I have made a simple integer addition program in C to learn Gramin. I have modified the helloworld example's template file and it runs fine with both gramin-direct and gramin-sgx.

I am then trying to dump the process memory in both cases using gcore, and grepping for a known string to verify it is indeed encrypted in gramin-sgx. The output isn't what I'd expect. Please advise what I am doing wrong here.

With gramin-direct, I get three instances of the same string in the process dump:

$ strings calc-direct.dump.39653 | grep Enter
Enter two numbers A and B : 
Enter two numbers A and B : 
Enter two numbers A and B : 
$

And with gramin-sgx, I get two instances of the same string:

$ strings calc-sgx.dump.43151 |grep Enter
Enter two numbers A and B : 
Enter two numbers A and B : 
$ 

I was expecting just one instance with gramin-direct and zero instances with gramin-sgx.
Below is the code I am using for this.

calculator.c

#include <stdio.h>

int main()
{
    int A, B, sum = 0;
    printf("Enter two numbers A and B : \n");
    fflush(stdout);
    scanf("%d%d", &A, &B);
    sum = A + B;
    printf("%d + %d = %d\n", A, B, sum);
    return 0;
}

calculator.manifest.template

# Hello World manifest file example

loader.entrypoint = "file:{{ gramine.libos }}"
libos.entrypoint = "/calculator"
loader.log_level = "{{ log_level }}"

loader.env.LD_LIBRARY_PATH = "/lib"

fs.mounts = [
  { path = "/lib", uri = "file:{{ gramine.runtimedir() }}" },
  { path = "/calculator", uri = "file:calculator" },
]

sgx.debug = false
sgx.edmm_enable = {{ 'true' if env.get('EDMM', '0') == '1' else 'false' }}

sgx.trusted_files = [
  "file:{{ gramine.libos }}",
  "file:calculator",
  "file:{{ gramine.runtimedir() }}/",
]

[dimakuv]

How exactly do you use gcore on Gramine?

Anyway, the gramine-sgx process consists of two parts: an SGX enclave part which is encrypted in memory, and an untrusted part which is in plaintext in memory. Gramine needs to load the executable inside the enclave -- the only way to do it is to first read the executable contents into untrusted part of the Gramine process, and then to copy it into the enclave part. So here's one copy of the string already for you (this string is literally embedded in the executable file).

Next, your gramine-sgx process prints out this string to the terminal, and you see it in plaintext, right? This is because Gramine does not encrypt stdin/stdout/stderr, otherwise you won't be able to read Gramine logs. So when the string is printed on the terminal, this literally means that Gramine copies this string from the enclave part to the untrusted part, to be able to output it. This is the second copy.

Now the third copy that you see in gramine-direct, and don't see in gramine-sgx, is probably the second copy of the executable. Here I don't remember the details of Gramine-direct inner workings, but my assumption is that Gramine-direct also copies the executable from one place into another place (which makes no sense since both executables are in plaintext anyway).

So you're having plaintext as input to gramine-sgx (the executable contents) and plaintext as output of gramine-sgx (the printed string). No surprise gcore finds the string.

---

I would recommend to use an encrypted file (fs.mounts = [ type="encrypted", ...]). Put some secret string in a file, encrypt this file, specify it as encrypted in the Gramine manifest, read this file in your program and do NOT output this file from your program.

In this case, you'll have a clean experiment, and gramine-sgx won't show the secret string, whereas gramine-direct will.

[lokesh2019]

Thank you for the detailed explanation, @dimakuv. It makes sense now.

Re. How exactly do you use gcore on Gramine? - I used ps -elf to find PID on the process that runs gramin-direct/gramine-sgx and dumped the memory.
The test came from my experimentation with Anjuna's LibOS. Their tool prints exactly one copy of string on untrusted side and zero on TEE side. Different ways to skin the cat I guess. But I am not relying on hardcoded strings in the binaries (that'd be silly :) ) anyway.

I will experiment with the encrypted files as you suggest. Many thanks.

[dimakuv]

But I am not relying on hardcoded strings in the binaries (that'd be silly :) ) anyway.

What do you mean you don't rely on hardcoded strings in the binaries? That's literally what happens in your calculator.c example:

$ gcc -O3 calculator.c

$ strings a.out | grep "Enter"
Enter two numbers A and B :

I don't know how Anjuna works and what is "their tool", but the hardcoded string in the binary will be there, unless your calculator.c in the top-level description is a contrived example.

[lokesh2019]

I meant calculator.c was just Experiment-1. Next one won't rely on the hardcoded string.