Make the netrc path configurable

[naegelyd]

Currently, the NETRC_PATH is always set to the users home directory in this block. In the multiuser environment we are running on, I am encountering permission issues in this block. By making NETRC_PATH, it would allow users of solvebio to use a fixed path rather than using the home directory which will change user to user.

[davecap]

Thanks for the feedback. Is it that os.environ['HOME'] raises a KeyError? or is it that you cannot access ~/.netrc at all?

[davecap]

@naegelyd can you try the latest master branch and see if it fixes your issue? It supports a $NETRC_PATH environment variable now.

[naegelyd]

It was generating a KeyError. I worry that this approach will also be an issue in our multiuser setup. While we have multiple users who can deploy and restart the varify app, the app itself runs under its own user account. I have not fully looked into the process but I would really like if this could be defined in the django settings file like the SOLVEBIO_API_KEY is now. Do you think making this configurable via the django settings file is doable? I think that would be the most reliable way given our multi-user environment that I described above.

[davecap]

In the most recent commit, I changed it so that it won't raise the error you got unless you run the solvebio command-line script. The idea is that the .netrc credentials are used for local development, while apps like Varify should explicitly set the SOLVEBIO_API_KEY in local_settings.py or as an environment variable (bypassing the .netrc). Does that make sense?

[naegelyd]

Yea, that fixes the netrc issue I was seeing. I am now getting the following DatabaseError. Any idea what might be causing it?

relation "django_solvebio_datasetalias" does not exist LINE 1: ... "django_solvebio_datasetalias"."dataset_id" FROM "django_so... ^

Stacktrace (most recent call last):

  File "django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "django/views/decorators/cache.py", line 89, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "restlib2/resources.py", line 269, in __call__
    return self.dispatch(request, *args, **kwargs)
  File "restlib2/resources.py", line 280, in dispatch
    response = method_handler(request, *args, **kwargs)
  File "varify/samples/resources.py", line 237, in get
    data = self._cache_data(request, pk, key)
  File "varify/samples/resources.py", line 217, in _cache_data
    data['variant'] = VariantResource.get(request, data['variant_id'])
  File "varify/api/__init__.py", line 31, in decorate
    data = func(self, request, pk, *args, **kwargs)
  File "varify/variants/resources.py", line 104, in get
    clinvar = solvebio_client.query('clinvar', filters)
  File "solvebio/contrib/django_solvebio/client.py", line 40, in query
    dataset_id = self.get_dataset_id(alias)
  File "solvebio/contrib/django_solvebio/client.py", line 19, in get_dataset_id
    return DatasetAlias.objects.get(alias=alias).dataset_id
  File "django/db/models/manager.py", line 131, in get
    return self.get_query_set().get(*args, **kwargs)
  File "django/db/models/query.py", line 361, in get
    num = len(clone)
  File "django/db/models/query.py", line 85, in __len__
    self._result_cache = list(self.iterator())
  File "django/db/models/query.py", line 291, in iterator
    for row in compiler.results_iter():
  File "django/db/models/sql/compiler.py", line 763, in results_iter
    for rows in self.execute_sql(MULTI):
  File "django/db/models/sql/compiler.py", line 818, in execute_sql
    cursor.execute(sql, params)
  File "django/db/backends/util.py", line 40, in execute
    return self.cursor.execute(sql, params)
  File "django/db/backends/postgresql_psycopg2/base.py", line 52, in execute
    return self.cursor.execute(query, args)

[davecap]

Did you run the migrations on django_solvebio?

[naegelyd]

There was an issue with my settings file where it wasn't including solvebio in the installed apps so it wasn't being migrated during our deploy process. I updated it and that error is resolved.

[naegelyd]

Do you know what might be causing the clinvar query to fail? Whenever retrieving variants, I keep ended up on this line.

[davecap]

That's my fault for poor error handling. I'm going to have the query function raise an exception instead of returning None.

[davecap]

Check out the latest solvebio package, as well as the pull request I just made on Varify. That should give you more detailed info on query errors.

[naegelyd]

@davecap I'm still getting an exception in this block. This might be a firewall/config issue on our end. Are there any known ports that need to be open in order to connect to and interact with solvebio datasets? Here is the exception message:

'SolveBio ClinVar query failed: Cannot find the SolveBio dataset by alias "clinvar"'

[davecap]

From the exception it seems that there is just no alias defined for clinvar. See if you have something like this set in your django settings:

SOLVEBIO_DATASET_ALIASES = {
    'clinvar': 'ClinVar/0.0.2/ClinVar'
}

Or you can go to the django admin and add an alias to the django_solvebio app (DatasetAlias model).

[naegelyd]

Good catch. Thanks @davecap. It's all working well now.