add exporting table files #120

fengshuo · 2018-10-10T22:00:33Z

Add capability to export tables from within the Python package (see here for more info)
Update documentation

jjmar · 2018-10-12T14:56:19Z

quandl/bulkdownloadtable.py

+from .message import Message
+
+
+def bulkdownloadtable(datatable_code, **kwargs):


This function should be bulk_download_table to follow python convention.

Although now I see theres already a bulkdownload function - maybe it would make sense to keep this name the way it is now to keep consistency.

yeah i was following the bulkdownload for timeseries data

jjmar · 2018-10-12T15:03:56Z

quandl/model/datatable.py

+            return file_path
+        else:
+            print(Message.LONG_GENERATION_TIME)
+            self._url_request(file_or_folder_path, **options)


Correct me if I'm wrong but doesn't this line prevent the sleep(30) from being run since it will always continue to recurse on _url_request?

ah yeah i changed the order here yesterday good catch

jjmar · 2018-10-12T15:35:13Z

Theres documentation for quandl.bulkdownload, we should probably add some for this new function as well

sushant-hukeri · 2018-10-12T19:09:12Z

quandl/model/datatable.py

+        if ApiConfig.api_version:
+            options['params']['api_version'] = ApiConfig.api_version
+
+        if list(options.keys()):


Did you mean if options['params']:?

I updated this part since this api doesn't need to support more parameters now

sushant-hukeri · 2018-10-12T19:22:52Z

quandl/model/datatable.py

+
+            file_path = file_or_folder_path
+            if os.path.isdir(file_or_folder_path):
+                file_path = file_or_folder_path + '/' + code_name.replace('/', '_') + '.zip'


This will probably be better: file_path = os.path.join(file_or_folder_path, '{}.{}'.format(code_name.replace('/', '_') , 'zip')

sushant-hukeri · 2018-10-12T19:33:27Z

test/test_datatable.py

+    def setUpClass(cls):
+        httpretty.enable()
+        httpretty.register_uri(httpretty.GET,
+                               re.compile(


Is there any reason for such formatting?

not sure, I was looking at the old tests...

sushant-hukeri · 2018-10-12T19:35:58Z

test/test_datatable.py

@@ -48,3 +50,46 @@ def test_dataset_column_names_match_expected(self):
        metadata = Datatable('ZACKS/FC').data_fields()
        six.assertCountEqual(self,
                             metadata, [u'datatable_code', u'id', u'name', u'vendor_code'])
+
+
+class BulkDownloadDataTableTest(unittest.TestCase):


Don't we need any positive test case?

added one positive test case

jjmar · 2018-10-15T19:16:38Z

FOR_ANALYSTS.md

+```
+After the download is finished, the filename of the downloaded zip file will be returned.
+
+Sometime it takes a while to generate the zip file, you'll get a message while the file is being generated. Once the file is generated, it will start download the zip file.


Small grammar issues- should say Sometimes instead of Sometime. Also it will start download the should be it will start the download of the

haha, will update!

jjmar · 2018-10-15T19:17:08Z

FOR_DEVELOPERS.md

+```
+After the download is finished, the filename of the downloaded zip file will be returned.
+
+Sometime it takes a while to generate the zip file, you'll get a message while the file is being generated. Once the file is generated, it will start download the zip file.


Same grammar issue as I posted in the other doc file

sushant-hukeri · 2018-10-15T19:28:07Z

README.md

@@ -117,6 +117,8 @@ The following are instructions for running our tests:
    `python setup.py install`
 4. Run the following command to test the plugin in all versions of python we support:
    `tox`
+
+Once you have all required package installed, you can run tests locally with `python -m unittest -v test.[test file name].[test case name].[individual test name]`.


Should be packages,

-v, which stands for verbose, is optional,

Shouldn't it be class name instead of test case name?

Also, this is to run individual tests. Users will most likely be interested in running all tests which can be accomplished by python -m unittest

junos · 2018-10-15T19:54:34Z

quandl/model/datatable.py

+            return file_path
+        else:
+            print(Message.LONG_GENERATION_TIME)
+            sleep(30)


could you put this 30 seconds waiting time into a config file or define it as const variable with meaningful?

junos · 2018-10-15T19:57:56Z

quandl/model/datatable.py

+
+        return self._url_request(file_or_folder_path, **options['params'])
+
+    def _url_request(self, file_or_folder_path, **options):


could you refactoring this method smaller and easy to read?

mbasset · 2018-10-15T20:39:16Z

README.md

+
+Once you have all required package installed, you can run tests locally with:
+
+```ptyhon


sushant-hukeri · 2018-10-15T20:40:34Z

README.md

@@ -117,6 +117,24 @@ The following are instructions for running our tests:
    `python setup.py install`
 4. Run the following command to test the plugin in all versions of python we support:
    `tox`
+
+Once you have all required package installed, you can run tests locally with:


Should be packages

sushant-hukeri · 2018-10-15T20:41:05Z

README.md

+
+python -W always setup.py -q test
+
+# runnging an individual test


Typo. running

sushant-hukeri · 2018-10-15T20:41:57Z

README.md

+
+python -m unittest test.[test file name].[class name].[individual test name]`
+
+# such as


Perhaps Example: is better than such as.

all updated

sushant-hukeri · 2018-10-16T14:54:42Z

FOR_DEVELOPERS.md

+quandl.export_table('MER/F1', params={'compnumber': '39102', 'mapcode':'-5370','reporttype': 'A', 'qopts': {'columns': ['reportdate', 'amount']}})
+```
+
+After the download is finished, the filename of the downloaded zip file will be returned.


Are you still returning the filename? I think you changed the method to only print the filename.

mbasset · 2018-10-16T15:22:50Z

quandl/model/datatable.py

+        if 'params' not in options:
+            options['params'] = {}
+
+        while True:


status = false while !status: status = self._request_file_info(file_or_folder_path, **options['params']) if !status sleep(self.WAIT_GENERATION_INTERVAL)

add exporting table files

319e032

fengshuo added the work in progress label Oct 10, 2018

fengshuo self-assigned this Oct 10, 2018

Shuo Feng added 3 commits October 11, 2018 16:35

modify the way we handle long generation time

d3c4e58

modify test

87ff77a

flake 8 fix

ff664a8

fengshuo added ready for review and removed work in progress labels Oct 12, 2018

fengshuo removed their assignment Oct 12, 2018

jjmar reviewed Oct 12, 2018

View reviewed changes

sushant-hukeri reviewed Oct 12, 2018

View reviewed changes

Shuo Feng added 2 commits October 12, 2018 22:00

update docs and some small changes

debf27c

fix in test

57aa3b9

jjmar reviewed Oct 15, 2018

View reviewed changes

sushant-hukeri reviewed Oct 15, 2018

View reviewed changes

Shuo Feng added 2 commits October 15, 2018 15:32

support filters, rename function name

5ca5c98

minor update in readme

c5e998b

jjmar approved these changes Oct 15, 2018

View reviewed changes

junos suggested changes Oct 15, 2018

View reviewed changes

Shuo Feng added 2 commits October 15, 2018 15:58

PR review feedback

7c35fd4

small funtion update

1c94ae3

junos approved these changes Oct 15, 2018

View reviewed changes

mbasset reviewed Oct 15, 2018

View reviewed changes

sushant-hukeri suggested changes Oct 15, 2018

View reviewed changes

Shuo Feng added 2 commits October 15, 2018 17:42

use while to avoid recursion

f2e219f

text tweak

63eb730

update readme

c713f0c

sushant-hukeri reviewed Oct 16, 2018

View reviewed changes

text tweak

3d8ea55

sushant-hukeri approved these changes Oct 16, 2018

View reviewed changes

mbasset reviewed Oct 16, 2018

View reviewed changes

Shuo Feng added 4 commits October 16, 2018 13:31

refactor and add test

c681509

change the way parameter is being passed

87b1a14

change parameters

4c2924e

flake8 fix

1bbf98e

fengshuo merged commit 579b682 into master Oct 19, 2018

fengshuo deleted the CP-7116/python-download branch October 19, 2018 16:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add exporting table files #120

add exporting table files #120

fengshuo commented Oct 10, 2018 •

edited

Loading

jjmar Oct 12, 2018 •

edited

Loading

fengshuo Oct 12, 2018

jjmar Oct 12, 2018

fengshuo Oct 12, 2018

jjmar commented Oct 12, 2018

sushant-hukeri Oct 12, 2018

fengshuo Oct 13, 2018

sushant-hukeri Oct 12, 2018

sushant-hukeri Oct 12, 2018

fengshuo Oct 12, 2018

sushant-hukeri Oct 12, 2018

fengshuo Oct 13, 2018

jjmar Oct 15, 2018

fengshuo Oct 15, 2018

jjmar Oct 15, 2018

sushant-hukeri Oct 15, 2018

junos Oct 15, 2018

fengshuo Oct 15, 2018

junos Oct 15, 2018

fengshuo Oct 15, 2018

mbasset Oct 15, 2018

fengshuo Oct 15, 2018

sushant-hukeri Oct 15, 2018

sushant-hukeri Oct 15, 2018

sushant-hukeri Oct 15, 2018

fengshuo Oct 15, 2018

sushant-hukeri Oct 16, 2018

fengshuo Oct 16, 2018

mbasset Oct 16, 2018

		from .message import Message


		def bulkdownloadtable(datatable_code, **kwargs):


		return self._url_request(file_or_folder_path, **options['params'])

		def _url_request(self, file_or_folder_path, **options):


		Once you have all required package installed, you can run tests locally with:

		```ptyhon


		python -W always setup.py -q test

		# runnging an individual test


		python -m unittest test.[test file name].[class name].[individual test name]`

		# such as

add exporting table files #120

add exporting table files #120

Conversation

fengshuo commented Oct 10, 2018 • edited Loading

jjmar Oct 12, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jjmar commented Oct 12, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fengshuo commented Oct 10, 2018 •

edited

Loading

jjmar Oct 12, 2018 •

edited

Loading