How does it work?

[LaurentBerder]

Hey,

I saw your comment on Prophet's project and decided to have a look.
Would you mind giving a little walkthrough in your readme file, so I know how to use your project?

Thanks

[pravj]

Hi @LaurentBerder,

Thanks for checking it out.

It's my first React based project, and in the quest to follow best practices that are scattered on the internet, I didn't pay much attention to develop it in an environment agnostic way.

In short, at this moment some of the paths in the project are hard-coded for my machine, given that it's in development phase.

UPLOAD_DIR = '/Users/pravj-mac/Projects/prophetly-modules/prophetly-react/uploads'

Now that you have asked for this, I will be working to deliver the requirement.

I'm keeping it open, and will notify when you can build it via the GitHub repo, and then I'll work on publishing it on npm and pip.

[pravj]

Hi @LaurentBerder,

Hope you're doing good.

I've been working to deliver the installation requirement for Prophetly, by refactoring it as two different projects.

prophetly-client

React application which represents the front-end for Prophetly.

prophetly-server

PyPI package which serves the static content generated by prophetly-client using Tornado server.

That means, you just need to have Python installed on your system to use it, all the NPM dependencies will be included as package_data.

Because of matplotlib being a dependency, it requires Python 2.7, 3.4, 3.5, or 3.6

Build from source

I haven't published it on PyPI, but you can try from the GitHub source.

Clone and create a virtual environment

$ git clone git@github.com:Prophetly/prophetly-server.git
$ cd prophetly-server
$ virtualenv prophetly-server-venv
...
$ source prophetly-server-venv/bin/activate

Install Python implementation of fbprophet according to your system

Prophet Installation

Install an environment specific prophetly-server

$ python setup.py install

Start the Prophetly server

$ prophetly-server start

This should run an instance on port 9009, where you can upload a csv dataset following Prophet's requirement.

It'll be helpful if you can test it out and notify any hiccups you encounter during the process.

Thanks

[LaurentBerder]

So far, I haven't been able to upload a csv.

I'm trying with a csv called LHR-BKK2.csv, here's what I can read in the console:

INFO:tornado.access:304 GET / (127.0.0.1) 4.49ms INFO:tornado.access:304 GET /static/media/repo-push.27cdcdb1.svg (127.0.0.1) 2.43ms INFO:tornado.access:304 GET /static/media/graph.3a838cb4.svg (127.0.0.1) 0.54ms INFO:tornado.access:304 GET /static/media/desktop-download.39501215.svg (127.0.0.1) 0.51ms ERROR:tornado.application:Uncaught exception GET /upload (127.0.0.1) HTTPServerRequest(protocol='http', host='localhost:9009', method='GET', uri='/upload', version='HTTP/1.1', remote_ip='127.0.0.1', headers={'Accept-Language': 'fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4', 'Accept-Encoding': 'gzip, deflate, sdch, br', 'Host': 'localhost:9009', 'Accept': 'application/json, text/plain, */*', 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36', 'Connection': 'keep-alive', 'Referer': 'http://localhost:9009/'}) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/tornado-4.3-py2.7-linux-x86_64.egg/tornado/web.py", line 1443, in _execute result = method(*self.path_args, **self.path_kwargs) File "/usr/local/lib/python2.7/dist-packages/prophetly_server-0.2.36.dev0-py2.7.egg/prophetly/server/handlers/upload_handler.py", line 21, in get os.makedirs(self.settings['upload_path']) File "/usr/lib/python2.7/os.py", line 157, in makedirs mkdir(name, mode) OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/prophetly_server-0.2.36.dev0-py2.7.egg/prophetly/server/uploads' ERROR:tornado.access:500 GET /upload (127.0.0.1) 1.02ms ERROR:tornado.application:Uncaught exception POST /upload (127.0.0.1) HTTPServerRequest(protocol='http', host='localhost:9009', method='POST', uri='/upload', version='HTTP/1.1', remote_ip='127.0.0.1', headers={'Origin': 'http://localhost:9009', 'Content-Length': '13642', 'Accept-Language': 'fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4', 'Accept-Encoding': 'gzip, deflate, br', 'Host': 'localhost:9009', 'Accept': 'application/json', 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36', 'Connection': 'keep-alive', 'X-Requested-With': 'XMLHttpRequest', 'Cache-Control': 'no-cache', 'Referer': 'http://localhost:9009/', 'Content-Type': 'multipart/form-data; boundary=----WebKitFormBoundaryleFGGYag6mH7gzQb'}) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/tornado-4.3-py2.7-linux-x86_64.egg/tornado/web.py", line 1443, in _execute result = method(*self.path_args, **self.path_kwargs) File "/usr/local/lib/python2.7/dist-packages/prophetly_server-0.2.36.dev0-py2.7.egg/prophetly/server/handlers/upload_handler.py", line 12, in post with open(os.path.join(self.settings['upload_path'], file_info.filename), 'w') as req_file: IOError: [Errno 2] No such file or directory: u'/usr/local/lib/python2.7/dist-packages/prophetly_server-0.2.36.dev0-py2.7.egg/prophetly/server/uploads/LHR-BKK2.csv' ERROR:tornado.access:500 POST /upload (127.0.0.1) 0.80ms

[pravj]

Can you please tell me the OS you are using?

I think it's happening because the package is unable to create a (upload) directory.

"upload_handler.py", line 21
in get os.makedirs(self.settings['upload_path'])
File "/usr/lib/python2.7/os.py", line 157
in makedirs mkdir(name, mode) OSError: [Errno 13] Permission denied: 

...
21 def get(self):
22        if not os.path.isdir(self.settings['upload_path']):
23            os.makedirs(self.settings['upload_path'])
...

[pravj]

Ahh, I guess it's Linux, as given in headers.

[pravj]

Hi @LaurentBerder,

I believe it should be fixed by the commit #0874fa.

It was trying to create the upload path in the place where pip install the package (site-package).

Now it will create that directory in the current working directory.

Please let me know if this works for you.

[LaurentBerder]

Yes, it is a Linux OS, Ubuntu 16.04.
Just tried renewing the experience with your latest commit, and I still get this:

INFO:tornado.access:304 GET / (127.0.0.1) 4.78ms INFO:tornado.access:304 GET /static/media/repo-push.27cdcdb1.svg (127.0.0.1) 5.10ms INFO:tornado.access:304 GET /static/media/graph.3a838cb4.svg (127.0.0.1) 0.96ms INFO:tornado.access:304 GET /static/media/desktop-download.39501215.svg (127.0.0.1) 0.70ms ERROR:tornado.application:Uncaught exception GET /upload (127.0.0.1) HTTPServerRequest(protocol='http', host='localhost:9009', method='GET', uri='/upload', version='HTTP/1.1', remote_ip='127.0.0.1', headers={'Accept-Language': 'fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4', 'Accept-Encoding': 'gzip, deflate, sdch, br', 'Host': 'localhost:9009', 'Accept': 'application/json, text/plain, */*', 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36', 'Connection': 'keep-alive', 'Referer': 'http://localhost:9009/'}) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/tornado-4.3-py2.7-linux-x86_64.egg/tornado/web.py", line 1443, in _execute result = method(*self.path_args, **self.path_kwargs) File "/usr/local/lib/python2.7/dist-packages/prophetly_server-0.2.36.dev0-py2.7.egg/prophetly/server/handlers/upload_handler.py", line 21, in get os.makedirs(self.settings['upload_path']) File "/usr/lib/python2.7/os.py", line 157, in makedirs mkdir(name, mode) OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/prophetly_server-0.2.36.dev0-py2.7.egg/prophetly/server/uploads' ERROR:tornado.access:500 GET /upload (127.0.0.1) 1.14ms ERROR:tornado.application:Uncaught exception POST /upload (127.0.0.1) HTTPServerRequest(protocol='http', host='localhost:9009', method='POST', uri='/upload', version='HTTP/1.1', remote_ip='127.0.0.1', headers={'Origin': 'http://localhost:9009', 'Content-Length': '13642', 'Accept-Language': 'fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4', 'Accept-Encoding': 'gzip, deflate, br', 'Host': 'localhost:9009', 'Accept': 'application/json', 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36', 'Connection': 'keep-alive', 'X-Requested-With': 'XMLHttpRequest', 'Cache-Control': 'no-cache', 'Referer': 'http://localhost:9009/', 'Content-Type': 'multipart/form-data; boundary=----WebKitFormBoundaryfGT9BWDvTnPsWD9v'}) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/tornado-4.3-py2.7-linux-x86_64.egg/tornado/web.py", line 1443, in _execute result = method(*self.path_args, **self.path_kwargs) File "/usr/local/lib/python2.7/dist-packages/prophetly_server-0.2.36.dev0-py2.7.egg/prophetly/server/handlers/upload_handler.py", line 12, in post with open(os.path.join(self.settings['upload_path'], file_info.filename), 'w') as req_file: IOError: [Errno 2] No such file or directory: u'/usr/local/lib/python2.7/dist-packages/prophetly_server-0.2.36.dev0-py2.7.egg/prophetly/server/uploads/LHR-BKK2.csv' ERROR:tornado.access:500 POST /upload (127.0.0.1) 0.76ms

And the file doesn't get uploaded

[pravj]

@LaurentBerder,

I think you're still using the same old version 0.2.36.dev0, the updated version on GitHub is 0.2.44.dev0.

So you will have to clone again and repeat the process to get it working.

[LaurentBerder]

Yes I am. But that's what I get whenever I git clone https://github.com/Prophetly/prophetly-server.git.

prophetly-server --version
Prophetly Server 0.2.36-dev

[pravj]

That's weird, it's working for me.

Can you please tell me the output of following command after you do a fresh clone?

git log --pretty=format:'%h' -n 1

[LaurentBerder]

0874fa0
Without the --pretty=format:'%h, I get:

commit 0874fa0246197f2328b29af696b43b52b67cf12b
Author: Pravendra Singh <hackpravj@gmail.com>
Date: Wed Apr 12 21:53:46 2017 +0530

Create upload path in current working directory

[pravj]

Then I suspect you aren't installing the updated package you get after clone.

You will have to run the $ python setup.py install again.

[LaurentBerder]

I did already. However, the setup.py is apparently not updated up to 0.2.44.dev0 but 0.2.36.dev0, so I don't see how it would solve the issue.

[pravj]

Pardon me, can you please elaborate this?

the setup.py is apparently not updated up to 0.2.44.dev0 but 0.2.36.dev0

[LaurentBerder]

Well, it seems like when I git clone https://github.com/Prophetly/prophetly-server.git, I only get Prophetly Server 0.2.36-dev while you said you'd brought correction for the upload issue was in 0.2.44.dev0.

[pravj]

Unable to catch the culprit here, I'm getting the desired source code as I clone.

• Are you using virtualenv?
• What do you get by $ which prophetly-server?

Why don't we just delete the entire directory and start afresh.
Just like the age old, restart your PC and the problem should die.

[LaurentBerder]

What do you get by $ which prophetly-server?

I get /home/laurent/prophetly-server/prophetly-server-venv/bin/prophetly-server

Here's what I did:

I deleted the whole prophetly-server folder, and rebooted the computer.

Then I typed:

• git clone https://github.com/Prophetly/prophetly-server.git

• cd prophetly-server

• git log -n 1
commit 0874fa0246197f2328b29af696b43b52b67cf12b Author: Pravendra Singh <hackpravj@gmail.com> Date: Wed Apr 12 21:53:46 2017 +0530 (still no change)

• virtualenv prophetly-server-venv

• source prophetly-server-venv/bin/activate

• python setup.py install

Now, I run into another problem:

No matter if I try prophetly-server --version or prophetly-server start, I get an error for missing libraries.
Traceback (most recent call last): File "/home/laurent/prophetly-server/prophetly-server-venv/bin/prophetly-server", line 6, in <module> from pkg_resources import load_entry_point File "/home/laurent/prophetly-server/prophetly-server-venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2927, in <module> @_call_aside File "/home/laurent/prophetly-server/prophetly-server-venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2913, in _call_aside f(*args, **kwargs) File "/home/laurent/prophetly-server/prophetly-server-venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2940, in _initialize_master_working_set working_set = WorkingSet._build_master() File "/home/laurent/prophetly-server/prophetly-server-venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 637, in _build_master return cls._build_from_requirements(__requires__) File "/home/laurent/prophetly-server/prophetly-server-venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 650, in _build_from_requirements dists = ws.resolve(reqs, Environment()) File "/home/laurent/prophetly-server/prophetly-server-venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 829, in resolve raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'tornado==4.3' distribution was not found and is required by prophetly-server

I did make sure that tornado was up to date: I'm on version tornado-4.4.3

[pravj]

Hi @LaurentBerder,

First off, I'm thankful to you for helping me here.
And I'm sorry if you are getting angry because of the issues.

• The commit that fixed the directory creation issue was #0874fa, so it's completely fine, the clone is working.

• I have used the specific version 4.3 of tornado because I wanted to support Python 2.6, and tornado dropped the support for Python 2.6 after its 4.3 release. But later I found that even the new matplotlib doesn't have a support for it and realised that majority of legacy Python versions is 2.7.

So I forgot to revert tornado to its latest, or version independent, ie. 4.4.3

Now, I have fixed this in the new commit #a17bddd.

And you will not have to do the clone this time.

I've uploaded the package at the testpypi

You can install it through pip.

$ pip install --user --extra-index-url https://testpypi.python.org/pypi prophetly-server

This should result in version 0.0.2.a1.

Once you confirm its working, we will prepare for publishing it to the real PyPI.

[LaurentBerder]

Thanks @pravj, that works.

First of all, don't worry, I'm not getting angry, just trying to be useful. I'm actually not entirely sure I'll be using your tool, just giving you a hand in configuring it.

I now work with prophetly-server --version: Prophetly Server 0.2.44-dev, and the upload is properly functionning!

Now, when I select my "datestamp column" and my "y column" and specify the number of days to forecast, I get the following error in the GUI:
AttributeError: 'list' object has no attribute 'savefig'
The console displays this error: ERROR:tornado.access:500 GET /data?ds=Month&y=passengers&file=LHR-BKK2.csv&plotComponents=true&futureDurationValue=1000 (127.0.0.1) 2569.81ms

[pravj]

Awesome!

Can I have the file LHR-BKK2.csv?
That will help me debug the GUI error.

[kwtneoai]

hey there pravj, am interested in trying this out. I have managed to install the server and the client is running. However when i try to upload a timeseries file, nothing seems to happen. how do i get it to work? can i just put the file into the uploads directory? what format does the file need to be?
thanks !

[kwtneoai]

ok i get an error message as it is tyring to load the file - TypeError: b'TimeStamp' is not JSON serializable
ERROR:tornado.access:500 GET /column/fx.audusd_D1_PTSRPout.csv (127.0.0.1) 48.00ms
INFO:tornado.access:200 GET /filedata/fx.audusd_D1_PTSRPout.csv (::1) 1696.17ms
What format do the dates need ot be?

[kwtneoai]

NVM. i fixed it. the encoding was off.
i removed the encoding call in column_handler like so:
res = {'columns': [{'value': col, 'label': col} for col in df.columns.tolist()]}
and it works now.

[pravj]

Hey! @kwtneoai,

Glad that you've got it working.
I missed it because you didn't mention me in the comment.