More Erlang frameworks

[b0oh]

I added three more Erlang frameworks: Mochiweb, Yaws and Misultin. Will be glad if someone can run benchmark, because I haven't possibility to run it. Thanks and have a nice day.

[b0oh]

Fixes: #547, #936, #937

[aschneider-techempower]

Assumes package is in directory erlang-misultin but lines 11-13 have a cwd of misultin. Same issues with the other ones. Pick one of the directories to use and rename as appropriate.

[b0oh]

done

[aschneider-techempower]

After fixing the issues that I commented about, I ran the test and it seemed to hang without any log updates - any ideas as to why that happened?

[b0oh]

Mochiweb is more priority, so better to check mochiweb.
If I run same commands avoid Makefile in terminal I have output like this: https://gist.github.com/b0oh/28019e30021b35dec8cc.
We can change setup.py like this and see.

subprocess.check_call("./rebar clean get-deps compile", shell=True, cwd="erlang-mochiweb", stderr=errfile, stdout=logfile)
subprocess.check_call("erl +K true +sbwt very_long +swt very_low -pa ebin deps/*/ebin -boot start_sasl -config priv/app.config -s mochiweb_bench_app -noshell -detached", shell=True, cwd="erlang-mochiweb", stderr=errfile, stdout=logfile)

Thanks.

[aidanhs]

Do we really want Misultin?
https://github.com/ostinelli/misultin - "Misultin development has been discontinued."

[b0oh]

Misultin is not developed anymore, but it's still using in production and it's really interesting in performance aspect.

[hamiltont]

Will be glad if someone can run benchmark, because I haven't possibility to run it

@b0oh - If you rebase onto master, and update the .travis.yml to include a link for TESTDIR=erlang-misultin, then Travis-CI will verify these changes

[b0oh]

@hamiltont thanks, done.

[hamiltont]

Looks pretty close, but not quite there. Click for the build details, and scroll to the rows for erlang-* to see the output logs. A quick look shows these two errors:

INFO:run-ci:ERROR: OTP release R14B04 does not match required regex R15B0[^1]|R16B$|R16B[^0]|R16B0[^3]|R16B03-1|17
INFO:run-ci:ERROR: compile failed while processing /home/travis/build/TechEmpower/FrameworkBenchmarks/erlang-mochiweb/deps/eunit_formatters: rebar_abort

and

INFO:run-ci:WARN:  Expected /home/travis/build/TechEmpower/FrameworkBenchmarks/erlang-mochiweb/deps/erl_bench to be an app dir (containing ebin/*.app), but no .app found.

[b0oh]

Yes, I am already check log, we can ignore last warning, bacause rebar will create *.app files at compile section. About first error I am not sure how to avoid that, what version of erlang do you have? I suppose R14, if so how I can request new version? Locally I tested my code with R16 and R17.

[hamiltont]

We are using this script to install erlang in our verification suite (which is ubuntu 12.04).

I hate this script, because we don't know what version of erlang is installed (e.g. if you run this script on 12.04, you will probably get a different version than if you run this on 14.04). If you have the know-how, I would reallllly appreciate a pull request that changes this to either 1) download the erlang source code and build it or 2) just wget a specific version of erlang and untar it into a static folder. I would be very happy to help you on thid. I know how to setup the PATH properly for the tests, integrate this change with our other codebase, etc. I don't know where to download a specific version of erlang, or if that will even work.

Additional Info: we are testing pull requestd on ubuntu 12.04 64bit, but we are running the benchmarks on ubuntu 14.04. So this scripting problem is quite relevant for erlang - you could successfully pass the verification, get merged into master, and then see round 10 come out with failures for all the erlang-* stuff because it used a different version. Obviously no one wants that, but I've got other issues to fix before I could personally take a look at fixing erlang.sh

[b0oh]

I was check https://www.erlang-solutions.com/downloads/download-erlang-otp, and as they saying this file should be like that:

deb http://packages.erlang-solutions.com/ubuntu precise contrib

and this file like that:

#!/bin/bash

RETCODE=$(fw_exists /usr/bin/erl)
[ ! "$RETCODE" == 0 ] || { return 0; }

sudo cp $FWROOT/config/erlang.list /etc/apt/sources.list.d/erlang.list

fw_get http://binaries.erlang-solutions.com/debian/erlang_solutions.asc

sudo apt-key add erlang_solutions.asc
sudo apt-get -y update
sudo apt-get install -y erlang

Note I change esl-erlang to erlang.

This should install R16B version.
Many thanks for your attention.

[hamiltont]

I'll update it in a bit. Could you tell me a one-line command to print out the erlang version? I will add taht in so the exact version is in the logs

[b0oh]

$ erl -eval 'erlang:display(erlang:system_info(otp_release)).' -s erlang halt -noshell

[hamiltont]

I'm working on fixing up the erlang install in my branch, this is blocked until that's working

[b0oh]

Ok, thanks.

[hamiltont]

I am officially giving up on getting Erlang to work properly at the moment - perhaps someone is willing to merge in my branch and continue the work? No matter what I do I'm getting you compiled beam against version 153, but this vm only supports version 152. I think @msmith-techempower has an idea how to fix this. I frankly know nothing about the erlang language or tools and have tried everything I can think of--short of learning more about Erlang (which I don't have time to do ATM) I don't think I can help anymore

[hamiltont]

My only word of caution: Travis-CI comes with Erlang installed, so you sometimes get a pass result that you cannot trust because it used the built-in version of Erlang and not the one we installed in erlang.sh. It has to use the version we installed in erlang.sh or it won't run during the actual benchmark round!

[msmith-techempower]

All of the Erlang tests have their own beam executable (the Erlang VM), and it was probably compiled against an older version of the Erlang language (similar to trying to use .class files compiled against Java7 in a Java6 JVM).

My best guess (and we should get an Erlang expert to confirm this) is that we need to remove the beam executable from each Erlang test directory and add it to the install.sh files.

[hamiltont]

@msmith-techempower this issue sounds related to #995 in that we could benefit from an isolated environment for each Erlang framework

[hamiltont]

(in the long term...for now if we can at least get it to work that's a solution ;-) )

[msmith-techempower]

I don't know if it's related to #995 or not, but from what I can tell, but the analogy goes as follows:

beam is to Erlang as JVM is to Java

If the jvm is globally installed, then it stands to reason that beam might ought to be as well.

[hamiltont]

@msmith-techempower also keep #495 in mind as something that could potentially be closed concurrently if this problem is resolved

If the jvm is globally installed, then it stands to reason that beam might ought to be as well

JVM is only globally installed because it's easier than a prefix install, but the same thought process works for beam too -- if it's easier than just do a global install at the moment ;-)

[msmith-techempower]

Need to retest this since it's been over half-a-year and we may have addressed the rebar issues.

[b0oh]

@msmith-techempower nice to hear that

[msmith-techempower]

I am working on a branch that simplifies the setup process quite a bit, but I'll pull this branch and see if I can't get it working.

[msmith-techempower]

Pinging @waiteb3 as our resident Erlang expert to try and update these new frameworks to the new suite.

[msmith-techempower]

@ssmith-techempower We have fixed a lot of our issues with Erlang and Rebar, could you pull these changes, fix the merge conflicts, and test to see if we are ready to go here?

[b0oh]

sure, will give a look

[ssmith-techempower]

Looks like these haven't been updated since we moved away from using Python scripts to stop/start the frameworks. At least the following needs to be done:

• Move each framework into FrameworkBenchmarks/frameworks/Erlang/<framework-name>
• Rename the benchmark_config files to benchmark_config.json
• Consolidate the install.sh, setup.py, start.sh scripts into a single setup.sh bash script. This should have the fw_depends line, building the app, and running it. Frameworks don't need to be explicitly stopped anymore so that code can just be omitted.

You can look at cowboy for an example of an Erlang framework that works with the current tooling.

[NateBrady23]

Unfortunately, rebar now breaks with the version of yaws attempted here, but I will keep working to find another solution. Closing this now in favor of #1849 and #1850.

[b0oh]

@ssmith-techempower, @nbrady-techempower I lost my interest in benching erlang frameworks and as you used my code, I would like you to copy an erl_bench dependency as well that I will be able to remove it from my account. Thanks.