Release 0.10.0

[busbey]

Hi Folks!

I'm going to spin a release candidate shortly. If there's an in-progress PR that's close that you'd like to see included please try to get it in soon.

I'll comment with the current set of issues later this evening.

[busbey]

Changes since 0.9.0:

• [rados] Development of RADOS binding of Ceph FS #670 [rados] add binding for RADOS of Ceph FS
• Shell scripts to run YCSB. #737 [scripts] move to posix and windows specific shell scripts instead of python
• Fix a typo in Histogram generator #751 [core] correct calculation of weighted histograms
• Add batching in insertion in JDBC client bindings #755 [jdbc] add batched inserts to JDBC binding
• [mongodb] added logback.xml to mongodb module #758 [mongodb] added logback.xml to mongodb module
• [mongodb] return OK on updates with w=0 #759 [mongodb] always return OK status when mongodb-async uses w=0
• ycsb-0.9.0 missing cassandra2-binding #766 [cassandra2] ycsb prebuilt artifact doesn't include cassanda2 module (no commits)

#737 is going to require updates to our docs before final release.

it looks like we'll need to retest a minimum of the following: MySQL (via JDBC), mongo, mongo-async, cassandra2.

I'd like to get rados tested since it's new.

Anything else folks would like to see?

[busbey]

I'm not sure how I missed this in reviewing #737, but I now notice that the python script for running ycsb is still in place. Should we leave this in parallel with the dedicated scripts for this release? I wouldn't want to keep it around much longer than that.

[busbey]

RC1 is up, if folks want to start testing:

https://github.com/brianfrankcooper/YCSB/releases/tag/0.10.0-RC1

• @jaemyoun could you test out the rados binding?
• @allanbank will you have time to test out the mongodb bindings?
• @kruthar do you think you'll have time to re-check mysql with the recent jdbc changes?
• @cmatser think you could test out the new posix script?

Any volunteers for checking that the cassandra2 module works?

I'll get things tested on windows (both with the python wrapper and the new batch script).

[sekikn]

FWIW, I tried jdbc-binding with PostgreSQL 9.5.2 and its JDBC4 Driver. All workloads succeeded with db.batchsize=100 :) https://gist.github.com/sekikn/cc6c7d034bc3343253df7061c75fb1bb

[engineerakki]

I am testing the Cassandra2 module in 0.10.0RC1 and will update about it here.

[cmatser]

I'll runs some tests on the scripts. I'd appreciate it if everyone running tests would also try out the new scripts and report any issues. Please try out "ycsb.sh" instead of "ycsb", or "ycsb.bat" for Windows systems.

I wasn't sure weather to leave the python script there either. I agree, though, probably should not keep it around too long if we plan to switch to the platform scripts.

[cmatser]

I created pull request #777 to address issues I found when testing on Windows. It would be good to include for this release.

[engineerakki]

I am getting this when I run YCSB on Cassandra 2.1.0

[WARN] Running against a source checkout. In order to get our runtime dependencies we'll have to invoke Maven. Depending on the state of your system, this may take ~30-45 seconds [DEBUG] Running 'mvn -pl com.yahoo.ycsb:cassandra2-binding -am package -DskipTests dependency:build-classpath -DincludeScope=compile -Dmdep.outputFilterFile=true' Traceback (most recent call last): File "./bin/ycsb", line 295, in <module> sys.exit(main()) File "./bin/ycsb", line 268, in main maven_says = get_classpath_from_maven(project) File "./bin/ycsb", line 203, in get_classpath_from_maven "-Dmdep.outputFilterFile=true"]) File "./bin/ycsb", line 152, in check_output process = subprocess.Popen(stdout=subprocess.PIPE, *popenargs, **kwargs) File "/usr/local/lib/python2.7/subprocess.py", line 710, in __init__ errread, errwrite) File "/usr/local/lib/python2.7/subprocess.py", line 1335, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory

I think its a maven error, I do not use/know maven so maybe somebody else can test this.

[jaemyoun]

I am sorry for the delayed reply.
I tired rados-binding with Ceph Jewel (10.2.2), and all test performed
successfully.

On Wed, Jun 22, 2016 at 2:01 PM Sean Busbey notifications@github.com
wrote:

RC1 is up, if folks want to start testing:

https://github.com/brianfrankcooper/YCSB/releases/tag/0.10.0-RC1

• @jaemyoun https://github.com/jaemyoun could you test out the rados
  binding?
• @allanbank https://github.com/allanbank will you have time to test
  out the mongodb bindings?
• @kruthar https://github.com/kruthar do you think you'll have time
  to re-check mysql with the recent jdbc changes?
• @cmatser https://github.com/cmatser think you could test out the
  new posix script?

Any volunteers for checking that the cassandra2 module works?

I'll get things tested on windows (both with the python wrapper and the
new batch script).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#772 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AE444tFf0x2RVASMbr_t-7tBsJNtvIf1ks5qOMGigaJpZM4I6JXe
.

Jaemyoun Lee

CPS Lab. (Cyber-Physical Systems Laboratory in Hanyang University)
E-mail : jaemyoun@hanyang.ac.kr
Website : http://cpslab.hanyang.ac.kr

[cmatser]

@engineerakki The release candidate is a source code release. Try to do a maven build first. You can check that you have maven v3 installed with "mvn -v". Then build with "mvn clean package".

[busbey]

Okay, @cmatser I've got your change from #777 on the staging branch. I'll cut a new RC later today and then go through my windows tests.

@engineerakki if you need more help with Cassandra2, please let us know.

If anyone can take care of mongo or mysql, let me know.

[busbey]

RC2 is up:

https://github.com/brianfrankcooper/YCSB/releases/tag/0.10.0-RC2

[cmatser]

@busbey , I can run tests on mysql.

[cmatser]

Tested MySQL 5.5 on ubuntu with JDBC binding. All good.
ycsb.sh and ycsb.bat working as expected.

[busbey]

found #784, new script files not found in binary tarball.

[busbey]

on the plus side, the new batch file appears to solve #335.

[daschl]

@busbey as mentioned on the other thread if we can cherry pick 44660de and 5edb1d9 that would be great, and yes it requires 2.3.1 which is set on master (https://github.com/brianfrankcooper/YCSB/blob/master/pom.xml#L95)

[busbey]

That'd be including #773 and #786, right? Normally I'd leave out the sdk upgrade due to feature freeze (since 0.11 should start in mid-july), but if you're ready to run through testing on a quick turnaround I'll roll them into the next RC.

[daschl]

@busbey yes! thanks much - once merged in I can do the testing.

[busbey]

Release Candidate 3 is ready, should include the fix for the new scripts and the couchbase changes:

https://github.com/brianfrankcooper/YCSB/releases/tag/0.10.0-RC3

[busbey]

looks like testing to go:

• couchbase
• mongo
• cassandra2

I'd like to cut the release tomorrow evening.

[busbey]

anyone have a copy of the script @allanbank uses to test mongodb?

@engineerakki you going to be able to get to the cassandra2 testing still?

[busbey]

Draft release notes, with cassandra, couchbase, and mongodb moved into the "untested" category.

https://gist.github.com/busbey/0cb1bf8ad36fc9e4bbb0e2fe1f3b7507

[allanbank]

Sorry that I am slow to the party. I started the script last night but forgot to set Java to 1.7 for the build to work. I am re-running now with the script now and will let you know the results. It takes a couple of hours to run.

[busbey]

Sweet! Any chance of publishing the script so that others can share the task next time?

[allanbank]

Just put up a gist. No complaining about my bash skills.

https://gist.github.com/allanbank/3c89da1868bb25a16fdafb1786585af8

You can just copy it into an empty directory and run it from a shell and it will pull down everything from the internet:

sh integration-test.sh 0.10.0-RC3

I run a grep after the tests to check for errors from the same directory.

grep -P 'Return=(?!OK)' logs/*.out

I need to get a new computer because the script has only gotten through these MongoDB versions" 3.2.7, 3.0.12 and part of 2.6.12. No issues so far and I think that is good enough for the release.

[allanbank]

All of the MongoDB versions finished and look good. Versions tested:

• 3.2.7, 3.0.12, 2.6.12, 2.4.14, 2.2.7, 2.0.9, and 1.8.5 (async only)

For the release notes I would change the note for MongoDB from:

mongodb-async now returns status OK for all operations when at write concern 0, rather than the previous NOT_FOUND.

to

mongodb-async now returns status OK for all operations when at write concern NONE, rather than the previous NOT_FOUND.

'NONE' is the current/prefered name for 'w:0'.

Other than that the release notes look good to me.

[daschl]

Sorry for the delay on Couchbase, been traveling andd will complete testing on Monday!

[daschl]

testing done, found and fixed some regressions recently introduced, please see #789

[busbey]

I merged #789 and have cherry picked the changes back to the staging branch. @daschl would you like another RC to confirm that things work as intended, or are you fine with me cutting the release directly from the staging branch?

[busbey]

Also, what couchbase version number should I record as having been tested?

[busbey]

Thanks @allanbank! the script is great and I kind of want to start making something like that for all the bindings.

[daschl]

@busbey no need for an extra RC for me. Tested with 4.5 and 4.0.

[busbey]

okay, release notes updated for the couchbase2 fix, and the testing for couchbase and mongodb:

https://gist.github.com/busbey/0cb1bf8ad36fc9e4bbb0e2fe1f3b7507

I won't be able to grab my signing machine until later this evening, but I'll start making the release artifacts now. If someone needs something to make it in to 0.10.0 still, speak up soon.

[busbey]

release is up!

https://github.com/brianfrankcooper/YCSB/releases/tag/0.10.0

I'll start sending out announcement emails this evening.

[busbey]

README update: #792

[busbey]

anyone know if Rados has a user forum where I can announce the release?

[busbey]

I think we're past the reasonable window for announcing this release. I'm giving up on it. Let's just make sure the 0.11.0 release is really good. :)