Directory structure re-organization

[bhauer]

With so many framework implementations (it's awe-inspiring, really), the root directory is a little unruly.

Assuming it isn't especially difficult to do so, I'd like to re-organize the root directory structure slightly:

• /frameworks - Move all of the framework directories currently found at the root into here.
• /config - Leave the config directory at the root.
• /results - Leave the results directory at the root.
  • /round6
  • /round6/final - Wrk logs from the final version of Round 6
  • /round7
  • /round7/preview1 - Wrk logs from a preview 1 of Round 7
  • /round7/preview2
  • /round7/final
• /toolset (or /harness or /scripts or whatever) - The Python, Bash, and Windows Powershell scripts.

Leaving just README.md and .gitignore files at the root.

Thoughts?

[kppullin]

I like it. I'll further throw out the idea of grouping items in the /frameworks directory by platform/language.

• /frameworks/jvm
• /frameworks/python
• /frameworks/dotnet
• etc

[fernandoacorreia]

I agree. Regarding /toolset, I was going to suggest to create a directory for installation / execution scripts. But I think we should take it a step further and subdivide the /toolset directory by category.

The Azure deployment scripts I'm working on will require a directory of their own. I can envision a similar deployment script for AWS which would create the instances and automatically setup the environment. Those two environments could share some code (what is executed after the instances are created). This common code could be used to automate deployment for other environments (e.g. Rackspace, on-premises Microsoft System Center, etc.)

There is also setup for Linux server and client, Windows server and SQL server, each with their own bootstrap and auxiliary files. It's easy to see the /toolset directory getting out of hand soon.

One attempt to organize it:

/toolset: README.md for the toolset, run-tests.py
/toolset/benchmark: benchmarker.py, framework_test.py, latest.json
/toolset/deployment/aws: AWS deployment scripts
/toolset/deployment/azure: Windows Azure deployment scripts
/toolset/deployment/common: shared deployment scripts
/toolset/setup/common: installer.py, setup_util.py, unbuffered.py
/toolset/setup/windows: installer-bootstrap.ps1, installer.ps1
/toolset/setup/sqlserver: setup-sqlserver-bootstrap.ps1, setup-sqlserver.ps1

[bhauer]

@kppullin I like that idea of grouping by platform. It would further reduce the clutter in any single directory and help people find what they are looking for.

@fernandoacorreia That plan of further organization within the toolset directory sounds great too. If you have time and interest in making the change, I'd be happy to see it. If not, we'll get to it over here when we can find some time.

Thanks!

[fernandoacorreia]

I've made an initial pass at restructuring the toolset directory. You can see the commit in my fork and the forked branch.

I'm currently testing the changes and they may need a few adjustments in scripts due to changed directory names.

You'll notice that the "deployment" directory seems pointless now, but it will soon receive automated deployment scripts I'm working on.

[bhauer]

Looks like great progress. Thanks again for your continued contributions, @fernandoacorreia!

[bhauer]

Updated issue description at top to include a proposed layout for the Results directory.

(Aside: I don't know how to do more than 2 levels of bullet indentation in Markdown!)

[serroba]

I like the idea, but I think there should be as well a directory for the implementation using the nake language without framework

For instance, the PHP fremework test are using as base the implementation in vanilla PHP. I think in general is a good idea to have the vanilla implementation and a place to hold the frameworks, which should as well be aggregated on base of the platform/language

[methane]

@serroba I think frameworks/php/raw or frameworks/php/vanilla is enough.

[serroba]

@methane that's as well a good idea.

Actually the name raw is used across the project when using the DB against ORM or Raw.

I don't mind the name, but I mind that should be easy to read/understand for anyone. What I would like is that someone that is landing in the project, can start to write code quick and not to have to spend long time just trying to understand the tree and how things are organized.

[bhauer]

About "raw," @serroba makes a good point: I'd like to avoid using that term for anything other than the means of database connectivity to reduce confusion. To-date, we have used the word "plain" to refer to a platform without a framework. E.g., plain PHP, plain Servlet, plain Netty. But I would be okay with "vanilla" as @methane suggested as well.

However, I would prefer to organize the /frameworks directory by language first, and framework/platform second. Platform is a looser concept than language, and the precise definition of platform varies by language.

So for example, I would like something like the following:

/frameworks/php/plain   (or /frameworks/php/vanilla)
/frameworks/php/laravel
/frameworks/java/servlet
/frameworks/java/netty
/frameworks/java/spark
/frameworks/java/wicket

I think this would be awesome. Thoughts?

[serroba]

Although I like the most the term vanilla, I still remember a couple of
years ago that I didn't understand what they were talking with the vanilla
'preposition'. As I no native English speaker I remember that I needed the
explanation why vanilla.

Plain on the other hand is self explanatory. So thinking in the non native
English speakers I think I would vote for plain instead of vanilla.
On Jul 4, 2014 1:30 AM, "Brian Hauer" notifications@github.com wrote:

About "raw," @serroba https://github.com/serroba makes a good point:
I'd like to avoid using that term for anything other than the means of
database connectivity to reduce confusion. To-date, we have used the word
"plain" to refer to a platform without a framework. E.g., plain PHP, plain
Servlet, plain Netty. But I would be okay with "vanilla" as @methane
https://github.com/methane suggested as well.

However, I would prefer to organize the /frameworks directory by language
first, and framework/platform second. Platform is a looser concept than
language, and the precise definition of platform varies by language.

So for example, I would like something like the following:

/frameworks/php/plain (or /frameworks/php/vanilla)
/frameworks/php/laravel
/frameworks/java/servlet
/frameworks/java/netty
/frameworks/java/spark
/frameworks/java/wicket

I think this would be awesome. Thoughts?

—
Reply to this email directly or view it on GitHub
#448 (comment)
.

[hamiltont]

@jyentechempower I think closing this is a bit premature - we should ideally wait until a directory restructure has been completed and tested. #915 make doing this a lot simpler, because you can start using variables instead of hard directory positions, but it doesn't actually restructure the folder locations at all

[hamiltont]

Some thoughts on this:

• Using language-specific directories would make it feasible to setup per-framework package management tools. For example, Perl installation script is too big #966 discusses using Carton to ensure each framework gets to declare it's own Perl modules. Options exist for isolating environments in a variety of languages, but right now it would be tough to set this up without massive confusion. With the current setup, it's entirely possible that framework A runs fine on a clean VM, but does not run at all if framework B is installed first
• I would love to see huge chunks of the main README moved into correct directories (e.g. into frameworks, frameworks/python, etc)

ps - @fernandoacorreia were you ever able to complete/merge your work on automating the deployment? That would be amazing...

[msmith-techempower]

@hamiltont

With the current setup, it's entirely possible that framework A runs fine on a clean VM, but does not run at all if framework B is installed first

This is terrifying - can you give an example where this is true? I expect, apparently incorrectly, that each test is sandboxed with regard to everything except its dependencies (like nginx, java, etc).

[hamiltont]

@msmith-techempower Given that you excepted dependencies, I'll give a contrived example for a potential problem that is not dependency (meaning some script we run under /toolset, not dependency in the broader sense) related. I don't have an example case where this happens currently, but that might just be because these issues would be very hard to spot - they would just look like a normal test failure, which we have a lot of at the moment.

contrived example : mojolicious currently requires the perl module Mango version 0.30. If a new test is added (mojo-orm or something) that requries mango 0.9, it's install.sh will install 0.9. The install.sh for mojolicious wouldn't roll back the mango version, it would just think everything is ok because a newer version is installed, and bam failure when you run mojolicious.
Disclaimer: I don't think this is actually possible for mojolicious or our other perl modules, because @jberger setup some nice perl module management stuff that I believe can declare "I specifically need mango 0.30"

bonus round : You may have noticed me complaining in general about installing stuff to the local system instead of installing stuff to a separate folder. There's a good reason, this causes potential for the exact same error as above, but for our dependency scripts. Pop up an ubuntu 12 box and run fw_depends mono nginx and watch the build fail with gcc: internal compiler error. Fun to do! But yea....bad for running our suite.

To completely invalidate my sentence, we need (1) all of our dependency scripts to install stuff into folders, and we need (2) to use language-specific package management solutions (I say environment in some places because a few languages can even change things like environment variables on a project-by-project basis). (1) is pretty simple, I think it will happen naturally as we're closing PRs. (2) is also pretty simple, but it depends heavily on closing this issue. Without a clean directory organization, (2) is the path to brimstone and dragons as you start changing little bits of toolset/ all over the place to detect what package management solution you're using

[hamiltont]

organize the /frameworks directory by language first, and framework/platform second

This isn't supported yet, but it should be fairly simple to add in now. That PR was just getting too big to continue work on

[bhauer]

Awesome to have this one closed!