We're very happy to have Round 9 completed. The 10-gigabit 40-HT core hardware has been quite amusing to work with. In most respects, these servers utterly humble our in-house workstations that we've been using for previous rounds.
If anyone has any questions about this round or thoughts for future rounds, please let me know!
I'm curious, would you be interested in microbenchmarks for things like concurrency models? I'm thinking about preforking vs. thread-per-request vs. events for C++, NIO vs InputStreams vs. Fibers for Java, etc. The idea would be to measure how much overhead is coming from the OS vs. the language vs. the framework so that framework authors could make informed choices for their framework architecture. Or would that be out of scope?
We are comfortable accepting and including what we call permutations.
Clearly, our project is about testing web application frameworks and platforms (we just say "frameworks" for conservation of typing). Testing NIO vs. InputStreams, for example, in isolation is a different project.
But as part of a web framework—let's say you had a framework that provided support for both NIO and InputStreams—we'd be happy to include the multiple permutations.
Obviously, this is within reason. If you submitted 512 permutations, I might have a small heart attack. Also, a general principal that we ask all participants to bear in mind is that each test should represent a viable production-grade configuration. This guideline might rule out a great number of wilder permutations.
If anyone has any questions about this round or thoughts for future rounds, please let me know!