We're trying to build a common database for people building these tools; at the moment, trawling through CVE disclosures and various mailing lists is a largely manual process that we can reap economies of scale by pooling our efforts together.
It's free and volunteer run. I would like to encourage you and anyone else reading this who is interested in bolstering the security ecosystem to consider using and contributing to the advisory database.
It's the database that powers http://github.com/rubysec/bundler-audit and (disclaimer: I made this) https://gemcanary.com and the more people contributing the more we can all benefit from improving the ways we can notify end users of their vulnerable dependencies.
I believe [Code Climate](https://codeclimate.com/) will report vulnerable gem versions in your app. Their marketing page suggests that they also analyze how you use a vulnerable library to see if you're affected or not.
Yeah, like I mentioned above the app I built (https://gemcanary.com) automates disclosure notification, and I believe? https://codeclimate.com/ may use the advisory database as well, but I do not know for a fact.
There's a few others but they've slipped my mind at the moment.
I use bundler-audit when performing security audits for clients. You could wrap it in a git commit hook and have it publish results, but to my knowledge no one's done it that way.
Is there any way to run these scanners locally? For example, is there something blocking the ability to simply hook Gem Canary into say a git hook or similar local action, and checking Gemfile.lock with the scanning service?
I'm not super-comfortable connecting private Github repos to remote web services. Github's (and other applications) recent OAuth exploits aren't very comforting and I'm generally the paranoid type.
Bundler audit is entirely local; you install it as a gem, and it fetches a fresh copy of the database on its own.
Regarding Gemcanary, we're actually in the midst of building something just like that because we deeply hate storing Github tokens - but the ease of use was too good to not go for with our proof of concept. Once we release the new feature it'll be our goal to move as many people as possible away from giving us access to their private github repos.
Codesake-dawn gem (I wrote), and brakeman scanner are the ones you're looking for. You can run locally on your projects and having some remediation hints.
I think the big thing here that threw me for a loop is just how many gems are pulled in by projects. 133 seems like a ridiculous amount of dependencies for a single project. Our entire company runs on Python in the backend and we don't even have 50 (and that's including ones I use for command-line utilities or corncobs, for example, which aren't in our public-facing software).
Even if the Python modules we have had the same number of vulnerabilities as gems, our exposure for a given system would be 1/10th, just from not using so many external libraries. Couple that with the fact that many of those gems are outside of end-user control, of generally unknown quality, might only be pulled in for one small piece of functionality, and might themselves depend on other gems which you weren't necessarily aware of and haven't necessarily vetted.
I think part of it is that people tend to separate things out into lots of gems. For example, if you're using Ember in your Rails app, that's two gems right there just for that library. If you want to allow delegated authentication with the common providers (Facebook, Twitter, OpenID), that's like 10 gems. Your unit testing and error reporting comes from gems, your app server is a gem, every database you want to support is at least one gem, etc. It gets explosive, but it's really just an organization thing. One thing to keep in mind, though, is that not all gems are actually used at a given time. Your gems for testing, development and production all go in the Gemfile, but you're probably not going to install all of them on one system.
Not all gems are created equal. Large amount of gems don't necessarily mean that they are arbitrarily included. In fact, it's often better to pull in a dependency for things that specialize in a particular task because they are more likely to have hardened security and better performance.
The flip side is actually knowing that this "meme generator" gem you are using actually pulls in 50 dependencies, many of which are terribly implemented. I've seen this problem just as rampant in the Node.js and jQuery worlds.
As developers, we should be as aware as possible of the state of our critical apps dependencies. I've recently gone through an app that had over 50 gem dependencies and reduced it to under 20.
One thing that's probably not considered is that testing frameworks (rspec) have tons of dependencies that are not scoped to production. I'm not sure if these reports considered that.
It would be interesting to know why this is. Are each of these dependencies really small on average? Or is there some big cascade of dependencies? Or is there a particular framework that's to blame?
I've had a similar issue when using the Spring Framework in Java, where including it in your project at all brings a whole host of other (often not very good) libraries in. This is a problem in Java, which doesn't have any built in capacity for loading multiple versions of the same library.
In Python, I suspect that I've been in a similar situation when using things like Pyproj, Numpy or Scipy, because those all depend on components written in C or Fortran. Perhaps next time I use any of those libraries I shall take some time to read more of the code and try to trace out what things they may be brining in.
> Are each of these dependencies really small on average? Or is there some big cascade of dependencies? Or is there a particular framework that's to blame?
All of that, really. For better or worse, scores of gems have dependencies. Rails and a few popular others add large dependencies that cascade to other dependencies at that — some large, some not. It's extremely rare to run into quality gems with very few if any dependencies. (Sequel comes to mind, and that's just about the only 1st-class gem I can think of in pure Ruby.)
A lot of them really are small. There is a tendency for Ruby libraries to be separated into "core" and "feature" gems that you can pull in separately, For example, if you want to use DataMapper, you might be pulling in dm-core, dm-rails, dm-validations, dm-migrations, dm-timestamps, dm-postgres-adapter, dm-mysql-adapter, dm-is-searchable…
I agree. The fact that a large number of Ruby/Rails apps fall behind in terms of keeping dependencies up-to-date speaks more to Ruby's adoption than to its inherent insecurity. Without comparing these figures to other languages/frameworks (and accounting for flaws in completely custom-built apps), it's hard to draw any real conclusions here.
This doesn't surprise me, and I'm not even an experienced Rails developer.
I think a big part of the problem is that there are a ton of gems that are simply hobby projects that gained traction and became popular. They were originally architected by enthusiasts, rather than experts. Some of these projects may also have been abandoned by their authors, but still in use because they may be the only way to accomplish a complex task or integration. You can't expect stuff like that to be very secure. You just have to think carefully about whether the risks of using them are worth the gains made from not having to implement the functionality yourself.
This can be said of any open source library, but it is unfortunately true that the average Ruby gem seems to be more hobby-grade than enterprise-grade.
The fact of the matter is, if you are developing an application and are integrating 3rd-party code, always pay attention to what that 3rd-party code is, how well-maintained it is, and how the project has responded to bugs and vulnerabilities in the past.
Yup. I've done enterprise Rails consulting, which is almost a contradiction in terms. But it's possible and I've brought fixes like net ldap to work with AD.
Edit: Can't take all the credit, client hacked together the first version but we managed to extract it for "the greater good." Also fun stuff like XML-RPC (Xen API -> gem xenapi, VMware (gem rbvmomi)), which wasn't bad and worked OOTB. Wished MS exposed their APIs as RESTful endpoints, because WinRM + gem winrm just doesn't cut it with some products... generated powershell run by an agent instead. For some products, even having the (.someextiforgot) files that describe the API, there's no MS docs on them, so lots of trial-and-error in PowerGUI to find the right objects and methods (Yuck).
Yes, definitely. I got burned by this very recently myself. I'm developing a Rails app (my very first!) in which users need to be able to enter and edit content in a WYSIWYG editor. I found a very popular one that also had a great user experience and started using it. Once I tried to deploy it though, I ran into various issues - turns out the app was quite out of date as its author had moved on to other things a year and a half ago. So I had to rip it out and switch to an alternative.
I'm now much more cognizant about stuff like how many contributors a project has and whether it is in active development.
Generally: don't trust anything from the outside world or anything that can transit untrusted infrastructure, that means check types and sanitize values before passing along. Break loudly and quickly to get attention for a fix. Keep the codebase as tiny as possible too.
Ruby: recompile with minimized OpenSSL 1.0.1+ (LibreSSL when possible) and with patches that improve Ruby's default OpenSSL security.
The graphs of Gem distribution and Vulnerability distribution are utter nonsense. You can't just measure your mean and standard deviation and expect a bell curve that fits those two parameters to correspond to your distribution. Especially if you cut off one of the tails.
The current graph is actively misleading because it misrepresents the data as following a (truncated) normal distribution when that's very unlikely to be the case. A histogram would show the actual distribution.
The kind of graph being used here -- a "best fit" bell curve based on the sample mean/s.d. -- can be useful when overlaid on a histogram, to illustrate how closely the data can be approximated by a normal distribution. Without that context, it's essentially meaningless.
I am not a ruby developer, but it does seem shocking that 66% of all gemfiles contain a known previously-disclosed 5+ exploit. Yes, most software is flawed, but you would think once it's been reported, it would be fixed.
First one is finding, reporting and getting vulnerabilities.
Most people don't perform security audits. Most vendors take forever to reply back to security reports. Infuriatingly, many vendors will roll security fixes into the next major release instead of backporting patches and minor versions.
The second one is about finding out about disclosed and patched vulnerabilities.
Outside of larger institutions where you have someone whose job it is to worry over configuration management and subscribe to every mailing list, the ecosystem for disseminating this information is broken. That's why we've started the https://github.com/rubysec/ruby-advisory-db, at least for the Ruby ecosystem.
It appears that you failed to read the data correctly. He says that 66% of the Gemfiles examined contained any vulnerability and 13% of them contained a 5+ issue.
That doesn't even mean an exploit btw. Some of those gems might be in the Gemfile but never actually used (deprecated but not removed, hence not updated), or the vulnerable component might never be used. The gem might only be used on internal data, not user-manipulateable data.
Furthermore, you can't extrapolate 13% of the examined gemfiles containing such an issue to all gemfiles, which you did.
The fact that it's reported certainly doesn't mean it will be fixed in all downstreams (what this article refers to). Do you read every CVE? Every single one? Didn't think so. Most gems are relatively unpopular and any issues in them won't be widely publicized. Sure, Rails issues are shouted far and wide, but most of the rest are easy to miss.
Hell, some CVEs don't even get fixed in the gem itself, let alone all the consumers of that gem; the gem just remains vulnerable because there's no maintainer or the maintainer insists that it's "not an issue".
Please don't make such misinformed comments without even reading the article with sufficient attention to detail to get the statistic right.
The gems have probably been updated by the developers of the gem.
The data used in the article contains a list of the versions being used in production by real app, not the latest versions released by the gem developers.
It's possible some of these are false positives, at least with regards to Rails, as most of the recent vulnerabilities I can remember come with workarounds and patch files. Some people need to stay on a specific Rails version, but may have fixed the problem.
I'm not ready to speculate which is more probable.
Hi there I'm the man behind the codesake-dawn security scanner for ruby code. It will be great having some of you comparing the results obtainw with haikiri or other scanner and mine. Just for sake to reach an enterprise level tool.
This article kind of confuses me - what does it mean for a gem do be 'secure' ? The idea of many gems being 'secure' or 'non-secure' is a non-sequitur. Obviously for some projects, like Rails, it's fairly clear what is meant when someone talks about a vulnerability - that you can hacked somehow. However, if someone wrote a gem to wrap eval into a command line tool so they can use some random ruby commands or libraries from their shell, well, secure would be a non-sequitur, but also beside the point.
But this article does not analyze that. It's only scanning CVE's and most have already been patched when the CVE is issued. Of the 53 mentioned for Rails for instance, all have been patched.
I don't understand... The article first analyzes apps that use a certain number of vulnerable gems. Sure, most of them are probably patched, but it doesn't change the fact that certain apps use old unpatched versions of these gems.
The second part focuses on historical vulnerability data for Ruby gems.
> Sure, most of them are probably patched, but it doesn't change the fact that certain apps use old unpatched versions of these gems
Why is that news?? That is true across every and all software platforms.
> In what way any of it is misleading?
Because the article assumes or implies the conclusion that there are a huge number of vulnerable Rails apps. The only way to discover that is to not simply count how many CVEs have been issued for each gem, but how many systems in production have not been patched.
From the comments here, many people seem to assume this is an analysis of the number of unpatched vulnerable systems in production, which it most definitely is not.
It's not exactly news but it's nice to see it broken down in a cohesive way.
> Because the article assumes or implies the conclusion that there are a huge number of vulnerable Rails apps.
I think the article quite clearly indicates where all data points came from (Hakiri Facets) and then analyzes them. There is a post scriptum at the end that encourages readers to not evaluate their or any other projects based on the number of CVE vulnerabilities and goes into detail why it's not always a great idea.
This doesn't surprise me in the least, but the increase in 2013 is obviously due to increased security scrutiny, which is good so long as people act on that information.
As a side note, I'd be interested to see a similar analysis of popular Java projects.
I'm one of the maintainers behind the Ruby advisory database: https://github.com/rubysec/ruby-advisory-db
We're trying to build a common database for people building these tools; at the moment, trawling through CVE disclosures and various mailing lists is a largely manual process that we can reap economies of scale by pooling our efforts together.
It's free and volunteer run. I would like to encourage you and anyone else reading this who is interested in bolstering the security ecosystem to consider using and contributing to the advisory database.
It's the database that powers http://github.com/rubysec/bundler-audit and (disclaimer: I made this) https://gemcanary.com and the more people contributing the more we can all benefit from improving the ways we can notify end users of their vulnerable dependencies.
Thanks,