Don't use a filesystem as a database. We did that back in 1998 mostly just because we were using 1998-era computing power, and the more static content you could serve the less likely your box would get hosed by Slashdot. Now my watch has more computing power than those old servers. Use a real database, even a file-backed one like BerkeleyDB (or god forbid, SQLite)
"God forbid, SQLite"? It's honestly a great product and a wonderful stepping-stone. I wish more new projects used it rather than half-measures like Mongo.
100% agreed, it can hand a few hundred thousand hits a day. Obviously not okay for HN scale - but it's what my personal site and a bunch of side projects use.
I used SQLite in my side project few years ago. It was very simple (like 2 tables/collections), but a lot of updates every day. I eventually switched to MongoDB because SQLite was too slow (and also I wanted to play with NoSQL stuff).
A curious example from my memory. Subversion VCS circa 2005 initially provided two backends: Berkeley and file system. The everyday Subversion usage proved that file system backend was more reliable, more scalable, easier to manage and less prone to occasional data inconsistencies. So, file system has its practical use as a DB.
Another prominent example is Mercurial SCM. It uses a fine breed of file system and custom index files and shows miracles in terms of reliability and performance across variety of platforms.
And with SSD's and HDD memory plus OS disk caching, HDD storage can be very fast. It works great as a simple key/value storage. Duplication (read slaves) and sharding also works great. Modern file systems such as ZFS also helps speed things up and provides nice features like "snapshots" that isn't even possible in modern databases.
You are probably downvoted because you are essentially saying "Use a database for the sake of using a database!" which is not very good advice. Use the best, simplest tech your problem needs. If a filesystem is sufficcient, that's really good because it's stupid simple and cheap to operate.
A filesystem is a filesystem. A database is a database. We have different words for things for a reason.
If you wanted to compare a filesystem to a database, it's more a hierarchial collection of tables whose columns are the file stat() with a binary record at the end of each row. But it's somewhat of a useless database because its features are all oriented at its storage backend, not accessing the data. A real database allows you to optimize queries and search, index, join and select data at once and efficiently handles the more complex aspects of accessing and writing so you don't have to implement it yourself.
If you need a database, _use a database_. Tech hipsters need to stop badly reinventing the wheel.
No, I was saying to use a database because that's what the OP was asking about. And I got downvoted because people think using the simplest tech possible is a good idea. If that were true, we wouldn't be using a containerized virtualized app written in a high level language. Stupid simple now means it's a pain in the ass later.
So you have a post on a forum. You can just save that to disk by itself and access it later. And here come the problems.
How many posts are there? Filesystems commonly have problems dealing with too many files, like inode limits, slowness reading large directories or in deep paths, slowness traversing large trees, path name max size, etc. Now you have to engineer a bunch of hacks around the filesystem, or try to find a perfect filesystem and be stuck with it.
What if you want to search posts? Well, you have to open and read all the posts. But what if you want to speed that up, like with a word index? Congrats, you're about to write a database/search engine.
What if dynamic processing is too resource-intense and you want to serve static content? Once you apply transforms to the user content and create your static resource, you get to decide if you keep the original content, because it's not useful to you just sitting on disk.
What if you want to merge, upgrade, convert, export, etc parts of the data? You have a big job ahead of you if you didn't keep the original content, and if you did, I hope you kept metadata to make this task easier.
What if you want to spread the load, store records efficiently, control I/O methods and cache, support strong locking, platform independence, and storage backend independence?
Databases are a powerful and simple tool to add features to an application and make it easier and more efficient to process data. If you don't want app features, data processing features, platform independence, etc then don't use a database.
What about the converse question? Why does the filesystem not provide indexing, tabulation, relationships among document, archival tools, tools for merging two diverged versions of a folder tree, etc.? (Is there one that does it? I don't know!). E.g. Why can't I store a slidedeck by slides and show a "document" that is composed of transcluded bits and pieces? I think pondering alternate systems from time to time is valuable.
