ToolsLib recently switched to a brand new dashboard. However, behind the scenes, there was another switch happening. We used block based filesystem for underlying storage. However, as we grew, the filesystem was proving difficult to scale.
So, we were looking to move from block based storage to an object store system, when we found Minio. After initial analysis, Minio looked perfect for our requirements and we switched to Minio.
Not only the switch was smooth, team is very happy with the new storage system. We thought this was a great success story for open source software and needed to be shared with our community.
Situation
We used to store all of our unstructured data on a block based filesystem. While the filesystem served us well initially, as we grew it was proving difficult to scale. Also, the block based filesystem was difficult to synchronize in a reliable way between several servers, creating a bottleneck in our overall operations.
Solution
Object storage is more suited to organize our data, a bucket represents a project which stores several files — it was much easier for us to manage using this abstraction than using traditional folders.
Currently, we have deployed a single Minio server with 16GB RAM, running two SSDs in RAID1 configuration. Minio server already hosts several GB of data, and manages between 10 to 50 requests per second. Since the software stack is in PHP, we use the AWS PHP SDK. The upload is using multipart with a filesize up to 1.5GB and a stream on the download side.
Some of very popular software in our library create huge request peak in very few seconds. To handle that load efficiently and monitor it pro-actively we use mc events and mc watch.
The migration has allowed us to simplify our storage management, while giving flexibility to develop services.
To learn about other projects/companies that use Minio, check out Awesome Minio.
- Read our previous blog post
- Follow us on Facebook
- Follow us on Twitter @toolslib