As I mentioned at tonight's meeting, I have mostly assembled a new
file/virtualization server to replace Fastor/Schizo/Hubuntu. This will
also be the home for our new authentication server and voting system.
One machine replacing three should significantly reduce the noise from
the server corner (especially as I have it set up properly with fan
control).
Currently we have 4 1TB drives in a Raid6 configuration, yielding
something like 1.8ish Terabytes of usable space. There are 4 empty slots
available. I was originally simply going to suggest buying 4 more 1TB
hard drives but it was suggested to me that we could instead buy larger
drives now to avoid upgrading later.
This would be set up as a freely available space for members to
store/share files (within the bounds of the law of course) and for
storage of files from shared projects (i.e. pictures/raw video from
makerfaires, ptdr, etc.). One of our visitors at the meeting tonight
indicated that he had ~5TB of videos of panel talks and such from
various hacker conventions. It would also be a virtualization server so
having a large chunk of hard drive space for that would be helpful. I've
heard lots of ideas floated about what we could do with this space.
Based on this I am proposing three options, with a vote for each:
I shall go ahead and cast my vote, I really like the reliabilty that raid 6 provides us.
Vote 1 and 2 both consist of the most "cost efficient" purchase of the hard drives. I think at this time in our technology era it's silly to purchase 1TB drives its cost per GB is around double. If it was me I probably wouldn't do raid 6 for vote 2 but, you are the one doing it so who am I to judge .
I vote yes for Option 1, I wanna get those videos! I probably don’t have the background to understand some of them, but I’ll still learn something! And this machine is going to be replacing multiple machines, The 1 TB drives can be used in other machines at the hive, or be used as backup copies for them.
I was thinking in terms of later upgrading the remaining drives. I
suppose we could set it up as a raid 5 and upgrade it to a raid 6 later
when adding more capacity, but this is a serious pain in the ass and (i
think) would require the array being offline while it's working. I'm not
100% sure on that though, I've never had to convert between raid levels
before. Someone more raid savvy than me please chime in if I'm wrong.
As someone that is currently in a project at work to clean up the network storage, the idea of paying money to immediately burn up 5TB of storage on data of unknown value that will likely sit around and collect digital dust is not very exciting.
For a low investment of $300 we could obtain roughly x5 the total combined amount of storage we currently have across all the systems. How much free space do we currently have in our 1TB of storage? And how much of that is not stale data? Given what I can see on the current machines that exist, we don’t even need to purchase more space than the 4 1TB drive you currently have. The argument to add 4 more drives now at the creation of the the storage array is makes sense though, which is why I would support that.
I understand the desire to get a bigger e-peen, and routinely feel the need for a larger one as well. 24TB is a respectable digital phallus, but I can’t see a reason we need that at the space for a significant cost. Given the current systems we have on the network, the virtualization of them will take a trivial amount of space and there are better locations to store something like videos that would be accessible to more people and could be easily streamed to the site. Digital Hoarding is just like regular hoarding, except you don’t visually see it as often.
Will has a good point. Generally I store my files on the cloud (google drive to be exact), I suspect that a lot of other people do as well. My work flow is to work on something at home, save it to the cloud, pull it down at the Hive and create stuff. I generally need enough space to hold onto a file that I will likely delete that day, assuming I remember.
I believe the purpose of this was to also allow video and file sharing purposes not just “storage” but sharing of the information too. I don’t believe this is only to use for hive projects but more of a service to members (and maybe non members even). Additionally Ian was wanting to handle virtual environments, I believe anyways.
Ian, I know you already know about UnRaid. Can you share your feelings why this is the wrong solution for the Hive? It seems like this could be a good, economical, flexible solution because it provides redundancy and MUCH more flexibility in adding more space (drives) as needed.
Meh, “The Cloud” is just a fancy word for someone elses server. Local storage is cheaper, especially if you aren’t paying for the electricity. In the long run its not that big of an investment to avoid an upgrade later and we have plenty of potential projects that could use the space. “If you build it” etc. etc.
Also I forgot to mention in the initial email another planned use for this machine: I want to set up real automatic backups for the hives pcs. Scrounging for old copies of configs and such whenever one of our dust-laden machines inevitably dies is not fun.
If that’s the case, would we be better off with a Hive account to a cloud storage system? That would allow access to shared information that wouldn’t put an additional maintenance burden on the CTO, while being generally available to everybody. While I’m sure it would cost some money, for $1400 I suspect we could get a lot of storage for years, and not have anything stored on site.
While I still use UnRaid at home, this is largely due to the fact that it would be a pain in the ass to switch. Unraid is alright for file storage alone but it’s virtualization management is a bit lacking. Also the underlying OS, despite being linux based, is really locked down and hard to work with.
maybe the scope isn’t clear to me but currently we have:
1 TB storage (FASTOR)
200GB OS (FASTOR)
136GB (SHIZO) - currently available to people to virtualize environments as a lab?)
143GB (HUBUNTU)
Assuming that ALL got virtualized,
that is 500Gig +1TB storage.
The workstations are:
I believe you said that the new workstations had 250g SSD’s so if we back them do FULL backups of them if they are drives are COMPLETELY full, that works out to 1 TB
Laser:
CNC:
Electronics:
Lounge: (does’t really need to be backed up, but sure for the sake of argument.)
So we are at 1.5 TB to reproduce the current rack as is, and 1 TB in a worst case backup solution.
That leaves us at ~2.5TB used of the ~6TB available for $300.
Even if we went crazy and added another 1 TB to the pool for more room for members to play with vitualized machines (which commonly run around 20GB-60GB used disk space in a lab environment) we are at ~3.5TB used.
6TB - 3.5TB = +2.5 TB additional storage over the current 1 TB of storage.
Unless I am missing something I still don’t understand why it would be of value to spend more that the $300 that would give us more than 3x the storage and add capacity for between 20 - 50 virtual machines.
It is possible I’m missing something, and if so let me know, but otherwise I still don’t see the need.
Never overbuy on storage. By the time you need that additional storage, you can buy twice as much for half the cost. Over 4.5x the cost to purchase more storage than we will theoretically use before needing to replace drives due to their age.
Agreed… just spending money to get something that is not useful at the moment seems to be a waste… especially considering that when we get close to that timeframe, these will more than likely be at the end of their useful life anyway and we’ll be able to purchase 10x for half the cost.
Seems the question is how much we want to start storing more video, per Ian’s original post:
This would be set up as a freely available space for members to
store/share files (within the bounds of the law of course) and for
storage of files from shared projects (i.e. pictures/raw video from
makerfaires, ptdr, etc.). One of our visitors at the meeting tonight
indicated that he had ~5TB of videos of panel talks and such from
various hacker conventions
.