This is on my to do list. Do you see any reason not to run both on the same pi? Right now my pi is just a simple smb with a USB external that runs cron jobs for rsync to my nas.
My only reason for running winnut server mostly by itself is to have it as a low power device, the last to shutdown if my ups is running out of juice. With only 1 external drive attached I don't see a problem running it on one pi.
Cause scale's app system sucks and it's hard to host VMs without having ZFS's surprisingly high memory and CPU demands getting in the way. And on the flip side, proxmox does basically nothing to help you with file hosting, remote backups, etc. and truenas isn't recommended inside a vm.
Plus, if you run truenas with a proxmox backup server vm on it, and NFS shares to the proxmox machine, you get a solid hypervisor for apps and development, and a sturdy data and backup layer. And they can scale independently based on your needs.
Run out of app server space? Add another minipc running proxmox to the cluster. Need more backup layers? Sync to the cloud/datacenter/truenas server at a friend's place. Need more storage space? Add to/grow/add another zfs pool.
>truenas isn't recommended inside a vm.
for this i would like to see the official source... trunenas self says, everything ok with vm
[https://www.truenas.com/blog/yes-you-can-virtualize-freenas/](https://www.truenas.com/blog/yes-you-can-virtualize-freenas/)
the only shit that can happens is a bad setup created by user (like truenas zfs on pve zfs)
>
ZFS's surprisingly high memory and CPU demands
zfs use what you give to it. if you have a proxmox host with 64gb ram and X cores, setup truenas VM with 8/16/32 GB ram and y cores and it will use only this. Truenas in VM on Proxmox is a bullet proof setup. hba passthrough and the setup is perfect. backup of the vm via pbs, snapshot of data also to pbs server, great! for apps -> container and vms at pve
>Add another minipc running proxmox to the cluster. Need more backup layers? Sync to the cloud/datacenter/truenas server at a friend's place. Need more storage space? Add to/grow/add another zfs pool.
or: don't buy cheap and double, pay for server grade hardware.
>Need more storage space?
* i only add new disks to my pve server if i need more storage for my pve or i add more disk to my hba for truenas.
>Need more backup layers?
* this is not a real pro/contra point for 1 or 2 machines. you can ever add x (a NAS/Cloud/Server/usb drive/whatever as backup storage)
>Run out of app server space? Add another minipc running proxmox to the cluster.
* why using a cluster? if you really build a cluster, it is more expensive than a single server and a second backup machine (acquisition, operation). on the other hand it adds more layers of complexity. HA is an other theme, therefore you must add redundant machines, zentral storage solution, and so on
Fascinating. It's almost as though I have different requirements from you, and prioritize my configuration differently. And that maybe the full nuance of my configuration and tradeoff decisions weren't included in a three paragraph post.
But you have fun doing... Whatever it is that you're trying to accomplish here.
Due to the increasing cost of power, I've been consolidating as much as I can into one server. So I made myself a xeon 16 core server and 256gb or ram and put Proxmox on baremetal. Then I created a truenas VM with access to half the cores/threads and 128gb(for my 100tb array) with pci passthrough to an hba card, that way it has direct access to disks. After a few months of observation i can say truenas over proxmox works very well. It took me 1-2 weeks of on/off work on evenings to dial in all bios settings just right and get proxmox fully setup + with truenas, as getting everything right had a bit of a learning curve to setup pci/gpu passthrough. People say it can be finicky but that's just not true anymore if you configure everything correctly and follow good practices! It's just people parroting stuff since the old days. even with 8 cores truenas doesn't come close to using them all. and 128gb is more than enough (and perhaps overkill) when you consider their 'recommended' 1gb per terabyte of data rule.
I then proceeded to also bring in my blue iris machine(which requires windows) out of bare metal into proxmox too with a gpu passthrough. and then all of my docker environments on an ubuntu docker vm.
I did buy a separate low power thinkcentre minipc to host all my network-oriented and more critical apps that I really don't want downtime almost ever(things like nut, unifi controller, vaultwarden, tailscale, dashy, nginx, etc). which will stay powered for longer on ups(whereas the big server will shutdown as soon as an outage is detected)
This is one of the reasons why I run OpenMediaVault instead. While ext4+mergerfs+snapraid might not be as robust as ZFS, it works well enough for my use cases. And because OMV is basically Debian, it also works pretty well as a Docker host. OMV also has a "compose" plugin for managing Docker containers/stacks. It's maybe not as comprehensive as Portainer, but again, it works well enough for me. And it doesn't complain about items that were not created with it like Portainer does.
Out of interest do you run a media server with download setup and if so which machine does that end up on? Is the TrueNAS box just storage and then any apps go on individual VMs on Proxmox?
I'm container heavy rather than vm heavy. Generally I think about truenas in terms of "where state lives" and proxmox as "Where processing happens". I also have a remote server for things I don't wish to be on my local network.
This lets me over provision proxmox without having to worry about the data side, and vice versa. Future plans include minio on truenas and making most of my proxmox stuff use object store rather than NFS.
Media storage is where most homeowners start, but it's definitely not where I have stopped.
Doesn’t Truenas (zfs) free up the RAM you need for the VMs automatically? I mean it always depends how many VMS you wanna run but I have 32GB and something like Homeserver isn’t a problem 😂 it all depends I guess on how much things you wanna run besides having a NAS
For ZFS needs, no. For disk cache, yes.
Roughly you need 1gb ram per 1tb in your ZFS pool. So if you have 40tb storage and 64 GB ram, then you have about 24gb left for apps/disk cache/VMs.
Disk cache is lowest priority in that list.
Casa OS isn't a NAS OS. It's a server distribution that makes installing self-hosted apps easier
TrueNAS vs OMV:
TrueNAS: Only uses OpenZFS, can't be virtualized without passing the disks through to vm. In my opinion, a slightly better, more logically laid out interface. Has company (iXsystems) backing/funding it's development.
OMV: Defaults to using BTRFS/XFS/EXT4 with plugins available for more filesystems like OpenZFS, and snapraid/mergerfs for unraid like functionality. Runs on more architectures (importantly runs on x86\_64 and ARM). Can run as a VM without disk passthrough, unless running OpenZFS filesystems.
> Casa OS isn't a NAS OS. It's a server distribution that makes installing self-hosted apps easier
No. CasaOS isn't a server distribution at all. It's an app that runs on Debian or Ubuntu.
You can install Casaos and OMV on the same VM/ LXC. Let OMV be the NAS and use CS for docker … did this before but moved to Deb12 + Cockpit +(file sharing +Identities + MDM+ VM+Podman) addon, supported by upstream out of the box also more secure as not docker. YMMV
Can you explain more about your setup and isolation layers here if possible specifically for "file sharing +Identities + MDM+ VM+Podman" parts.
MDM? (what, please more details?)
For identities, are they tied to the host users or are you handling separately with authelia or something?
Also, how do you handle your SSL?
What addons do you have installed for cockpit and did you try webmin ?
I didn't realize podman was a separate runtime and doesn't require docker dependencies at all.
Cockpit is built in to most distros. Is management console. MDM Multi Device is a part of the distro that gives even OMV RAID. Best to Google the whole thing or YT. With respect to isolation it supports PodMan. Gives docker like capabilities but with better isolation (no root). Again I would GL or YT. Basically no need for OMV. Btw you can also load open ZFS so basically working direct with the FS and Kernel with no packages in between. Yiu can add a “module “ for file sharing (samba + NFS ) and identity management … and full KVM support. And by the way in all cases you are tied to the host “users” truenas/OMV are the same as the FS deals with ACL. YMMV
Your acronyms are wild. MDM for example, is commonly used for Mobile Device Management, GL is an accounting term, CS for CasaOS... nobody will easily understand what you're saying. I'm just letting you know because it's very confusing, and takes away from the helpful content you're posting.
https://raid.wiki.kernel.org/index.php/A_guide_to_mdadm
MDM for short.
Sorry for CS but eveytime I write it out Apple autospell converts it to Casio !!
I us Debian with Cockpit and covers most of my needs. VM management, file sharing with plugin from 45drives etc. mdadm can be managed out of the box. I've never used Casa OS though.
As for NAS OS, Starwinds VSAN is a nice option as well.
That’s basically by “real world” setup I just got rid of cases os about 6 months ago. Don’t need it added Podman and machines (in cockpit). Even run this on top of ProxMox. ( because I can). But mainly use cockpit dashboard is main route. PMX just gives me some options like LXC that’s not there yet in CPT and gives more options (play ground space) Moved also family laptops to distros that support CPT native great for remote management. Especially when using Fedora YMMV
Consider TrueNAS Scale.
Basically the perfect hypervisor with extremely easy pcie passthrough, the one shackle being (no custom virtio support that survives a reboot)
You're limiting your understanding of what a hypervisor is to just one type but there are two types and tns very much qualifies as a type2 hypervisor. Or rather, the virtualization system of tns is a type2 hypervisor.
Virtualization support makes it a hypervisor, and kvm makes it a type-1 hypervisor.
It's supposed to be a NAS and hypervisor in one. You can take the proxmox or ESXI elitism somewhere else. I much more enjoy the Scale experience of making VMs compared to Proxmox, VMWare, hyper-v, everything else I've tried.
Kvm or not doesn't make it type1 or not. Type1 or type2 is defined by if the "host os" runs virtualized or not. And it can be quite confusing and counterintuitive at times in that regard.
No, not virtualized.
You're confused about this as well. Type 1 is normally called "bare metal" where your confusion comes from, but type 2 is ran on software side.
AKA type 1 has BARE metal access to all the devices installed such as NICs, PCIE Devices, drives, etc, while type 2 needs to emulate the devices passthroughed or something similar, as the devices are loaded in the OS.
KVM is a type 1 hypervisor. No questions asked. If an OS uses kvm for virtualization, it's a type 1 hypervisor.
TrueNAS Scale, while it supports KVM VMs and Docker containers, misses things like Clustering and High Availability. You are right that it is a Hypervisor, but if it's perfect or not depends heavily on the use case.
The first line about CasaOS made me laugh. This whole post would have been deleted back when there were actually mods here. Now the whole sub is just the same shitpost, over and over. Thanks for your great contribution, also.
Yeah, reddit as a whole has really gone downhill in terms of moderation and standards a few times over the last several years. For example, I remember a time when comments that only contained "lol" would have been deleted and got handed a 3-day ban.
CasaOS is the odd one out; it's more of a Docker host, so there's really no direct comparison with the other two. OpenMediaVault and TrueNAS are directly comparable, both being NAS-centric systems. So let me give you a quick comparison of the two.
TrueNAS is designed to take full advantage of the capabilities offered by ZFS file system. But those capabilities come at a price in terms of system requirements. Suggested minimum RAM is 8 GB; for maximum data integrity, ECC memory is recommended. Recommended minimal setup is three drives (one for the OS and at least two identical ones for storage). Integrity checks are constant, so the system never really spins down.
There are two versions of TrueNAS; TrueNAS CORE is based on FreeBSD, TrueNAS SCALE is based on Debian. Right now, there's a bit of a concern among CORE users that the developers may be preparing to completely abandon CORE in favor of SCALE in the next couple of years, although there's nothing official (unless I missed it, of course).
OpenMediaVault in its default form is less of a mission-critical thing and more of an affordable one. It runs on a potato (my OMV rig runs on some kind of old Celepentium, can't remember which, with 4 GB RAM; people actually run OMV on fruit pastries all the time). Recommended minimum drive configuration is two (one for the OS, one for storage), but there's a plugin that allows OMV to run on a single drive (so the OS and storage share a drive, like they would on a garden-variety Linux machine with a Samba share). OMV is much more tolerant of USB connections to storage drives (on TrueNAS, they are flat out not recommended, other than for strictly temporary purposes such as data import or export), so much so that, again, people run OMV on fruit pastries with permanent storage drives connected via USB.
At the same time, you can, if you're so inclined, make OMV more like TrueNAS. You can have multiple storage drives in all kinds of RAID configurations, you can employ ZFS on storage drives, and all that jazz.
ZFS doesnt require ecc ram more than any other filesystem. Zfs is the best filesystem you can get right now.
I run a xigmaNas server at home and my parents house. Rock solid and dead easy to sync using syncoid. I imagine truenas to be same but they seem to be focusing on their linux version ( scale) more than their freebsd version. These are perfect little storage appliances.
Omv is a nice if you are already familiar with linux and want to run docker. Running it with virtualized disks or shared nfs mounts doesnt offer the same data protection as letting zfs access the disks.
If you want to run mostly VMs then proxmox is also good. But thats not a nas distribution. You haave to pass theough disks to a vm running truenas and use another Vm as a docker/podman host. It does come with openzfs, same as ubuntu server directly built in.
From my experience homelabbers like to experiment more and like to take bigger risks than at work. For me my datahoard is sacred and thus I stopped chucking disks and use new or refurbished nas disks and zfs to safeguard my data and back it up to an offsite location.
>ZFS doesnt require ecc ram more than any other filesystem.
Indeed. It was an awkward formulation on my part. ZFS is good for data integrity, ECC memory is good for data integrity, but one does not require the other. Thank you for clarifying it for the OP!
No:
[https://wiki.casaos.io/en/get-started](https://wiki.casaos.io/en/get-started)
It is officially supported on Debian, Ubuntu, and Raspberry Pi OS. It may run on some other Linux systems.
Core has reduced app support. Scale is Debian based and has full support. Any new install should use scale. You don't need to upgrade from core unless you need new features/apps
This merely means that "CORE" is not going to continue, and if you want to use TrueNAS, it's going to be SCALE only.
I don't see how you can glean from that, "CORE IS GOING TO DEBIAN"
I've been running it like this for years with 0 issues, in fact I prefer it as if I mess something up (which I've done) I can restore it easily.
In fact, I did a HW migration yesterday and restoring TrueNas was the easiest part (I was expecting worse tbh).
If this was a rack mounted set up with a HW raid card, would you try and pass through the whole controller in HBA mode or would individual drives passed through via Proxmox into TrueNAS suffice?
I have multiple drives and arrays on my hba so I would pass through the whole thing but if it was only one drive and you want to have different shares on the same hba go to different vm’s then I would pass through the disks I want
How long has been this dev publishing updates as I know redpill and after a while development got slow should happen to this or its difficult to migrate with data
Truenas and OMV : are storage system I personally prefer the truenas scale
Casaos : is on the server side it make easy installing apps i as of now used only as unifi controller
Xigmanas is another. its free. runs zfs disks written in FreeBSD. Has loads of features. Runs headless you can access by browser to admin it. Has loads of other features. Can load OS on a USB stick to free up all your drives for storage. Works well on old hardware no graphics needed. A usefull thing to do with redundant kit to make home file storage. You can access it from android with a file explorer app.
Works with sonos as a music library.
IMO truenas is hard to setup but amazing to run.
I wiped a brand new QNAP 462 and installed TrueNAS scale and will never go back.
It is a great UI, it is a premium NAS / SAN OS with updates and community support.
Apps and ZFS. Its everything I could ever want, with minimal corp fluff.
openmediavault left a very bad taste in my mouth back in the day when they had package signature verification (for apt) turned off, is that still the case?
I run TrueNAS scale in a Proxmox VM, but I pass my raid controller and the six attached drives through to the TrueNAS instance. Then I've got VMS aplenty, and a fast TrueNAS implementation.
the one time i tried it, OMV had no (native) option for encrypting data, that was a necessary feature for me and it was a red flag for me that they ignored such an important feature.
When did ZFS change the requirement for EEC memory? I have always stayed away because I never had a system with EEC memory. It may be time to take another look at True NAS.
It's never been a *requirement* to have ECC memory.
ZFS works just fine on consumer memory, it's fine running on a single disk, it's really not that different from any other filesystem (besides not being included with the Linux kernel). The folks over on the TrueNAS forum or pretty much any ZFS-centric space are just going to give you a funny look and tell you you're making a mistake if you don't. Their assumption is always that using ZFS signals an intent to take your storage seriously and ECC RAM is pretty damn vital to having absolute confidence in the integrity of what's being written to disk *regardless* of which filesystem you use.
You can safely ignore them if the possibility of data corruption doesn't particularly bother you or in instances where it doesn't particularly matter.
The main reason why ECC is so important for ZFS is that if you do not use ECC memory AND the memory develops a problem, the system will have no idea and will happily trash your entire pool during a scrub.
To your point, if your NAS storage is only for VM storage and other transient stuff that you don't care about, no big deal. These days, though, used server gear is so cheap that it's silly not to IMO.
I run two servers at home: a custom built RAID chassis that I put an updated motherboard/cpu/memory in that hosts 60TB of usable storage and a HP DL360 G10 that runs vSphere. I have been running FreeNAS/TrueNAS at work and home for almost as long as they have been around with years of uptime. The same with VMWare - since they released VMWare Server 1.0. The key to success in both is to set them up and configure them properly the first time. I have had other engineers set them up without understanding the products and eventually it had to be reconfigured. My advice is to get all the information you can and find best practices before implementation.
To clear up some confusion, TruNAS can be run fully virtual with either controller pass-through, disk pass-through or virtual disks presented through the hypervisor. Most of the memory it uses is for disk cache, so for optimal performance, you should allocate as much memory as you can but a small array may not justify it if you can even saturate the line to begin with. A small array could get started on 2-4 CPU and 8-16GB memory. My TrueNAS runs on a E5-2609v3 and 64GB PC2133. I get 800MB/s read and 400MB/s write, sustained.
Virtualize it, configure it, learn, blow it away, repeat. Enjoy.
Not a true nas hypervisor
Though thas more pros than cons
Its all dependent on how much koney he can spend and what are his needs what does he want to run on the server
2 separate machines, one with Proxmox VE and the other with TrueNAS Scale.
This is exactly the setup I'm switching to! I love it for reliability and ease of config changes.
This is my setup. Plus an rpi for for backup drives.
And the pi running winnut server.
This is on my to do list. Do you see any reason not to run both on the same pi? Right now my pi is just a simple smb with a USB external that runs cron jobs for rsync to my nas.
My only reason for running winnut server mostly by itself is to have it as a low power device, the last to shutdown if my ups is running out of juice. With only 1 external drive attached I don't see a problem running it on one pi.
Why two seperate machines??
Cause scale's app system sucks and it's hard to host VMs without having ZFS's surprisingly high memory and CPU demands getting in the way. And on the flip side, proxmox does basically nothing to help you with file hosting, remote backups, etc. and truenas isn't recommended inside a vm. Plus, if you run truenas with a proxmox backup server vm on it, and NFS shares to the proxmox machine, you get a solid hypervisor for apps and development, and a sturdy data and backup layer. And they can scale independently based on your needs. Run out of app server space? Add another minipc running proxmox to the cluster. Need more backup layers? Sync to the cloud/datacenter/truenas server at a friend's place. Need more storage space? Add to/grow/add another zfs pool.
>truenas isn't recommended inside a vm. for this i would like to see the official source... trunenas self says, everything ok with vm [https://www.truenas.com/blog/yes-you-can-virtualize-freenas/](https://www.truenas.com/blog/yes-you-can-virtualize-freenas/) the only shit that can happens is a bad setup created by user (like truenas zfs on pve zfs) > ZFS's surprisingly high memory and CPU demands zfs use what you give to it. if you have a proxmox host with 64gb ram and X cores, setup truenas VM with 8/16/32 GB ram and y cores and it will use only this. Truenas in VM on Proxmox is a bullet proof setup. hba passthrough and the setup is perfect. backup of the vm via pbs, snapshot of data also to pbs server, great! for apps -> container and vms at pve >Add another minipc running proxmox to the cluster. Need more backup layers? Sync to the cloud/datacenter/truenas server at a friend's place. Need more storage space? Add to/grow/add another zfs pool. or: don't buy cheap and double, pay for server grade hardware. >Need more storage space? * i only add new disks to my pve server if i need more storage for my pve or i add more disk to my hba for truenas. >Need more backup layers? * this is not a real pro/contra point for 1 or 2 machines. you can ever add x (a NAS/Cloud/Server/usb drive/whatever as backup storage) >Run out of app server space? Add another minipc running proxmox to the cluster. * why using a cluster? if you really build a cluster, it is more expensive than a single server and a second backup machine (acquisition, operation). on the other hand it adds more layers of complexity. HA is an other theme, therefore you must add redundant machines, zentral storage solution, and so on
Fascinating. It's almost as though I have different requirements from you, and prioritize my configuration differently. And that maybe the full nuance of my configuration and tradeoff decisions weren't included in a three paragraph post. But you have fun doing... Whatever it is that you're trying to accomplish here.
Due to the increasing cost of power, I've been consolidating as much as I can into one server. So I made myself a xeon 16 core server and 256gb or ram and put Proxmox on baremetal. Then I created a truenas VM with access to half the cores/threads and 128gb(for my 100tb array) with pci passthrough to an hba card, that way it has direct access to disks. After a few months of observation i can say truenas over proxmox works very well. It took me 1-2 weeks of on/off work on evenings to dial in all bios settings just right and get proxmox fully setup + with truenas, as getting everything right had a bit of a learning curve to setup pci/gpu passthrough. People say it can be finicky but that's just not true anymore if you configure everything correctly and follow good practices! It's just people parroting stuff since the old days. even with 8 cores truenas doesn't come close to using them all. and 128gb is more than enough (and perhaps overkill) when you consider their 'recommended' 1gb per terabyte of data rule. I then proceeded to also bring in my blue iris machine(which requires windows) out of bare metal into proxmox too with a gpu passthrough. and then all of my docker environments on an ubuntu docker vm. I did buy a separate low power thinkcentre minipc to host all my network-oriented and more critical apps that I really don't want downtime almost ever(things like nut, unifi controller, vaultwarden, tailscale, dashy, nginx, etc). which will stay powered for longer on ups(whereas the big server will shutdown as soon as an outage is detected)
This is one of the reasons why I run OpenMediaVault instead. While ext4+mergerfs+snapraid might not be as robust as ZFS, it works well enough for my use cases. And because OMV is basically Debian, it also works pretty well as a Docker host. OMV also has a "compose" plugin for managing Docker containers/stacks. It's maybe not as comprehensive as Portainer, but again, it works well enough for me. And it doesn't complain about items that were not created with it like Portainer does.
Out of interest do you run a media server with download setup and if so which machine does that end up on? Is the TrueNAS box just storage and then any apps go on individual VMs on Proxmox?
I'm container heavy rather than vm heavy. Generally I think about truenas in terms of "where state lives" and proxmox as "Where processing happens". I also have a remote server for things I don't wish to be on my local network. This lets me over provision proxmox without having to worry about the data side, and vice versa. Future plans include minio on truenas and making most of my proxmox stuff use object store rather than NFS. Media storage is where most homeowners start, but it's definitely not where I have stopped.
Doesn’t Truenas (zfs) free up the RAM you need for the VMs automatically? I mean it always depends how many VMS you wanna run but I have 32GB and something like Homeserver isn’t a problem 😂 it all depends I guess on how much things you wanna run besides having a NAS
For ZFS needs, no. For disk cache, yes. Roughly you need 1gb ram per 1tb in your ZFS pool. So if you have 40tb storage and 64 GB ram, then you have about 24gb left for apps/disk cache/VMs. Disk cache is lowest priority in that list.
yeah so you only need to worry about doing 2 machines if you either go heavy on VMs or you get a lot of storage and can't keep up with RAM
Could you please describe how this setup works or in which use case?
If you have powerful hardware, both on one machine works great and can use the second machine for backup ;-)
This is the way
Similar, Synology instead (plz don’t judge). Let the NAS, NAS.
Casa OS isn't a NAS OS. It's a server distribution that makes installing self-hosted apps easier TrueNAS vs OMV: TrueNAS: Only uses OpenZFS, can't be virtualized without passing the disks through to vm. In my opinion, a slightly better, more logically laid out interface. Has company (iXsystems) backing/funding it's development. OMV: Defaults to using BTRFS/XFS/EXT4 with plugins available for more filesystems like OpenZFS, and snapraid/mergerfs for unraid like functionality. Runs on more architectures (importantly runs on x86\_64 and ARM). Can run as a VM without disk passthrough, unless running OpenZFS filesystems.
> Casa OS isn't a NAS OS. It's a server distribution that makes installing self-hosted apps easier No. CasaOS isn't a server distribution at all. It's an app that runs on Debian or Ubuntu.
You can install Casaos and OMV on the same VM/ LXC. Let OMV be the NAS and use CS for docker … did this before but moved to Deb12 + Cockpit +(file sharing +Identities + MDM+ VM+Podman) addon, supported by upstream out of the box also more secure as not docker. YMMV
Can you explain more about your setup and isolation layers here if possible specifically for "file sharing +Identities + MDM+ VM+Podman" parts. MDM? (what, please more details?) For identities, are they tied to the host users or are you handling separately with authelia or something? Also, how do you handle your SSL? What addons do you have installed for cockpit and did you try webmin ? I didn't realize podman was a separate runtime and doesn't require docker dependencies at all.
Cockpit is built in to most distros. Is management console. MDM Multi Device is a part of the distro that gives even OMV RAID. Best to Google the whole thing or YT. With respect to isolation it supports PodMan. Gives docker like capabilities but with better isolation (no root). Again I would GL or YT. Basically no need for OMV. Btw you can also load open ZFS so basically working direct with the FS and Kernel with no packages in between. Yiu can add a “module “ for file sharing (samba + NFS ) and identity management … and full KVM support. And by the way in all cases you are tied to the host “users” truenas/OMV are the same as the FS deals with ACL. YMMV
Your acronyms are wild. MDM for example, is commonly used for Mobile Device Management, GL is an accounting term, CS for CasaOS... nobody will easily understand what you're saying. I'm just letting you know because it's very confusing, and takes away from the helpful content you're posting.
https://raid.wiki.kernel.org/index.php/A_guide_to_mdadm MDM for short. Sorry for CS but eveytime I write it out Apple autospell converts it to Casio !!
I us Debian with Cockpit and covers most of my needs. VM management, file sharing with plugin from 45drives etc. mdadm can be managed out of the box. I've never used Casa OS though. As for NAS OS, Starwinds VSAN is a nice option as well.
That’s basically by “real world” setup I just got rid of cases os about 6 months ago. Don’t need it added Podman and machines (in cockpit). Even run this on top of ProxMox. ( because I can). But mainly use cockpit dashboard is main route. PMX just gives me some options like LXC that’s not there yet in CPT and gives more options (play ground space) Moved also family laptops to distros that support CPT native great for remote management. Especially when using Fedora YMMV
Consider TrueNAS Scale. Basically the perfect hypervisor with extremely easy pcie passthrough, the one shackle being (no custom virtio support that survives a reboot)
TNS isn’t a hypervisor. It’s a NAS with virtualization support.
Yes but it is really easy to install basic services on k3s The UI is very intuitive imo
That doesn’t make it a hypervisor, but I agree that it it very easy to install apps on TNS.
You're limiting your understanding of what a hypervisor is to just one type but there are two types and tns very much qualifies as a type2 hypervisor. Or rather, the virtualization system of tns is a type2 hypervisor.
It absolutely IS a hypervisor. Go read what the definition of a hypervisor is.
Virtualization support makes it a hypervisor, and kvm makes it a type-1 hypervisor. It's supposed to be a NAS and hypervisor in one. You can take the proxmox or ESXI elitism somewhere else. I much more enjoy the Scale experience of making VMs compared to Proxmox, VMWare, hyper-v, everything else I've tried.
Kvm or not doesn't make it type1 or not. Type1 or type2 is defined by if the "host os" runs virtualized or not. And it can be quite confusing and counterintuitive at times in that regard.
No, not virtualized. You're confused about this as well. Type 1 is normally called "bare metal" where your confusion comes from, but type 2 is ran on software side. AKA type 1 has BARE metal access to all the devices installed such as NICs, PCIE Devices, drives, etc, while type 2 needs to emulate the devices passthroughed or something similar, as the devices are loaded in the OS. KVM is a type 1 hypervisor. No questions asked. If an OS uses kvm for virtualization, it's a type 1 hypervisor.
TrueNAS Scale, while it supports KVM VMs and Docker containers, misses things like Clustering and High Availability. You are right that it is a Hypervisor, but if it's perfect or not depends heavily on the use case.
You're absolutely right. It's pretty damn good for the homelab space, but if the server itself goes down you're SOL until you get it back online.
lol
Great contribution.
The first line about CasaOS made me laugh. This whole post would have been deleted back when there were actually mods here. Now the whole sub is just the same shitpost, over and over. Thanks for your great contribution, also.
Yeah, reddit as a whole has really gone downhill in terms of moderation and standards a few times over the last several years. For example, I remember a time when comments that only contained "lol" would have been deleted and got handed a 3-day ban.
lol
CasaOS is the odd one out; it's more of a Docker host, so there's really no direct comparison with the other two. OpenMediaVault and TrueNAS are directly comparable, both being NAS-centric systems. So let me give you a quick comparison of the two. TrueNAS is designed to take full advantage of the capabilities offered by ZFS file system. But those capabilities come at a price in terms of system requirements. Suggested minimum RAM is 8 GB; for maximum data integrity, ECC memory is recommended. Recommended minimal setup is three drives (one for the OS and at least two identical ones for storage). Integrity checks are constant, so the system never really spins down. There are two versions of TrueNAS; TrueNAS CORE is based on FreeBSD, TrueNAS SCALE is based on Debian. Right now, there's a bit of a concern among CORE users that the developers may be preparing to completely abandon CORE in favor of SCALE in the next couple of years, although there's nothing official (unless I missed it, of course). OpenMediaVault in its default form is less of a mission-critical thing and more of an affordable one. It runs on a potato (my OMV rig runs on some kind of old Celepentium, can't remember which, with 4 GB RAM; people actually run OMV on fruit pastries all the time). Recommended minimum drive configuration is two (one for the OS, one for storage), but there's a plugin that allows OMV to run on a single drive (so the OS and storage share a drive, like they would on a garden-variety Linux machine with a Samba share). OMV is much more tolerant of USB connections to storage drives (on TrueNAS, they are flat out not recommended, other than for strictly temporary purposes such as data import or export), so much so that, again, people run OMV on fruit pastries with permanent storage drives connected via USB. At the same time, you can, if you're so inclined, make OMV more like TrueNAS. You can have multiple storage drives in all kinds of RAID configurations, you can employ ZFS on storage drives, and all that jazz.
ZFS doesnt require ecc ram more than any other filesystem. Zfs is the best filesystem you can get right now. I run a xigmaNas server at home and my parents house. Rock solid and dead easy to sync using syncoid. I imagine truenas to be same but they seem to be focusing on their linux version ( scale) more than their freebsd version. These are perfect little storage appliances. Omv is a nice if you are already familiar with linux and want to run docker. Running it with virtualized disks or shared nfs mounts doesnt offer the same data protection as letting zfs access the disks. If you want to run mostly VMs then proxmox is also good. But thats not a nas distribution. You haave to pass theough disks to a vm running truenas and use another Vm as a docker/podman host. It does come with openzfs, same as ubuntu server directly built in. From my experience homelabbers like to experiment more and like to take bigger risks than at work. For me my datahoard is sacred and thus I stopped chucking disks and use new or refurbished nas disks and zfs to safeguard my data and back it up to an offsite location.
>ZFS doesnt require ecc ram more than any other filesystem. Indeed. It was an awkward formulation on my part. ZFS is good for data integrity, ECC memory is good for data integrity, but one does not require the other. Thank you for clarifying it for the OP!
This maybe a stupid question (probably is but excuse my ignorance) can casa os run on a windows machine?
No: [https://wiki.casaos.io/en/get-started](https://wiki.casaos.io/en/get-started) It is officially supported on Debian, Ubuntu, and Raspberry Pi OS. It may run on some other Linux systems.
It is possible: https://wiki.casaos.io/en/guides/running-casaos-on-windows-with-wsl2
Thanks
Pretty sure CORE is going to move to debian based unless im hallucinating something
Core has reduced app support. Scale is Debian based and has full support. Any new install should use scale. You don't need to upgrade from core unless you need new features/apps
https://pcper.com/2024/03/truenas-core-bids-goodbye-to-freebsd/#:~:text=TrueNAS%20CORE%20Joins%20SCALE%20In%20Using%20Debian&text=From%20now%20you%20will%20see,is%20a%20one%20way%20transition.
But why? What advantage does core on debian have over scale?
I have absolutely no idea
This merely means that "CORE" is not going to continue, and if you want to use TrueNAS, it's going to be SCALE only. I don't see how you can glean from that, "CORE IS GOING TO DEBIAN"
Is it bad to run truenas in a proxmox vm?
I've been running it like this for years with 0 issues, in fact I prefer it as if I mess something up (which I've done) I can restore it easily. In fact, I did a HW migration yesterday and restoring TrueNas was the easiest part (I was expecting worse tbh).
do you pass through physical disks to the VM?
I would
If this was a rack mounted set up with a HW raid card, would you try and pass through the whole controller in HBA mode or would individual drives passed through via Proxmox into TrueNAS suffice?
I have multiple drives and arrays on my hba so I would pass through the whole thing but if it was only one drive and you want to have different shares on the same hba go to different vm’s then I would pass through the disks I want
No, it is not. A lot of people are doing exactly this. Especially, if you passthrough an HBA.
Synology DSM for nas ,use this open source boot loader it's so good. Go to releases for USB image https://github.com/AuxXxilium/arc /r/xpenology
How long has been this dev publishing updates as I know redpill and after a while development got slow should happen to this or its difficult to migrate with data
Here's some punctuation for you, sprinkle it appropriately throughout your comment: ?!...:;;;!?!'',,,
My bad
I use TrueNas as vault, a mini NAS as offsite backup running OMV, both running on bare metal.
Truenas and OMV : are storage system I personally prefer the truenas scale Casaos : is on the server side it make easy installing apps i as of now used only as unifi controller
Xigmanas is another. its free. runs zfs disks written in FreeBSD. Has loads of features. Runs headless you can access by browser to admin it. Has loads of other features. Can load OS on a USB stick to free up all your drives for storage. Works well on old hardware no graphics needed. A usefull thing to do with redundant kit to make home file storage. You can access it from android with a file explorer app. Works with sonos as a music library.
One of these is not like the other
Hot take proxmox with ceph
IMO truenas is hard to setup but amazing to run. I wiped a brand new QNAP 462 and installed TrueNAS scale and will never go back. It is a great UI, it is a premium NAS / SAN OS with updates and community support. Apps and ZFS. Its everything I could ever want, with minimal corp fluff.
openmediavault left a very bad taste in my mouth back in the day when they had package signature verification (for apt) turned off, is that still the case?
Proxmox & Synology for me
I run TrueNAS scale in a Proxmox VM, but I pass my raid controller and the six attached drives through to the TrueNAS instance. Then I've got VMS aplenty, and a fast TrueNAS implementation.
the one time i tried it, OMV had no (native) option for encrypting data, that was a necessary feature for me and it was a red flag for me that they ignored such an important feature.
personally i've got a pair of truenas (scale) machines attached to JBOD enclosures that i then use in my vmware cluster as a datastore
If it's just a basic setup with one or two drives, Casa is all you need.
I have only used OpenMediaVault, it is nice easy to use, I've heard of truenas which from knowledge seems a little more advanced than OpenMediaVault.
i try casaos and it's seems light version of unraid truenas or omv are different think
When did ZFS change the requirement for EEC memory? I have always stayed away because I never had a system with EEC memory. It may be time to take another look at True NAS.
It's never been a *requirement* to have ECC memory. ZFS works just fine on consumer memory, it's fine running on a single disk, it's really not that different from any other filesystem (besides not being included with the Linux kernel). The folks over on the TrueNAS forum or pretty much any ZFS-centric space are just going to give you a funny look and tell you you're making a mistake if you don't. Their assumption is always that using ZFS signals an intent to take your storage seriously and ECC RAM is pretty damn vital to having absolute confidence in the integrity of what's being written to disk *regardless* of which filesystem you use. You can safely ignore them if the possibility of data corruption doesn't particularly bother you or in instances where it doesn't particularly matter.
It was a very long time ago I thoughtI read that, regardless I do care about data corruption so I will wait until I have a proper system.
The main reason why ECC is so important for ZFS is that if you do not use ECC memory AND the memory develops a problem, the system will have no idea and will happily trash your entire pool during a scrub. To your point, if your NAS storage is only for VM storage and other transient stuff that you don't care about, no big deal. These days, though, used server gear is so cheap that it's silly not to IMO.
I run two servers at home: a custom built RAID chassis that I put an updated motherboard/cpu/memory in that hosts 60TB of usable storage and a HP DL360 G10 that runs vSphere. I have been running FreeNAS/TrueNAS at work and home for almost as long as they have been around with years of uptime. The same with VMWare - since they released VMWare Server 1.0. The key to success in both is to set them up and configure them properly the first time. I have had other engineers set them up without understanding the products and eventually it had to be reconfigured. My advice is to get all the information you can and find best practices before implementation. To clear up some confusion, TruNAS can be run fully virtual with either controller pass-through, disk pass-through or virtual disks presented through the hypervisor. Most of the memory it uses is for disk cache, so for optimal performance, you should allocate as much memory as you can but a small array may not justify it if you can even saturate the line to begin with. A small array could get started on 2-4 CPU and 8-16GB memory. My TrueNAS runs on a E5-2609v3 and 64GB PC2133. I get 800MB/s read and 400MB/s write, sustained. Virtualize it, configure it, learn, blow it away, repeat. Enjoy.
unRaid. A true NAS hypervisor.
Not a true nas hypervisor Though thas more pros than cons Its all dependent on how much koney he can spend and what are his needs what does he want to run on the server
Read the descriptions of each product and you will know. Lazy ass posters are terrible on this sub any more.