this post was submitted on 10 May 2024

3 points (100.0% liked)

Selfhosted

38707 readers

677 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago

MODERATORS

[email protected]

Move UnRaid from metal to Proxmox (lemmy.ml)

submitted 4 months ago by [email protected] to c/[email protected]

16 comments fedilink hide all child comments

I have a trusty UnRaid server that has been running great for almost 3 years now, with some kinks and headaches here and there, but mostly very stable. Now I'm entertaining the idea of setting that box up with ProxMox, and running UnRaid virtualized. The reason being that I want to use UnRaid exclusively as a NAS and then run all dockers and VMs on ProxMox (at least that's how I'm picturing it). I would like to know your opinion on this idea. All I have is Nextcloud, Immich, Vaultwarden, Jellyfin, Calibre, Kavita and a Windows VM I use to update some hardware every now and then. I mainly want to do that for the backup capabilities in ProxMox for each instance. Storage is not a concern, and I have 64GB of ECC Ram running in that box. What are the Pros and Cons, or is it even worth it to move all this to ProxMox?

top 16 comments

sorted by: hot top controversial new old

[–] [email protected] 1 points 4 months ago* (last edited 4 months ago)

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters	More Letters
HA	Home Assistant automation software
~	High Availability
LXC	Linux Containers
NAS	Network-Attached Storage
NFS	Network File System, a Unix-based file-sharing protocol known for performance and efficiency
NVMe	Non-Volatile Memory Express interface for mass storage
SATA	Serial AT Attachment interface for mass storage
SMB	Server Message Block protocol for file and printer sharing; Windows-native
SSD	Solid State Drive mass storage
SSH	Secure Shell for remote terminal access
ZFS	Solaris/Linux filesystem focusing on data integrity
k8s	Kubernetes container management package

[Thread #744 for this sub, first seen 10th May 2024, 17:55] [FAQ] [Full list] [Contact] [Source code]

[–] [email protected] 0 points 4 months ago (1 children)

If you virtualize unraid, unraid wont have direct drive access - you can get around this by getting an HBA card and forwarding that to the unraid VM. Others have mentioned that proxmox doesn't have docker support, I personally run docker containers within lxc boxes on proxmox. There are solutions to make managing containers easier, like portainer, if you want to go down that route.

[–] [email protected] -1 points 4 months ago

Run docker within lxc within proxmox. This gave me an aneurism. You’ve lost the whole point of not actually virtualizing with containers by putting in two layers deep in virtualization. At this point your shit is so convoluted why don’t you just run kubernetes

[–] [email protected] 0 points 4 months ago* (last edited 4 months ago) (1 children)

Oof. No.

Wouldn't do it for a litany of reasons, but the main being that it's not meant for such things. You want it to be as close to the OS and drivers as possible. Anything getting between Unraid managing the disks is overly complex, and asking for trouble. What happens if the container dies? What happens if the container gets OOMkill'd?

If you're not going to use it to manage your disks, then I guess no issues, but there's better suited software for such things.

Isn't Unraid also a VM host of sorts?

[–] [email protected] 0 points 4 months ago (1 children)

Yeah, UnRaid does all of that, but from my very basic testing of ProxMox in an old computer, the VM management is much better than in UnRaid. The same goes for VLAN awareness with just 1 nic.

I'm in no way unsatisfied with UnRaid, but I watched a video by Christian Lempa doing something similar, only with TrueNAS instead of UnRaid, which is what got my brain thinking about all these potential options.

https://www.youtube.com/watch?v=M3pKprTdNqQ

[–] [email protected] 0 points 4 months ago (1 children)

There's the question of "CAN I do this?" vs "SHOULD I do this?". I don't think abstracting your main storage handling software away from where it definitely needs to be is going to net you anything positive, but add more issues and complications.

I'm sure you can find videos of people running drivers out of containers just because it's possible. Should you though? Nope.

[–] [email protected] 0 points 4 months ago (1 children)

I do have the advantage of having a mirror of my server 2.5K miles away in my brother's house. That's probably why I'm thinking about being so candidly careless.

I appreciate the great advise. But now I'm willing to take one for the team and come back with either am horror story or an epic win.

BRB.

[–] [email protected] 0 points 4 months ago (2 children)

Have you considered the increase in disk io and that hypervisor prefer to be in control of all hardware? Including disks...

If you are set on proxmox consider that it can directly share your data itself. This could be made easy with cockpit and the zfs plugin. The plugin helps if you have existing pools. Both can be installed directly on proxmox and present a separate web UI with different options for system management.

The safe things here to use are the filesharing and pool management operations. Basically use the proxmox webui for everything it permits first.

Either way have fun.

[–] [email protected] 1 points 4 months ago

That sounds like a great idea.

At the moment I am using Openmediavault as a VM within proxmox - I pass my HDDs through to this VM. Openmediavault let's me do all the stuff I want to: Share folders via SSH, NFS and raid-management.

Do you know if I can do the same with proxmox directly? Do you maybe have a link where this way is described in detail?

[–] [email protected] 0 points 4 months ago (1 children)

I actually never considered this. And if I'm understanding you correctly, this would render using UnRaid unnecessary.

This is great info. I'm going to fit my current ProxMox test rig with a few disks I have (old small disks I have replaced over the years that still work) and test this option first. This might make this easier.

If this works out, I can still keep the server I set up off-site to mirror my storage, right? Even if that is still UnRaid? I need more coffee.

[–] [email protected] 0 points 4 months ago (1 children)

Yup you can. In fact you likely should and will probably find yourself improving disk io dramatically compared to your original thoughts doing this. It's better in my opinion to let the hypervisor manage disks operations. That means in my opinion it should also share files with smb and NFS especially if you are already considering nas type operations.

Since proxmox supports zfs out of the box along with btrfs and even XFS you have a myriad of options. You combine that with cockpit and you have a nice management interface.

I went the zfs route because I'm familiar with it and I appreciate it's native sharing options built into the filesystem. It's cool to have the option to create a new dataset off the pool and directly pass it into a new lxc container.

[–] [email protected] 0 points 4 months ago (2 children)

I'm very inclined to use this method instead.

I would like to ask for some suggestions on the initial process to migrate the data from UnRaid.

Considering that:

My disk pool is made out of 2 10TB disks, for a total of 20TB
It also has a 10TB parity disk
The pool is using just -6TB of the storage

The option I see is:

Get another 10TB disk
I can clear the parity drive and copy my data from the pool to that disk for migrating
Configure the pool disks to RaidZ and once I complete that, use the other 2 disks as parity pool

Or, I bite the bullet, get brand new 10TB disks, 12 to make it Raidz2 and have a storage pool of 40TB (35 usable?). I'm thinking 4 groups of 3 disks each should do the trick. Then use the same method to migrate my data.

With 64GB of ECC RAM, I should have a pretty swift storage IOPS that way.

[–] [email protected] 1 points 4 months ago (1 children)

Bookmark this if you utilize zfs at all. It will serve you well.

https://jrs-s.net/2018/08/17/zfs-tuning-cheat-sheet/

You will be amused with zfs performance in proxmox due to all the tuning that is possible. If this is going to be an existing zfs pool keep in mind it's easier to just install proxmox with the zfs option and let it create a zfs rpool during setup. For the rpool tweak a couple options. Make sure ashift is at least 12 during the install or 13 if you are using some crazy fast SSD as proxdisk for the rpool.

It needs to be 12 if it's a modern day spinner and probably a good setting for most ssds. Do not go over 12 if it's a spinning disk.

Now beyond that you can directly import your existing zfs pool into proxmox with a single import command. Assuming you have an existing zfs pool.

In this scenario zfs would be fully maintaining disk operations for both an rpool and a media pool.

You should consider tweaking a couple things to really improve performance via the guide de I linked.

Proxmox vms/zvols live in their own dataset. Before you start getting to crazy creating vms make sure you are taking advantage of all the performance tweaks you can. By default proxmox sets a default record size for all datasets to 128k. qcow2, raw, and even zvols will benefit from record size of 64k because it tends to improve the underlying filesystem performance of things like ext4, XFS, even UFS. Imo it's silly to create vm filesystems like btrfs if you're vm is sitting on top of a cow filesystem.

Another huge improvement is tweaking the compression algorithm. lz4 is blazing fast and should be your default go to for zfs. The new one is pretty good but can slow things down a bit for active operations like active vm disks. So make sure your default compression is lz4 for datasets with vm disks. Honestly it's just a good default to specify for the entire pool. You can select other compressions for datasets with more static data.

If you have a media dataset full of files like music, vids, pics. Setting a record size of 1mb will heavily improve disk io operations.

In proxmox it will default to grabbing half of your memory for arc. Make sure you change that after install. It's a file that defines arc_max in byte number format. Set the max to something more reasonable if you have 64 gigs of memory. You can also define the arc_min

Some other huge improvements? If you are using an SSD for your proxmox install I highly recommend you install log2ram on your hypervisor. It will stop all those constant log writes on your SSD. It will also sync them to disk on a timer and shutdown/reboot. It's also a huge performance and SSD lifespan improvement to migrate /tmp and /var/tmp to tmpfs

So many knobs to turn. I hope you have fun playing with this.

[–] [email protected] 1 points 4 months ago

Thanks so much.

All this info brought me back to the drawing board.

This led me to start searching for new components, as I'm pretty sure that I will want to build a new rig and just probably donate my current box.

Thank you, I really appreciate it. My bank account, not so much 🤣🤣

[–] [email protected] 1 points 4 months ago (1 children)

Another thing to keep in mind with zfs is underlying vm disks will perform better if the zfs pool is a type of mirror or stripe of mirrors. Z1 Z2 type pools are better for media and files. Cm disk io will improve on the mirror type style dramatically. Just passing what I've learned over time in optimizing systems.

[–] [email protected] 1 points 4 months ago

I'll be studying that link you sent me deeply before I start my adventure here.

I didn't know this rabbit hole was so deep. Love it!