Somehow, their web developer managed to break scrolling on Safari, so I am unable to navigate the linked site. If anyone else was looking for a list of what has changed in recent releases, it can be found at https://pve.proxmox.com/wiki/Roadmap
Nutanix is popular with traditional larger enterprise VMware type customers, Proxmox is popular with the smaller or homelabber refugees. Exceptions exist to each of course.
Talking to midmarket and enterprise customers and nobody is taking Proxmox seriously quite yet, I think due to concerns around support availability and long term viability. Hyper-V and Azure Local come up a lot in these conversations if you run a lot of Windows (Healthcare in the US is nearly entirely Windows based). Have some folks kicking tires on OpenShift, which is a HEAVY lift and not much less expensive than modern Broadcom licenses.
My personal dark horse favorite right now is HPE VM Essentials. HPE has a terrible track record of being awesome at enterprise software, but their support org is solid and the solution checks a heck of a lot of boxes, including broad support for non-HPE servers, storage, and networking. Solution is priced to move and I expect HPE smells blood in these waters, they're clearly dumping a lot of development resources into the product in this past year.
I've used them professionally during 0.9 times (2008.) and it was already quite useful and very stable (all advertised features worked).
17 years looks pretty good to me, Proxmox will not go away (neither product or company)
>(Healthcare in the US is nearly entirely Windows based).
This wasn't my experience in over a decade in the industry.
It's Windows dominant, but our environment was typically around a 70/30 split of Windows/Linux servers.
Cerner shops in particular are going to have a larger Linux footprint. Radiology, biomed, interface engines, and med records also tended to have quite a bit of nix infrastructure.
One thing that can be said is that containerization has basically zero penetration with any vendors in the space. Pretty much everyone is still doing a pets over cattle model in the industry.
HPE VM Essentials and Proxmox are just UI/wrappers/+ on top of kvm/virsh/libvirt for the virtualization side.
You can grow out of either by just moving to self hosted, or you can avoid both for the virtualization part if you don't care about the VMware like GUI if you are an automation focused company.
If we could do it 20 years ago once VT-x for production Oracle EBS instances for a smaller but publicly traded company with a IT team of 4, almost any midmarket enterprise could do it today, especially with modern tools.
It is culture and web-ui requirements and FUD that cause issues, not the underlying products that are stable today, but hidden from view.
You can have a public company that invests in private companies, as opposed to investing in publicly listed companies (like $BRK/Buffett does (in addition to PE stuff)).
The only thing missing making Proxmox difficult in traditional environment is a replacement for VMware's VMFS (cluster aware VM file system).
Lots and lots of organizations already have SAN/storage fabric networks presenting block storage over the network which was heavily used for VMware environments.
You could use NFS if your arrays support it, but MPIO block storage via iscsi is ubiquitous in my experience.
Not really, that works if you want to have converged storage in your hypervisors, but most large VMWare deployments I've seen use external storage from remote arrays.
Shared across a cluster of multiple hosts, such that you can hot migrate VMs? I am not aware of that being possible in Proxmox the same way you can in VMware with VMFS.
Watching hypervisors slowly improve over the last few years has been amazing. They aren't quite to the point that I will install them under any new hardware I buy and then put my daily driver OS on top, but they are very close. I think a strong focus on creating 'the OS under your OS' experience seamless could open up a lot more here.
I have a PC where I installed Proxmox on bare metal and put a daily-use desktop OS on top. It works surprisingly well, the trickiest part was making sure the desktop OS took control of video/audio/peripherals.
VMware has been so good and reasonably priced for so long that there hasn't been a competitive market in the enterprise virtualization space for the past two decades. In a way, I think Broadcom's moves here might be healthy for the enterprise datacenter longer term, it has created the opportunity for others to step in and broadened the ecosystem significantly.
For many folk's workflows, I'd wager that hypervisors are there and ready. I had a nice time setting up xcp-ng before deciding microk8s fits my needs more betterer; they're just plum good, well documented, and blazing fast.
I'm not sure I would want my daily driver to be a hypervisor... Whats controlling audio, do I really need audio kernel extensions on my hypervisor? Whos in charge when I shut the lid on my laptop...
But the moment you stop trying to do everything locally Proxmox, as it is today, is a dream.
It's easy enough to spin up a VM, throw a clients docker/podman + other insanity onto it and have a running dev instance in minutes. It's easy enough to work remotely in your favorite IDE/dev env. DO I need to "try something wild", clone it... build a new one... back it up and restore if it doesn't work...
Do I need to emulate production at a more fine grained level than what docker can provide: easy enough to build something that looks like production on my proxmox box.
And when I'm done with all that work... my daily driver laptop and desktop remain free of cruft and clutter.
So with support for OCI container images, does this mean I can run docker images as LXCs natively in proxmox? I guess it's an entirely manual process, no mature orchestration like portainer or even docker-compose, no easy upgrades, manually setting up bind mounts, etc. It would be a nice first step.
15-20 years ago this wouldn't have been a company. It would have been a strong but informal open collaboration where
smart and just people funded by various entities around the world kept it running.
Then the opportunity to get rich by offering an open source product combined with closed source extras+support was invented. I don't like this new world.
Edit: Somewhere along the line, we also lost the concept of having a sysadmin/developer person working at like a municipality contributring like 20% of their time towards maintenance of such projects. Invaluable when keeping things running.
Funny enough, Proxmox VE is 17 years old. I want to say it was ballpark 13-14 years ago I was using it to replace ESXi to get features (HA/Live migration) that only came with expensive licensing. 15-20 years ago there were definitely companies doing exactly this.
But in many cases, like Proxmox, there is nothing proprietary. What they provide is the glue, the polish, the interface that ties together the technologies they build on. If they began to be nasty, you can just leave (or even, continue to use it however long you like).
In general, I don't think this is a threat. I think the problems begin with proprietary offerings, like they so often do in the cloud. Then's the time when vendor lock-in takes its toll. But even with AWS, if you stick to open interfaces, it's easy to leave.
What is this "application containers" BS, just add native docker stack support. Most folks in the self hosting community already deploy nested dockers in LXCs, just add native support so we can cut out the middle man and squeeze out that indirection.
Sorry, but I bought Proxmox 7, but it is not comparable. Incus does everything (and more) with better interface, WAY better reliability, and also not like a hundred EUR or whatever. (100 EUR is fine with me if better, but not if not better...)
I’ve been looking at incus, and some aspects are appealing (creating a vm/container via cli). But I think proxmox having better clustering, and built in support for ceph, backups (with proxmox backup server)… proxmox just had a little more maturity behind it. I’ll be watching incus though.
I was looking to setup Proxmox for my homelab soon but this comment got me interested in Incus. Mostly because I've never heard of any Proxmox alternatives before this. You can try out Incus in your browser here: https://linuxcontainers.org/incus/try-it/
The demo does take ~10m to get into a working instance.
also, looking at the link you posted, it looks like incus can only do like a fraction of what proxmox can do. is that the case or is that web ui a limiting factor?
Incus looks nice, though it looks to be more API driven , at least from the landing page. I can't attest to Proxmox in a production/cluster environment but (barring GPU passthrough) it's very accessible for homelab and small network.
I don't remember if I tried and failed, or if it seemed too much for me... I have an Arc A series; if you have a verified guide I would like to take a look!
Somehow, their web developer managed to break scrolling on Safari, so I am unable to navigate the linked site. If anyone else was looking for a list of what has changed in recent releases, it can be found at https://pve.proxmox.com/wiki/Roadmap
Proxmox (and XCP-ng?) seems to be "the" (?) popular alternative to VMware after Broadcom's private equity-fuel cash grab.
(Perhaps if you're a Microsoft shop you're looking at Hyper-V?)
Nutanix is popular with traditional larger enterprise VMware type customers, Proxmox is popular with the smaller or homelabber refugees. Exceptions exist to each of course.
Talking to midmarket and enterprise customers and nobody is taking Proxmox seriously quite yet, I think due to concerns around support availability and long term viability. Hyper-V and Azure Local come up a lot in these conversations if you run a lot of Windows (Healthcare in the US is nearly entirely Windows based). Have some folks kicking tires on OpenShift, which is a HEAVY lift and not much less expensive than modern Broadcom licenses.
My personal dark horse favorite right now is HPE VM Essentials. HPE has a terrible track record of being awesome at enterprise software, but their support org is solid and the solution checks a heck of a lot of boxes, including broad support for non-HPE servers, storage, and networking. Solution is priced to move and I expect HPE smells blood in these waters, they're clearly dumping a lot of development resources into the product in this past year.
I've used them professionally during 0.9 times (2008.) and it was already quite useful and very stable (all advertised features worked). 17 years looks pretty good to me, Proxmox will not go away (neither product or company)
>(Healthcare in the US is nearly entirely Windows based).
This wasn't my experience in over a decade in the industry.
It's Windows dominant, but our environment was typically around a 70/30 split of Windows/Linux servers.
Cerner shops in particular are going to have a larger Linux footprint. Radiology, biomed, interface engines, and med records also tended to have quite a bit of nix infrastructure.
One thing that can be said is that containerization has basically zero penetration with any vendors in the space. Pretty much everyone is still doing a pets over cattle model in the industry.
HPE VM Essentials and Proxmox are just UI/wrappers/+ on top of kvm/virsh/libvirt for the virtualization side.
You can grow out of either by just moving to self hosted, or you can avoid both for the virtualization part if you don't care about the VMware like GUI if you are an automation focused company.
If we could do it 20 years ago once VT-x for production Oracle EBS instances for a smaller but publicly traded company with a IT team of 4, almost any midmarket enterprise could do it today, especially with modern tools.
It is culture and web-ui requirements and FUD that cause issues, not the underlying products that are stable today, but hidden from view.
Two days ago saw a shop that moved to Incus. Seems to be a viable alternative too.
um broadcom is publicly traded as $AVGO...?
So is $KKR:
> KKR & Co. Inc., also known as Kohlberg Kravis Roberts & Co., is an American global private equity and investment company.
* https://en.wikipedia.org/wiki/KKR_%26_Co.
You can have a public company that invests in private companies, as opposed to investing in publicly listed companies (like $BRK/Buffett does (in addition to PE stuff)).
Plenty of people describe Broadcom as "Publicly traded Private Equity"
now that is something I can totally get behind
The only thing missing making Proxmox difficult in traditional environment is a replacement for VMware's VMFS (cluster aware VM file system).
Lots and lots of organizations already have SAN/storage fabric networks presenting block storage over the network which was heavily used for VMware environments.
You could use NFS if your arrays support it, but MPIO block storage via iscsi is ubiquitous in my experience.
The Proxmox answer to this is Ceph - https://ceph.io/en/
Not really, that works if you want to have converged storage in your hypervisors, but most large VMWare deployments I've seen use external storage from remote arrays.
Proxmox works fine with iSCSI.
Shared across a cluster of multiple hosts, such that you can hot migrate VMs? I am not aware of that being possible in Proxmox the same way you can in VMware with VMFS.
Watching hypervisors slowly improve over the last few years has been amazing. They aren't quite to the point that I will install them under any new hardware I buy and then put my daily driver OS on top, but they are very close. I think a strong focus on creating 'the OS under your OS' experience seamless could open up a lot more here.
I have a PC where I installed Proxmox on bare metal and put a daily-use desktop OS on top. It works surprisingly well, the trickiest part was making sure the desktop OS took control of video/audio/peripherals.
VMware has been so good and reasonably priced for so long that there hasn't been a competitive market in the enterprise virtualization space for the past two decades. In a way, I think Broadcom's moves here might be healthy for the enterprise datacenter longer term, it has created the opportunity for others to step in and broadened the ecosystem significantly.
For many folk's workflows, I'd wager that hypervisors are there and ready. I had a nice time setting up xcp-ng before deciding microk8s fits my needs more betterer; they're just plum good, well documented, and blazing fast.
I'm not sure I would want my daily driver to be a hypervisor... Whats controlling audio, do I really need audio kernel extensions on my hypervisor? Whos in charge when I shut the lid on my laptop...
But the moment you stop trying to do everything locally Proxmox, as it is today, is a dream.
It's easy enough to spin up a VM, throw a clients docker/podman + other insanity onto it and have a running dev instance in minutes. It's easy enough to work remotely in your favorite IDE/dev env. DO I need to "try something wild", clone it... build a new one... back it up and restore if it doesn't work...
Do I need to emulate production at a more fine grained level than what docker can provide: easy enough to build something that looks like production on my proxmox box.
And when I'm done with all that work... my daily driver laptop and desktop remain free of cruft and clutter.
[dead]
So with support for OCI container images, does this mean I can run docker images as LXCs natively in proxmox? I guess it's an entirely manual process, no mature orchestration like portainer or even docker-compose, no easy upgrades, manually setting up bind mounts, etc. It would be a nice first step.
Been waiting to update from v8. Time might be right now
15-20 years ago this wouldn't have been a company. It would have been a strong but informal open collaboration where smart and just people funded by various entities around the world kept it running.
Then the opportunity to get rich by offering an open source product combined with closed source extras+support was invented. I don't like this new world.
Edit: Somewhere along the line, we also lost the concept of having a sysadmin/developer person working at like a municipality contributring like 20% of their time towards maintenance of such projects. Invaluable when keeping things running.
Funny enough, Proxmox VE is 17 years old. I want to say it was ballpark 13-14 years ago I was using it to replace ESXi to get features (HA/Live migration) that only came with expensive licensing. 15-20 years ago there were definitely companies doing exactly this.
What do you find wrong with "this new world"? For context, I'm using their free offering for my home server, for 6-7 years now. Happy as a clam.
It enables Oracle-like behavior. Once you're locked in as commercial user they can do whatever they want.
Remember: Not all commercial users are FAANG rich. Counties/local municipalities count as commercial users, as an example.
But in many cases, like Proxmox, there is nothing proprietary. What they provide is the glue, the polish, the interface that ties together the technologies they build on. If they began to be nasty, you can just leave (or even, continue to use it however long you like).
In general, I don't think this is a threat. I think the problems begin with proprietary offerings, like they so often do in the cloud. Then's the time when vendor lock-in takes its toll. But even with AWS, if you stick to open interfaces, it's easy to leave.
What is this "application containers" BS, just add native docker stack support. Most folks in the self hosting community already deploy nested dockers in LXCs, just add native support so we can cut out the middle man and squeeze out that indirection.
docker errors or escapes taking down the root system? Not for me....
Docker is mostly based on the same stuff that LXC uses under the hood.
Nah. Incus.
Sorry, but I bought Proxmox 7, but it is not comparable. Incus does everything (and more) with better interface, WAY better reliability, and also not like a hundred EUR or whatever. (100 EUR is fine with me if better, but not if not better...)
I’ve been looking at incus, and some aspects are appealing (creating a vm/container via cli). But I think proxmox having better clustering, and built in support for ceph, backups (with proxmox backup server)… proxmox just had a little more maturity behind it. I’ll be watching incus though.
> aspects are appealing (creating a vm/container via cli)
Nothing is stopping you from doing this with Proxmox, right?
I was looking to setup Proxmox for my homelab soon but this comment got me interested in Incus. Mostly because I've never heard of any Proxmox alternatives before this. You can try out Incus in your browser here: https://linuxcontainers.org/incus/try-it/
The demo does take ~10m to get into a working instance.
Their site might be getting hugged, even the non-demo page is taking ages to load.
Interesting. Does Incus has support for storing virtual machine assets in a NFS store so they could be easily migrated?
Looks like Incus has no GUI?
Proxmox has nice web GUI
It has one[1] (optional). Proxmox has a shittier, but more featureful, web UI.
[1]: https://blog.simos.info/how-to-install-and-setup-the-incus-w...
i like the proxmox web ui.
also, looking at the link you posted, it looks like incus can only do like a fraction of what proxmox can do. is that the case or is that web ui a limiting factor?
Dang it! I’ve just got comfortable with Proxmox, but now I have to start looking into Incus because of your comment.
Proxmox is free, too.
Incus looks nice, though it looks to be more API driven , at least from the landing page. I can't attest to Proxmox in a production/cluster environment but (barring GPU passthrough) it's very accessible for homelab and small network.
GPU passthrough works fine? I use that for transcoding in Jellyfin.
Takes 10 minutes to setup and one reboot. Works flawlessly, Linux or Windows. vGPU is a different story though.
I don't remember if I tried and failed, or if it seemed too much for me... I have an Arc A series; if you have a verified guide I would like to take a look!
This may have more to do with your underlying hardware than proxmox itself.
It's been forever, but to do passthrough you need proper bios support and configuration.
I tried demo'ing Incus from their "Try it online" page but it just spins endlessly and nothing happens.
Proxmox is entirely free and very, very reliable. Personal preference is fine, but I really don't think any of your claims are true.
I thought you had to pay a fee to access their updates repository? It's been a while though so I may be mistaken.
There's community update repositories you can swap out.
I'm not really sure what the difference is.
The difference: the free repositories get the updates before the enterprise repo.
So, the software versions that go into the enterprise repo are considered stable by then.
(If we're talking about Proxmox, that is.)
Enterprise repos are better tested and somewhat guarantee updates don’t explode anything.