Creating a Proxmox cluster with 3 old laptops

Summary

This video script details the process of creating a 'laptop cluster' using older laptops with Sandy Bridge and Ivy Bridge architectures, running Proxmox for virtualization. The experiment explores cluster stability, performance, and the feasibility of using such setups for virtual machines. It discusses the advantages of using laptops for low power consumption and ease of access, as well as the limitations in upgradability and storage. The script also covers the challenges faced with high availability and shared storage using Ceph, concluding that while the project is educational, it may not be practical for long-term use due to configuration complexities and hardware limitations.

Takeaways

• 💻 The video discusses creating a 'laptop cluster' using old laptops as nodes, leveraging their low cost and availability.
• 🔌 The laptops have been upgraded with additional RAM and gigabit Ethernet to meet the minimum requirements for running a hypervisor like Proxmox.
• 📈 The creator aims to test the stability, performance, and overall experience of using these old laptops in a cluster configuration.
• 🔄 The video highlights the challenges of setting up a cluster with only two nodes due to voting issues in Proxmox, which requires a third system for a more robust setup.
• 👍 Advantages of using laptops for a cluster include their small size, low power consumption, and built-in peripherals, which reduce the need for additional hardware.
• 👎 Disadvantages include limited upgradability, especially for memory and storage, compared to desktops or servers.
• 🛠️ The creator installs Proxmox VE 7.1.2 on all laptops, a free hypervisor known for its low hardware requirements and ease of setup.
• 🔄 The video demonstrates the process of creating a cluster in Proxmox, including setting up nodes and handling migration between them.
• 🔒 The creator discusses the importance of CPU compatibility when migrating VMs between different CPU architectures and the need to adjust settings accordingly.
• 🔄 The script covers the setup of High Availability (HA) in Proxmox, which allows VMs to be automatically restarted on other nodes in case of a node failure.
• 💾 The video also explores the use of replication and shared storage solutions like Ceph for data redundancy and improved VM migration capabilities.
• 🚧 The creator concludes that while the laptop cluster is a good learning experience, it may not be suitable for long-term use due to configuration complexities and performance limitations.

Q & A

What is the purpose of creating the 'laptop cluster' as described in the script?

The purpose is to utilize older laptops, which are essentially free for the user, to create a cluster using Proxmox and evaluate the experience, stability, and performance of such a setup.

Why are the 10-year-old laptops considered suitable for this project?

These laptops are suitable because they have quad-core processors, which provide a reasonable amount of performance, and they have between 8 to 12 gigabytes of RAM, which is the bare minimum for running Proxmox or another hypervisor.

What network improvements were made to the laptops for the cluster setup?

A gigabit Ethernet adapter was added to the laptop that only had a 100-megabit integrated network card to ensure reasonable network speeds for the cluster.

Why is having a third system in the cluster important for the voting mechanism in Proxmox?

A third system is important because Proxmox requires a certain number of votes for decisions in a cluster. With two systems, if one fails, the cluster cannot reach the required number of votes, but with three, there is always a majority to agree on actions.

What is the role of the older laptop with a Core 2 Duo system in the cluster?

The older laptop with a Core 2 Duo system serves as the third vote in the Proxmox cluster, ensuring that there is always a quorum for decision-making even if one of the faster systems goes offline.

What are some advantages of using laptops for a cluster as mentioned in the script?

Advantages include easy availability, small size, low power consumption, built-in keyboard, video monitor, and mouse, as well as a built-in battery that can act as a UPS.

What are the disadvantages of using laptops compared to small form factor desktops or servers?

Disadvantages include limited upgradability, such as a maximum of two hard drive bays in most cases, and limited memory upgradeability. Some laptops may also have lower-end network cards that can limit performance.

What software was installed on the laptops for the cluster, and why was it chosen?

A plain install of Proxmox VE 7.1.2 was installed because it is a free hypervisor that the user is comfortable with and that generally works well on low-power hardware with low hardware requirements.

How does the script describe the process of creating a cluster in Proxmox?

The process involves creating a cluster on the first node and then using the join information to add other systems to the cluster. Proxmox makes it easy to manage the cluster as if it were a single system.

What issues did the user encounter when trying to migrate VMs between different nodes with different CPU architectures?

The user encountered errors when trying to migrate VMs because the CPU architectures (Sandy Bridge and Ivy Bridge) support different features. The user had to set the CPU type to 'kvm64' to resolve the issue.

What is the user's conclusion about using old laptops for a long-term cluster setup?

The user concluded that it might not be practical for a long-term setup due to the complexity and issues encountered, such as difficulties with configuration and shared storage using Ceph on such old systems.

Outlines

00:00

💻 Creating a Laptop Cluster with Proxmox

The video script details the process of creating a 'laptop cluster' using several older laptops, approximately 10 years old, with Sandy Bridge and Ivy Bridge architectures. The purpose is to harness their quad-core processors and 8-12 GB of RAM to run Proxmox, a hypervisor, and evaluate the stability and performance of such a setup. The creator addresses the challenge of setting up a cluster with only two nodes and the solution of adding a third, older laptop to serve as the 'third vote' for cluster decisions. Advantages of using laptops include their low power consumption, small size, and built-in components. However, the creator also notes the limited upgradability and memory upgradeability compared to desktops or servers.

05:02

🔄 Setting Up and Managing the Cluster

The script continues with the setup process of the cluster in Proxmox, including the initial configuration and the addition of nodes. It explains how the cluster allows management as if it were a single system and demonstrates the creation of virtual machines (VMs) across different nodes. The creator also discusses the limitations of using local storage and the inability to use resources from other systems by default. The paragraph concludes with an attempt to migrate a running VM between nodes, which results in an error due to CPU feature differences, highlighting the need to configure the VM to match the lowest common CPU features across all nodes.

10:04

🔄 Exploring High Availability and Data Replication

This section delves into the concept of high availability (HA) in Proxmox, which ensures VMs continue running on other systems if one fails. The creator discusses the limitations of using local storage for HA and introduces the idea of replication, which involves copying VM data to another system at regular intervals. The script provides an example of setting up replication for a Windows Server 2019 VM and explains the use of shared storage as an alternative to local storage, mentioning the setup of a basic Ceph storage system using additional SSDs in the laptops.

15:06

📊 Assessing Performance and Usability of Ceph Storage

The script discusses the performance of the Ceph storage system set up on the old laptops, noting the slow write speeds and overall degraded performance. The creator uses benchmarking tools to illustrate the performance issues and shares insights on the potential causes, such as the need for network communication between systems and the limitations of using SSDs in a distributed storage setup. The paragraph also touches on the usability of Ceph for light use cases but advises against it for more demanding applications due to its performance limitations on the old hardware.

20:06

🚫 Challenges and Lessons Learned from the Laptop Cluster

In the final paragraph, the creator reflects on the challenges encountered while setting up the laptop cluster and the lessons learned. They express concerns about the complexity of managing a cluster on old hardware and the difficulties faced with Ceph storage, especially after simulating a node failure. The script describes issues with VM booting and data integrity after attempting to migrate VMs and test high availability. The creator concludes that while the experience was valuable for learning about clusters and failure handling, it might not be practical for long-term use or in new setups due to the complications and performance issues encountered.

📚 Conclusion and Call for Audience Experience

The video concludes with a call to action, inviting viewers to share their experiences with setting up clusters on old systems and how they have fared. The creator summarizes the project and its outcomes, highlighting the educational value of experimenting with clusters on physical hardware and the insights gained about the practical limitations of using old laptops for such purposes.

Keywords

💡Laptop Cluster

A 'Laptop Cluster' refers to a group of interconnected laptops working together to perform tasks that are typically handled by more powerful servers. In the context of the video, the creator is attempting to use old laptops to form a cluster using Proxmox, a hypervisor, to see how well they perform and cooperate as a unit. The idea is to repurpose older technology for new uses, demonstrating resourcefulness and the potential for older hardware to still be viable in certain applications.

💡Proxmox

Proxmox is an open-source hypervisor based on the Linux kernel that provides a platform for virtualization. It is used in the video to create a virtualization environment across the old laptops, allowing them to function as a cluster. The script discusses the setup and management of this environment, highlighting its ease of use and the challenges faced when dealing with a cluster of older hardware.

💡Sandy Bridge and Ivy Bridge

Sandy Bridge and Ivy Bridge are microarchitectures for Intel's processors, which were prevalent about a decade ago. In the video, the creator mentions that the old laptops have these architectures, indicating their age and the performance level they can offer. These terms are important as they set the expectation for the hardware capabilities of the laptops being used in the cluster.

💡Quad Cores

A 'Quad Core' refers to a processor with four independent processing units, or cores, capable of handling multiple tasks simultaneously. The video mentions that the old laptops have quad-core processors, which is significant because it suggests they have a 'reasonable amount of performance' for their age, making them suitable candidates for the cluster project.

💡Gigabit Ethernet

Gigabit Ethernet is a transmission technology based on the Ethernet frame format and protocol used in local area networks (LANs), which provides a data rate of 1 billion bits (one gigabit) per second. In the script, the creator adds a Gigabit Ethernet adapter to one of the laptops to improve network speeds, which is crucial for the cluster's performance and communication between nodes.

💡High Availability (HA)

High Availability (HA) in the context of the video refers to the system's ability to remain operational and functional despite the failure of some of its components. The creator discusses setting up HA in Proxmox to ensure that if one laptop in the cluster fails, the virtual machines can continue to run on another system, demonstrating a key feature of cluster configurations.

💡Replication

Replication in the video refers to the process of creating and maintaining copies of virtual machines on different systems within the cluster. This is a form of redundancy that ensures data availability and business continuity. The script describes setting up replication for a Windows Server 2019 VM, which would allow it to be restored on another system in the event of a failure.

💡Ceph

Ceph is a distributed storage system that provides a unified platform for block storage, object storage, and file storage. In the video, the creator attempts to set up a Ceph cluster using additional SSDs in the laptops for shared storage across the nodes. However, the performance and stability issues encountered highlight the complexity of implementing distributed storage solutions on older hardware.

💡ZFS Send/Receive

ZFS Send/Receive is a feature of the Z File System (ZFS) that allows for the efficient replication of data between ZFS storage pools. In the context of the video, it is used to facilitate the replication process in Proxmox, ensuring that VM data is copied from one system to another within the cluster, which is vital for the high availability setup.

💡VM Migration

VM Migration is the process of moving a virtual machine from one physical host to another with minimal downtime. The video script discusses the ease of migration within the cluster, especially when using shared storage like Ceph, and the importance of this feature for high availability and load balancing in a cluster environment.

💡e-waste

e-waste refers to electronic devices that are no longer needed or are obsolete and are discarded. The video script mentions obtaining old laptops from e-waste bins, emphasizing the environmental aspect and the potential for repurposing such devices in creative projects like setting up a laptop cluster.

Highlights

Creating a laptop cluster using older laptops to explore performance and stability with Proxmox.

Utilizing Sandy Bridge and Ivy Bridge architecture laptops with quad-cores for reasonable performance.

Ensuring a minimum of 8GB RAM for running Proxmox or another hypervisor.

Adding a gigabit Ethernet adapter to improve network speeds for the cluster.

Challenges of setting up a cluster with only two nodes due to voting and agreement requirements.

Using an older laptop or a Raspberry Pi as a third node to facilitate voting in the cluster.

Advantages of using laptops for a cluster: availability, low power consumption, and built-in peripherals.

Disadvantages include limited upgradability and memory upgradeability compared to desktops or servers.

Using Proxmox VE 7.1.2 for its low hardware requirements and suitability for low-power hardware.

Initial setup of the cluster includes hostname and IP address assignments for each node.

Creating and managing a cluster in Proxmox allows treating multiple systems as one entity.

Migrating VMs between nodes requires compatible CPU types and may involve copying disk images.

Issues with CPU feature compatibility between different generations of Intel processors.

Using replication in Proxmox to enable high availability for VMs in case of node failure.

Exploring the use of Ceph for shared storage in the cluster, despite performance limitations.

Performance benchmarks for Ceph storage showing moderate read speeds but slow writes.

Migrating VMs with shared storage is faster due to not needing to copy disk images.

Testing high availability and experiencing issues with VM booting after node failure.

Learning from the experiment that small clusters with old hardware can be challenging for distributed storage like Ceph.

Recommendation to use simpler setups for home servers and caution against complex configurations on old systems.

Overall conclusion that while the laptop cluster experiment had educational value, it may not be practical for long-term use.

Transcripts

today i'm going to be creating what i

want to call the laptop cluster i have a

few of these older laptops that are

about 10 years old now and i'm just

going to try putting them in a cluster

using proxmox and see how it goes see

how the experience works see how it is

stability wise see what type of

performance i can get out of it and just

see how the overall experience is

the main reason i'm trying this is

because laptops like this for me are

essentially free these are about 10 year

old laptops running sandy bridge and ivy

bridge architectures they luckily have

quad cores though so they have a

reasonable amount of performance i

believe these systems have between 8 and

12 gigabytes of ram which is what i'd

say the bare minimum for running like

proxmox or another hypervisor is

and i've added a gigabit nick one to the

one that only has 100 megabit integrated

nick and this guy is a built-in gigabit

nick so those reasonable network speeds

going on right now

now one issue is i only have two of

these reasonably fast laptops i can use

right here

and proxmox in most clusters get kind of

unhappy with two clusters because you

have a problem that if one node dies

then you only have one node in the

cluster the way that proxmox works with

voting is it has to reach a certain

number of votes to have anything happen

in a cluster and with two devices that

means both devices would have to agree

and if one node's offline

they can't that can't happen because one

of the nodes is offline and can't agree

so you can either set it so that one

node is essentially the master and says

what always happens

or you can make it so that you don't

need less votes to agree but basically

both of those were in the high

availability you can do it in a home lab

setting if you need to

but the best solution is actually just

get a third system that i have down here

this laptop is even older running a core

2 duo system with about 2 gigs of ram

but it's more than enough just to be

that third vote in proxmox i don't plan

on running any vms or containers on it

and you can also use something like a

raspberry pi for this use case you just

need enough horsepower for it to

basically be the third vote

so then if either the faster systems go

offline or just the third system there's

always two votes left and the two

systems can keep running as a full

cluster

now one thing you might think is why

might you want to use laptops well the

first reason is they're pretty easy to

get your hands on at least for me i seem

to get these fairly free from e-waste

bins and other places like that and just

find them used from other people

relatively easily

the next thing is um they're pretty

small low power and you can kind of tuck

them away easily these are opened up and

big right now but i can close the lid on

these stack them in a small pile and

kind of

tuck them in a corner pretty easily and

they use fairly minimal space

and also because they're designed to run

off battery power they use quite little

power compared to some other systems

so all three of these systems

together should be less than 100 watts

and probably like 30 watts 40 watts at

idle would be my guess as laptops are

typically quite good at very low idle

power consumption

the other advantages they essentially

have a built-in keyboard video monitor

and mouse so you don't have to add

anything if you want to administer a

system and they also have a built-in

battery which essentially works as a ups

so i can yank the power cord and the

system will just keep going so i don't

need an additional ups but the

disadvantages compared to a small form

factor desktop or a full on server is

much less upgradability a nice laptop

might have two hard drive bays but

that's pretty rare if with some newer

models and smaller ones and you're not

going to get more than that like you can

with a desktop or server

relatively limited memory upgradeability

this guy is fairly high-end and supports

32 gigabytes of memory but a lot of

systems from this era don't and it can

often be quite expensive to find the

largest dimms that you need to get that

32 or 16 gigs of ram that the laptop can

support up to

um like this guy here has here a good

amount of low end laptops put 100

megabit network cards instead of gigabit

which really limits things like nas uses

i'd say streaming any type of shared

storage on the network i'd really

strongly suggest you get a um

usb 3 to one gigabit nick because i find

it works better than 100 meg but neither

are great solutions

as for software i've just put a plain

install of proxmox ve 7.1.2 on all of

these which is just the newest version

of proxmox at this time

i chose proxmox as a free hypervisor i'm

pretty comfortable with it and it

generally works quite well on fairly low

power hardware like this as it has

fairly low hardware requirements itself

i've done some of the initial setup like

going through the installer updating

these systems and doing kind of my basic

setup for all proxmox systems i talked a

little bit more about that my proxmox

setup guide a few videos ago if you want

to see more details

so now i've gotten the three laptops

pulled up on my computer screen here i

have them open in the browser and in a

terminal with ssh and they look like

normal blank proxmox nodes nothing has

been set up i gave them all their own

host names and ip addresses

so let's go take a look at creating that

cluster so i'm going to take a look at

my first node here and just create a

cluster and let's call it the

lappy and it's going to start creating

the cluster right now and i've created

it

so now i can see the um join information

here which i want to copy and paste into

my other systems and that way they can

become part of the cluster proxmox is in

general pretty easy when it comes to

setting up clusters

so now my cluster is being created

looking in the web gui the biggest

change on the left is now i see my two

other laptops here and i can see and

manage all of those settings just like

if i was on the web page

so the easiest thing the cluster lets

you do is manage it as if it was one

system

also i can look in the command line with

pve cm status and it shows me my three

systems i'm running here the fact that

they're all votes are happy here shows

all the system shows how many votes they

have and everything is working correctly

in this cluster so now let's take a look

at creating a vm so i've uploaded an iso

of open sustain here and now i can see

my multiple nodes that i can put it on i

can call it like

open susse1

um and then i can just create the vm

like i normally would be able to on the

node so

nothing really special creating this vm

on here

now the one thing is when i'm creating

the vm i can still only see storage and

other devices on the local system so

just because i have the cluster and i

can see all the systems doesn't mean i

can use the storage or any of the

resources from the other systems by

default and it essentially just lets me

manage the other systems so now i've

installed a few vms on my cluster and

let's play around with running them on

different systems so i have a ubuntu vm

that's currently shut down and i have a

windows 10 vm that's doing what windows

10 loves to do and updating

so i'm going to take the shutdown ubuntu

vm and now i can actually migrate it to

other systems

so i can pick a different node in the

system so let's do laptop 2 or this guy

my iv bridge quad core

and it says it has a relatively large

disc so it might take a while

and with migrating between the systems

it will copy the vm configuration so all

the details of how the vm is made and

then it will also copy the disk image

that's what's going to take a while so

if i look at my screen here it's about a

gig out of 32 gigs down copying that 32

gigabyte vm image to the other system

and this isn't limited to be being done

on shutdown systems i can also do this

on a running system here

so it might take a while it also warns

me that and it also migrate my running

vm

but actually when doing this i got a

little bit of an error here which is

saying that it can't move it due to

missing instructions

so one of the things in proxmox that's

basically ignored at least by me if

you're running a single node is the type

of cpu you're running so if you go under

memory processors as an edit by default

it will do the kvm 64 which is that and

i normally recommend just using a host

with a single system

and host is just going to copy whatever

cpu your system has and just show that

to the vm

but if you're migrating between multiple

nodes like i'm doing here and especially

since these are different generations of

this one being a sandy bridge and this

one being an ivy bridge they support

different features

so i'm going to have to normally set to

the lowest common denominator which

should be sandy bridge here

let's take a look at that so this one

says it's set to sandy bridge but it

doesn't seem to be working because both

of these are at least sandy bridge

so i'm going to have to take a look at

why it doesn't like sandy bridge and

what exactly it's complaining about

so it looks like with a bit of googling

the ibrs involves some of the specters

mitigations that will be force required

if it has it if you use the ibrs one if

you don't use it it won't force require

it

but it's still interesting it doesn't

want to migrate my one with the sandy

bridges i believe they should both be at

least sandy bridge so once this finishes

i'm going to try turning it down to just

a kvm 64 and see if that will migrate

and taking a look at my ubuntu shutdown

vm that's finished it looks like it's

copied all the 32 gigs of the virtual

machine disk successfully so it's on the

new system and on the left i can see it

on the laptop too and let's just try

firing it up on this guy right now

so it looks like it started and it's

throwing that same error that it can't

start using that function with sandy

bridge so let's so i took a little bit

of a deeper look to see if i could

figure out what's going on google didn't

have an obvious answer other than just

use whatever the host says in kvm which

does work and taking a look at the cpu

features i ran a little online

difference and it looks like there's a

few features like smx aes and la hf

lm that are different that the older

system has this new one doesn't have and

i'm guessing that's the issue i'm not

sure why it isn't supported to my

knowledge intel didn't remove those

between generations

but it might also just be due to the

bios configuration or however dell set

the system up as it's often that those

cpu features that are disabled by either

the bios or the system manufacturer or

other things

but it does look like in this use case

setting it to kvm 64 does work fine so

i'm just going to do that for right now

firing up the console i can see that my

ubuntu vm is working correctly on the

other system just as i'd expect now that

i have a proxmox cluster set up and i

have migration working between my

laptops let's try to do a little bit

more of clustering and try to make these

systems work together a little bit more

so one way to do that in proxmox is ha

or high availability and the idea behind

it is if one of these laptops was to die

it'll keep running my vms on another

system now it's not perfect because it

shuts down the vm when one of the

systems dies and then the other one

fires it up in a little bit but it's

still pretty good and doesn't really

require any special software you just

have to have it start up when the vm

restarts

but the problem with this and my current

configuration is since i'm using local

storage on both of these laptops if this

system were to go down all of its vm

data and vm disks would now be

unaccessible to the other systems to

boot from

so there's a few ways to get around that

the easiest way in proxmox is with

replication so i'm going to take a look

of an example system i have replication

set up with here and replication

essentially just copies the data from

one vm to on one system to another vm on

another system by default it's every 15

minutes so the worst case scenario is

you have an image that's 15 minutes old

if it was to fail

and it'll also use zfs send and receive

under the hood to make this work so

looking at my system i have vm 103 right

now which is my windows server 2019

system set up that replication if i take

a look under data center application i

can see that it's on laptop one but it's

being replicated to laptop two every 15

minutes

so if i look at my data i'm storing on

here i can see that i have my vm 103

disk on the host that it's running off

of which makes sense and i also have it

on the other host that doesn't have the

vm and that way if this guy dies it

already has the data essentially on here

maybe a little bit old and can just

restart it on here

so that's pretty simple works pretty

well and if i was to fail a system it'll

just

keep working that way

now the other way to do it is to have

shared storage and the idea behind this

is you have some sort of storage system

that all the computers in the cluster

like all these laptops can access at the

same time and everyone can see it unlike

local storage where only that system can

see it

there's a few ways to do that like

having a separate nas or sam that

everyone can access but i already have

three laptops here so i don't want to

set up more than that so what i'd want

to do is try to make it so that each of

these laptops has an additional drive in

it and it uses that drive to share

storage

and in proxmox ceph is the best way to

set that up i am not an expert in seth

and really don't know that much but

proxmox is pretty plug and play and i

probably made one of the worst setups

you can create

both of these laptops have an extra ssd

in it and i just said create a cosd with

that in there

and taking a look at my cluster right

now

if i look at ceph on my systems i can

see that i have two nodes in my cef both

of the osds are working correctly

and it says it's really degraded and my

guess is this is due to that wanting to

have three copies of all my data but i

only have two systems that can store it

so it only has two copies i believe this

is edible somewhere but i can't find it

right now so it's going to stay in this

mode it does mean if one of these

systems were to go down it will keep it

running now one thing i was curious

about when setting up the ceph

configuration was how performance is

going to be like because compared to a

traditional single internal drive seth

has a lot more going on it has two

systems that has to talk to it has to go

over the network there's just a lot more

happening

so i installed a vm on ceph and ran the

most basic disk benchmark with crystal

disk mark on windows 150ish megabytes

per second read

44 right and those random writes are

really bad and my guess is that's

because it has to like confirm the right

with the second system and go over the

network every time there's a write

that's pretty slow and i really don't

know how exactly to speed up faster

networking would do it you might be able

to have it so one local drive confirms

it and then it writes it out later to

some sort of like caching solution

i'm not sure but i believe that a

commercial like a correctly set up seth

can do it

now the question is is this enough

performance

and if you're just playing around it

kind of is so i'm on windows 10 i'm just

going to fire up like edge for example

it takes a little bit longer than i

expect but it's kind of between hard

drive and ssd in most uses

i'd say this is usable for light use but

i probably wouldn't want to push it too

hard another thing i did to take a look

at ceph performance was using the gnome

disk benchmark tool so taking a look at

it here i can see those speeds popping

up really high likely due to ram cache

and then going down as probably the ram

cache isn't used as much

latency is .2 milliseconds average but i

can see some spikes in the couple of

milliseconds

and the write speeds are pretty crummy

just as crystal disk mark showed here

it's usable but this is one of the

longest windows installs i've done when

this guy was installing so i'd say

barely usable and this is two ssds on

two old laptops if you set it up

correctly seth can be very fast but this

is not a correct ceph setup

i'd probably stay away from using ceph

like this on these old systems while it

works it's pretty crummy and you're

really pushing a limit of what you can

do if seth i'd probably stick to

replication if you need to have an h a

solution now the one big upside with

steph and any other solution where all

the systems can access the data is that

migration works a lot better so let's

take a look at migration on here so i'm

going to migrate my windows 10 vm over

to the laptop system and because it's

shared storage it doesn't have to touch

the storage at all all it has to do is

copy the ram from this system to this

system and it'll just migrate the vm so

in this case it's super fast to migrate

only like a couple of seconds less than

a minute whereas it would take a lot

longer if it didn't have the storage

copied already so now it's almost done

copying the vm state 16 milliseconds of

downtime and i could take a look at my

virtual machine and now it's running on

a different host and works just fine the

vm doesn't even notice that it got moved

between hard systems now let's take a

look at testing that high availability

here and seeing how well that works

so i'm going to set up some high

availability in proxmox i set it up for

vms 103 which is my one that's being

replicated to the other system and i'm

also going to set it up for my one on

ceph storage so that's my windows 10 bvm

and i'm going to set it up with a group

i called good ones which is these

laptops that can actually run the vms if

you don't do it sometimes it put it on

my third laptop which can't run the vms

and you run into a lot of weird issues

so i'm going to add this guy to it and

then if this guy is to die which it's

going to do right now i'm just going to

go kill it

it's going to automatically it should

move all the vms to this system when it

detects it proxmox says this is a up to

two minutes

thing but we'll see how long it actually

takes

so it's detected that this laptop's dead

and now i can see that these two vms or

three of them actually are now running

on this system now

and it's they had to reboot because they

went offline it did force the vms off

you can't do it where it keeps the ram

state

but if i take a look at the console

right now it's still a functional vm

my luck on camera seems to be pretty bad

i did quite a bit of testing for this

video and did mini power off cycles and

just showed how everything would migrate

successfully from one system to the

other but every time i seem to try to

show it on camera it doesn't work and i

think i just found a way to brick two

vms

my windows 10 bvm that was stored on

ceph

is now just not happy it'll boot into

like the windows 10 logo where has the

dots and then it just dies and restarts

and then my windows 2019 system wasn't

happy when i migrated over to the

replicated disk says it can't boot from

the volume

and won't boot ever again

so i just broke two vms with trying to

migrate it

um there might be a way to look in

snapshots and get older versions of the

data and i don't know enough about ceph

to try to get this working correctly

but it just doesn't seem happy with what

i've done here

taking a little bit of a closer look at

ceph on here sef was not happy to lose

one of its nodes i'm guessing ceph is

kind of like proxmox where it really

wants three or more nodes so when it's

with two nodes and one of them dies it's

just kind of like i don't know what to

do and it just fails

um that's what it looks like

it showed it it's looking a little bit

better now because it detects both

systems are up

but it's i wasn't happy with two systems

you want to probably have the third

system actually in ceph all running but

i'm not going to get into ceph too much

more here

i've had replication work fairly around

proxmox just not now it seems to be so

overall is this something i'd want to

set up more for the long term or in a

new setup my answer is probably not for

me personally these laptops turned out

to be a lot more of a bayer and proxmox

to use than i expected i thought this

would be a pretty simple cluster to use

but i think i just found a way to keep

breaking things in ways i didn't expect

i probably want to do just a full wipe

on all these systems if i was to use it

more because i think i just have

something weird in my configuration now

that it really just doesn't seem to like

i think the other thing i've learned is

don't do shared storage like ceph on

such a small system it just really isn't

happy you gotta make sure these

distributed systems are set up with a

good configuration i've tried multiple

times to have a distributed storage on

old systems and it never works out well

you kind of just can't do it at a small

home level like this with older systems

you need to actually throw correct

hardware to get distributed storage

working correctly

but if you just want to play around with

clusters this is a pretty good way to do

it these laptops are very cheap now you

can get the kind of feel for setting up

clusters on real metal instead of having

a cluster running in a vm

and i think it can have a lot of

learning in how to set up clusters how

to handle failures and stuff like that

that you couldn't do if you just had a

single node but if you just want a

simple home server or something i'd do a

single node possibly set up two and

proxmox but don't do anything fancy just

migrate vms every once in a while if you

need to

thanks for watching this video on

setting up a laptop cluster of these old

systems let me know if you're using our

cluster on old systems like this and how

it's working for you