Technically, It Worked

How a local AI server became a router, an automation platform, three game servers and almost everything except the inference machine I wanted.

The first local model I ran on my Minisforum UM890 Pro worked.

Technically.

It loaded, accepted a prompt and eventually produced an answer. For a few minutes, I felt like I had built my own private AI server.

Then I noticed the speed.

Then the size of the model.

Then the difference between being able to run a model and being able to run the models I actually wanted.

The machine had plenty of RAM, but nowhere near the GPU compute required for the kind of local AI setup already forming in my head.

That should have sent me directly towards better hardware.

Instead, I did everything else.

The local LLMs were always the goal

I did not buy a mini PC to create a traditional homelab and then accidentally discover local AI.

The local LLMs were the reason for the machine from the beginning.

I wanted a computer that could remain online independently of my main PC. I wanted to run models privately, expose them as local APIs and experiment with agents, tools and workflows without paying for a new cloud service every time I had an idea.

Before buying anything, I spent far too much time comparing mini PCs.

There was always another option.

One had a better processor. Another had better cooling. Another was slightly cheaper, provided I accepted less memory. Some were clearly powerful enough for what I needed at the time, which made them less interesting than the ones that were probably excessive.

I eventually chose the Minisforum UM890 Pro with 64 GB of RAM.

Honorable mentions: Beelink SER8, and Geekom A7 Max.

I had a feeling the RAMpocalypse was coming.

I did not have data to support this theory. I simply saw models getting larger, applications consuming more memory and local AI becoming increasingly accessible. Buying a smaller configuration felt like something I would regret almost immediately.

For once, the unnecessary amount of RAM was not the problem.

The 64 GB gave Proxmox enough room for virtual machines, containers, automation services, game servers and most of the other things I would later install.

What it did not give me was a discrete GPU.

The UM890 Pro is a capable mini PC. Its integrated graphics are useful for many things.

Those things do not include comfortably running the large local models I was already imagining.

The first response

The first models I ran with any degree of success were Phi-3 Mini and a higly quantised DeepSeek model through an LM Studio server.

Seeing the first response appear felt great.

The prompt had not been sent to a hosted API. There was no remote provider generating the answer somewhere else. A small computer inside my home had loaded the model and produced the text locally.

I could expose the LM Studio server on my network and allow other applications to use it as an inference endpoint.

Technically, I had achieved the original objective.

That feeling lasted approximately five minutes.

The models were small. They were not always particularly clever. The responses were slow enough to remind me that the machine was thinking, although “thinking” may have been a generous description of what was happening.

The 64 GB of system memory meant I could load quantised models that would not fit on a typical laptop.

The lack of GPU compute meant I had plenty of time to appreciate each token individually.

I had focused on whether models would fit in memory.

I had not focused enough on whether using them would feel good.

Those are two different questions.

“It runs” is a technical result.

“It is fast and capable enough that I actually want to use it” is another one.

The UM890 Pro had enough memory to construct the entire neighbourhood around the model.

It did not have the GPU required to let the model move in comfortably.

Before the AI, I apparently needed four VLANs

Once Proxmox was installed, I had to choose the first service to deploy.

The logical choice would have been an AI-related application.

Instead, I installed OPNsense.

My reasoning was that I already had a server running continuously, so I might as well virtualise my router rather than buying another machine.

This sounded efficient.

Then I created four VLANs.

I did not need four VLANs.

I was not managing a hospital, a bank or a company with separate departments. I had some personal devices, a Proxmox server and several services that did not exist yet.

But those services were going to be organised.

I imagined a clean home network with trusted devices in one place, servers somewhere else, connected devices separated from everything important and perhaps another network for guests or experiments.

The diagram in my head looked professional.

The actual network mostly involved me changing firewall rules until the internet returned.

Sometimes devices could communicate across VLANs when they were not supposed to.

Sometimes they could not communicate at all.

Sometimes I could no longer remember which of those outcomes I was currently trying to achieve.

I spent a while maintaining the idea of a well-segmented network without receiving much practical benefit from the segmentation itself.

Eventually, I stopped using the VLANs.

Technically, they worked.

I simply did not need them.

I briefly made my internet depend on Proxmox

OPNsense lasted no longer than the VLANs.

For a while, it handled the routing for my home network from inside a Proxmox virtual machine.

Again, technically, it worked.

The problem was not OPNsense.

The problem was the person operating the Proxmox server.

This was my experimental machine.

I created and deleted containers on it. I changed networking settings. I installed software from GitHub repositories after reading enough of the documentation to become dangerous.

I unplugged cables.

I rebooted it when something looked strange.

My entire internet connection depended on that machine remaining powered on and correctly configured.

Turning off Proxmox also turned off the internet.

Rebooting the server became a household networking event.

This was particularly inconvenient when my girlfriend was using the connection and had not been informed that I was about to “quickly test something.”

The test was rarely quick.

I eventually removed OPNsense and returned the routing responsibilities to hardware that was not also being used as my playground.

I still like OPNsense.

I will probably buy a separate small machine for it eventually.

That way, the internet can continue working while I repeatedly destroy whatever is running inside Proxmox.

This feels like a better division of responsibilities.

Proxmox industrialised my ability to break Linux

At university, I reinstalled Linux seven+ times because I kept breaking it while exploring.

Proxmox was supposed to be the mature version of that behaviour.

I could create a VM or LXC container, take a snapshot, experiment and roll back if something went wrong.

Entire systems could be created, duplicated and destroyed from a browser.

This sounded ideal.

I then badly broke almost every virtual machine and container I created the first time.

Some environments did not survive long enough for me to take a useful snapshot.

I reinstalled certain containers more often than I had reinstalled Linux during university, which I had not believed was possible.

The usual process looked something like this:

1. Find an installation guide.
2. Follow most of it.
3. Decide that one step probably does not apply to my setup.
4. Modify something else for convenience.
5. Lose access to the service.
6. Find several forum posts describing a problem that is almost identical to mine.
7. Discover that everyone in those posts is using different hardware, a different operating-system version and a completely different network.
8. Reinstall the container.

Proxmox did not stop me from breaking Linux.

It made breaking Linux much more efficient.

I had finally found a way to reinstall entire operating systems repeatedly without leaving my browser.

Over time, I became slightly better at identifying what I had broken.

This did not necessarily prevent me from breaking it again.

It mostly improved the quality of my guesses.

Everything around the model

Once I stopped trying to make the Proxmox server responsible for my entire internet connection, I began installing the services I had actually bought the machine for.

I deployed Pulse to monitor the Proxmox environment.

I created an n8n LXC container for automation.

I installed SearXNG so my workflows and agents could search for information without relying entirely on a conventional search API.

I experimented with an Agent HQ for my language models and with Paperclip AI.

Each service added another container, IP address, hostname, port, credential and configuration file.

I like minimalist interfaces and clean workspaces.

I also enjoy installing interesting software.

These preferences are not compatible.

For a while, I tried to document everything correctly. Each service would have installation notes, dependencies, network information and recovery instructions.

Then I installed several applications during the same evening.

The documentation has not fully recovered.

Occasionally, I open the Proxmox interface and find a container whose name was apparently obvious to me at two in the morning several months earlier.

The machine could comfortably host all of these services.

It could run the automation, the search layer, the monitoring and the software intended to organise the agents.

The infrastructure around local AI was working better than the local AI itself.

Technically, I had built an AI stack.

The intelligence was simply the least convincing part of it.

Running a model was not enough

The first local models had already shown me that a chatbot was not what I really wanted.

I did not want to sit in front of LM Studio and ask a small model questions simply because the answers were being generated locally.

I wanted models that could do things.

I wanted them to search, call tools, interact with services, retain useful context and complete work across several steps.

That was when I discovered agent harnesses.

The model stopped being the entire project and became one component inside a larger system.

Now it needed tools.

The tools needed services.

The services needed containers.

The containers needed networking.

The networking needed me to remember what I had configured several weeks earlier.

Every attempt to make the AI more useful created another side quest.

A model capable of searching needed SearXNG or another search provider.

A model capable of completing a workflow needed orchestration.

An agent capable of changing something needed permissions.

Several agents working together needed some way to coordinate tasks, share context and avoid repeating the same mistake in parallel.

Then all of it needed monitoring so I could understand what was happening when the result made no sense.

At some point, I had bought a machine to run a model and ended up trying to build the organisation the model would work inside.

The UM890 Pro was not powerful enough to be the final inference machine.

It was perfectly capable of hosting the office around it.

The game servers arrived because the machine was already running

The Proxmox server was running all day anyway, which is the kind of sentence people use immediately before assigning a computer several additional jobs.

I started hosting game servers.

The machine now runs two modded Minecraft servers and one Satisfactory server.

“Some spare resources” is an optimistic description of what modded Minecraft considers sufficient.

The game servers gave the homelab actual users.

If one of my personal automation experiments stops working, I can ignore it until I feel like investigating.

If the Minecraft server stops working, someone notices.

They do not care whether the problem is coming from Proxmox, the VM, Java, the mod loader, a server-management tool or whichever configuration file I edited shortly before everything disappeared.

The server is either online or it is not.

Satisfactory creates a similar problem, except the server may contain a large industrial civilisation that several people have spent hours building.

Losing progress there could create actual diplomatic consequences.

The game servers were side projects, but they became some of the most consistently used services in the homelab.

Naturally, I eventually turned them back into AI experiments.

Minecraft needed an LLM

At some point, hosting Minecraft normally was no longer enough.

I built a bridge that allowed an LLM to interact through the Minecraft chat and modify server settings from inside the game.

This was not the simplest way to administer a server.

Opening a normal management interface would have been faster, safer and considerably less interesting.

The bridge allowed a player to write a request in Minecraft chat, have the model interpret it and then turn it into a change to the server configuration.

I mainly use it to tweak MineColonies settings.

Instead of leaving the game, finding the correct configuration file, checking the option name and restarting whatever needs restarting, I can ask for the change directly from the chat.

For example:

Increase the maximum number of citizens.

or:

Adjust this MineColonies setting for the current server.

The interesting part is that the model has to understand what I am asking for, identify the relevant configuration and change the correct value without touching something unrelated.

When it works, it feels much more natural than opening an admin panel or editing a file manually.

When it misunderstands the request, the result is no longer just a strange answer in a chat window. It can change how the colony behaves.

It can actually change something.

The bridge worked well enough for me to immediately start thinking about giving the model more capabilities.

Technically, the LLM could help administer the server.

This amount of success was enough to make giving it more permissions seem like a reasonable next step.

I had bought the server for local AI, become distracted by Minecraft and then placed the AI inside Minecraft so that the detour would look intentional.

The homelab occasionally does useful work

The server is not purely an expensive way to generate networking problems.

My n8n container now runs lightweight LLM-related workflows.

One of them performs automated lead discovery, gathers information and uses a language model to prepare material for cold calls.

The workflow searches for potential leads, collects context and generates a starting point I can review before contacting them.

This feels closer to what I originally wanted from local AI.

The model is no longer sitting in a chat window waiting for me to find something to ask it.

It has a role inside a process.

Sometimes the output is useful.

Sometimes it is generic.

Sometimes the model identifies a sales angle that no human should ever use during an actual phone call.

Sometimes SearXNG returns information that is technically related to the company but entirely irrelevant to the conversation.

Sometimes one container suddenly fails to resolve the hostname of another container, and the networking side quest returns for another episode.

Technically, the workflow works.

Whether every generated cold-call suggestion should be used on another person is a separate question.

It is not a finished autonomous sales platform.

It is a workflow running from a mini PC in my home that occasionally produces something valuable.

For now, that is enough.

Tailscale worked too easily

I also configured Tailscale.

After my experience with VLANs, OPNsense and virtual networking, I expected remote access to require another evening of troubleshooting.

Instead, it mostly worked.

Immediately.

This felt suspicious.

I then barely used it on my LAN.

This happens often in the homelab. I solve a problem before I have developed the use case that would make the solution necessary.

Tailscale remains configured for the day when I urgently need to administer a Minecraft server while away from home.

That day has not arrived yet.

It probably won't.

The computer I actually want

The UM890 Pro proved that I could run local AI.

It also made the missing part obvious.

The next machine needs GPU compute (even if I have an OCuLink port on the side I know).

My current imaginary configuration is built around a Ryzen Threadripper processor, 128 GB of system RAM and either two or four AMD AI Pro R9700 GPUs.

Two GPUs would already be excessive by normal personal-computer standards.

Four would provide up to 128 GB of total VRAM and enough compute for much larger models, longer contexts and several agents making requests at the same time.

It would also cost approximately the annual salary of an apprentice.

My current financing plan is therefore to convince someone to lend me the monetary equivalent of a junior employee so I can assemble a computer in my home.

The pitch still needs work.

The next version of the homelab may eventually contain four separate machines:

• a small dedicated box running OPNsense;
• the UM890 Pro running Proxmox, automation, search, monitoring, agents and game servers;
• a NAS;
• an irresponsibly expensive GPU AI workstation handling model inference.

At the moment, only one of those machines exists.

It works fairly well, provided I do not ask it to be the other three.

Current status

The local LLMs were always the goal.

The VLANs are gone.

OPNsense is waiting for its own machine.

The UM890 Pro runs automation, search, agents and three game servers.

The models still need a better GPU.

Technically, the homelab works.

More or less.