LM Studio vs Unsloth on AMD: From 13 to almost 43 Tokens per Second

I Got 43 Tokens per Second From an AMD iGPU. Sometimes.

I found the next unnecessary homelab experiment on Twitter.

Unsloth published this post, and it was enough to make me wonder how its local inference setup would behave on the hardware I already had.

So I tried it myself.

This was not the first time I had come across Unsloth.

While looking for ways to run useful AI models locally, the name kept appearing around:

• Efficient fine-tuning
• Lower-memory training
• Quantized models
• GGUF exports
• Consumer hardware
• Faster local inference
• Qwen releases

That is generally how software enters my homelab.

I see it mentioned once and ignore it.

I see it mentioned three more times and open seventeen tabs.

Eventually, it receives a virtual machine.

What Unsloth is

Unsloth is an open-source project focused on making open models more efficient to train and run.

It originally became known mainly for optimized fine-tuning, particularly LoRA and QLoRA workflows designed to use less memory and finish faster on limited hardware.

Unsloth Studio expands that idea into a local interface for:

• Downloading and running open models
• Training and fine-tuning models
• Loading GGUF and Safetensors files
• Comparing model outputs
• Exporting models to formats such as GGUF
• Serving models through a local API
• Running models locally without sending prompts to a cloud provider

For GGUF inference, Unsloth Studio uses llama.cpp underneath.

That made it relevant to my setup because I was already serving GGUF models through LM Studio.

I was not replacing one completely different inference engine with another.

I was testing what happened when another application configured and exposed the same general llama.cpp and Vulkan stack differently.

The machine that became responsible for this

My Minisforum UM890 Pro was not purchased because I enjoy benchmarking quantized language models inside virtual machines.

That simply became one of its responsibilities.

The machine has:

• An AMD Ryzen 9 8945HS
• 64 GB of physical RAM
• An integrated Radeon 780M GPU
• No discrete GPU

The benchmark itself ran inside a virtual machine with:

• 8 CPU cores allocated
• 36 GB of RAM allocated
• Access to the Radeon 780M through Vulkan
• A 65,536-token model context
• Four parallel model slots

So this was not a bare-metal benchmark.

The host had 64 GB available.

The VM received 36 GB because the rest of the homelab also had opinions about memory allocation.

Inside that VM, LM Studio had been serving a quantized Qwen3.5-9B model at approximately 13 tokens per second.

It worked.

Technically.

Then I installed Unsloth Studio.

With the same model, throughput increased to approximately 22 tokens per second.

Then I loaded Unsloth’s specialized dynamic Q4 variant.

The median reached 43.06 tokens per second.

For a few minutes, the Radeon 780M looked considerably more powerful than I remembered purchasing.

The individual results then ranged from 18.89 to 54.73 tokens per second.

The GPU had not suddenly become a workstation card.

Speculative decoding was simply having a very productive afternoon.

A disclaimer before the numbers

I do not currently have a fully stable benchmarking process.

I am working on one.

The current procedure is good enough to expose obvious differences between configurations, but it is not yet a polished benchmark suite that I would use to make universal hardware claims.

At the moment, the process still has several weaknesses:

• Only four requests contribute to each reported median
• The prompt represents one short sequential workload
• System temperature is not recorded consistently
• Host and guest resource utilisation are not captured together
• GPU utilisation is not included in the result artifact
• Cache state is difficult to prove completely
• Speculative-decoding statistics are read from server logs
• The process is not yet packaged into one reproducible script
• I have not run a separate output-quality evaluation
• I have not repeated every configuration across multiple sessions

The numbers are real measurements from this system.

They are not final conclusions.

This article is closer to a documented experiment than a definitive product review.

The benchmark process itself is now part of the project.

The result

Median decode throughput

data view

These are the three main configurations I tested:

Backend and model	Median TTFT	Median decode	Median total
LM Studio — Q4_K_S	381 ms	13.01 tok/s	10.23 s
Unsloth Studio — Q4_K_S	244 ms	22.24 tok/s	6.02 s
Unsloth Studio — UD-Q4_K_XL	336 ms	43.06 tok/s	3.66 s

This table contains two different comparisons.

The first comparison is relatively clean.

LM Studio and Unsloth Studio served the same Qwen3.5-9B-Q4_K_S.gguf artifact.

The second comparison changed the model artifact as well.

Unsloth’s UD-Q4_K_XL is still approximately within the Q4 quantization tier, but it is not the same quantization as Q4_K_S.

Therefore:

Unsloth reached 22.24 tok/s with the same model LM Studio ran at 13.01 tok/s.

And separately:

Unsloth’s specialized dynamic Q4 model reached a 43.06 tok/s median during this run.

Median total request time

data view

The first result is a backend and configuration comparison.

The second is a model-artifact comparison.

Combining them into “Unsloth is 3.31 times faster than LM Studio” would be excellent marketing and incomplete benchmarking.

The test environment

Diagram source: flowchart LR host["Minisforum UM890 Pro<br/>Ryzen 9 8945HS<br/>64 GB RAM"] vm["Virtual machine<br/>8 CPU cores<br/>36 GB RAM"] runtime["Radeon 780M<br/>Vulkan"] studio["LM Studio<br/>13.91 tok/s"] unsloth["Unsloth<br/>22.24 / 43.06 tok/s"] host --> vm vm --> runtime runtime --> studio runtime --> unsloth

The physical machine was:

Component	Configuration
System	Minisforum UM890 Pro
Processor	AMD Ryzen 9 8945HS
Host memory	64 GB
Integrated GPU	AMD Radeon 780M
Discrete GPU	None

The benchmark ran inside a virtual machine configured with:

VM resource	Allocation
CPU	8 cores
Memory	36 GB
GPU API	Vulkan
Model context	65,536 tokens
Parallel slots	4

The workload was kept identical between runs:

Benchmark procedure

procedure

1. 01Request 1Warm-up · discarded
2. 02Request 2Warm-up · discarded
3. 03Request 3Measured · median sample
4. 04Request 4Measured · median sample
5. 05Request 5Measured · median sample
6. 06Request 6Measured · median sample

• 48-token input prompt
• 128 generated tokens
• Temperature set to 0
• Thinking explicitly disabled
• Six sequential streaming requests
• First two requests discarded as warm-ups
• Medians calculated from the remaining four requests
• 120-second absolute deadline for each request

This was a short sequential-generation benchmark.

It was not a concurrency test.

It was not a long-context test.

It was not a test of whether the model could solve anything more complicated than producing 128 tokens without becoming distracted.

What running inside a VM changes

Running the benchmark inside a virtual machine has practical consequences.

The first is that these numbers do not represent the maximum theoretical performance of the UM890 Pro.

They represent the performance of this particular deployment:

UM890 Pro host
→ hypervisor
→ virtual machine
→ guest operating system
→ Vulkan driver path
→ llama.cpp server
→ model

Every layer introduces another place where performance can be affected.

Possible influences include:

• Virtual CPU scheduling
• Host processes competing for CPU time
• Guest memory allocation
• Host memory pressure
• Device passthrough overhead
• Vulkan driver behaviour inside the guest
• Shared memory bandwidth
• Host power management
• Thermal limits
• Background workloads elsewhere on the machine

The Radeon 780M is an integrated GPU.

It does not have a separate pool of dedicated VRAM in the same way a discrete graphics card does. It uses system memory, which means memory allocation and bandwidth are particularly important.

The VM received 36 GB of RAM, but that does not mean all 36 GB behaved like dedicated GPU memory.

The host, the guest, the CPU, and the integrated GPU still depend on the same physical machine and its available memory bandwidth.

Virtualisation can therefore reduce performance or add variability.

It can also make it harder to determine whether a slower request came from:

• The model
• The inference server
• The guest operating system
• The hypervisor
• Another process on the host
• A change in speculative acceptance
• The integrated GPU sharing bandwidth with the CPU

This is one reason I do not want to present the results as a general Radeon 780M benchmark.

They are a benchmark of a Radeon 780M inside my virtualised homelab configuration.

That distinction matters.

Why keep the VM at all?

The VM is not only a disadvantage.

It gives me:

• Isolation from the host
• Easier backups and snapshots
• A disposable testing environment
• Cleaner dependency management
• Separation between AI services and the rest of the system
• The ability to test updates without modifying the host
• A deployment closer to how I would actually operate a local AI service

Bare-metal benchmarking might produce better numbers.

The VM produces numbers that are more relevant to the way I intend to use the machine.

My goal is not to turn the UM890 Pro into a benchmark podium.

My goal is to make it run useful local services without allowing one experimental Python package to annex the entire operating system.

Eventually, I want to compare the virtual machine against a bare-metal installation.

That will help estimate how much performance I am trading for isolation and maintainability.

At the moment, that comparison does not exist.

The first result was too good

An earlier Unsloth run reported:

96.98 tok/s

I would have preferred to keep that number.

It was much more exciting.

Unfortunately, clean server reloads and cache clearing reduced the result to approximately 22 tok/s.

The original measurement could not be reproduced consistently.

That made it an anomaly rather than a benchmark.

Several things can distort a local inference test:

• Cached slot state
• Prompt reuse
• An old server process still running
• Differences in streaming measurements
• Speculative-decoding behaviour
• Incomplete model reloads
• Host resource contention
• Virtual-machine scheduling
• A benchmark script measuring something other than what I thought it was measuring

Local benchmarking is a good way to learn that computers can produce extremely precise numbers for experiments that were not controlled precisely enough.

After reloading both servers and clearing Unsloth’s slot cache, the measured Unsloth runs stayed between:

21.38–22.98 tok/s

That range was much less exciting.

It was also reproducible within that session.

So 22.24 tok/s became the useful result.

I am deliberately saying “within that session” because the process still needs to be repeated across clean restarts and separate test sessions.

The 96.98 tok/s number remains part of the story because hiding incorrect results does not improve the experiment.

It only improves the headline.

The same model became 1.71 times faster

Median time to first token

data view

The clean comparison used the same artifact on both servers:

Qwen3.5-9B-Q4_K_S.gguf

LM Studio produced:

13.01 tok/s

Unsloth Studio produced:

22.24 tok/s

That gave Unsloth:

• 1.71 times the decode throughput
• 36% lower time to first token
• 41.1% lower total request time

The difference was large enough to notice during normal use.

At 13 tokens per second, the model felt usable but deliberate.

At 22 tokens per second, it felt much closer to an interactive assistant rather than a machine carefully considering every word before releasing it.

The underlying physical hardware had not changed.

The virtual-machine allocation had not changed.

The model artifact had not changed.

The serving configuration had.

Speculative decoding did most of the work

Unsloth Studio launched its bundled llama.cpp server with:

--spec-default

This enabled speculative decoding.

Speculative decoding uses a faster draft process to propose future tokens. The main model then checks several of those proposals together.

When enough draft tokens are accepted, the server avoids performing every decode step individually.

During the clean Q4_K_S benchmark, draft acceptance ranged from:

0.63415–0.66102

Approximately 63% to 66% of the draft proposals were accepted.

That was enough to improve generation substantially.

This matters because both LM Studio and Unsloth were serving the GGUF model through llama.cpp and Vulkan.

This was not a comparison between llama.cpp and Unsloth’s Python or PyTorch inference stack.

The major practical difference was how the server was configured.

A previous control run started Unsloth with speculative decoding disabled:

--spec-type none

That configuration was slightly slower than LM Studio.

The exact non-speculative control still needs to be repeated using the latest clean benchmark procedure, but the current evidence points in one direction:

Unsloth’s advantage came primarily from its speculative-decoding defaults, not from an intrinsically faster non-speculative Vulkan path.

Defaults matter.

Especially when one application enables a useful optimization and another one does not.

Then I changed the model too

After establishing the same-model comparison, I downloaded Unsloth’s official specialized dynamic Q4 artifact:

Qwen3.5-9B-UD-Q4_K_XL.gguf

The repository did not provide a specialized UD-Q4_K_S artifact.

UD-Q4_K_XL was therefore the closest official specialized Q4 option.

It remained within roughly the same quantization category, but it was not identical to the previous model.

The downloaded file was:

5,966,095,584 bytes

Its SHA-256 hash was:

6f5d30666c2d8ae16a306e616d95341dcf3cc46810df84d7e6f5a7d1e4c1b293

The hash matched the official Hugging Face LFS object.

This confirmed that I had downloaded the intended model rather than accidentally benchmarking a corrupted six-gigabyte mystery file.

The model loaded successfully with the same:

• Virtual machine
• Radeon 780M
• Vulkan backend
• 65,536-token context
• Four parallel slots
• Prompt
• Generation length
• Temperature
• Sequential request procedure

Then it produced this:

Model in Unsloth	Median TTFT	Median decode	Median total
Q4_K_S	244 ms	22.24 tok/s	6.02 s
UD-Q4_K_XL	336 ms	43.06 tok/s	3.66 s

The specialized model delivered:

• 1.94 times the median decode throughput
• 39.2% lower total request time
• 37.6% higher time to first token

It began responding more slowly.

It then finished considerably faster.

This is why one performance number is rarely sufficient.

Time to first token affects how responsive the system feels.

Decode throughput affects how quickly the rest of the answer arrives.

Total time determines how long I actually spend looking at the terminal.

The median was fast

UD-Q4_K_XL request variability

data view

The specialized model’s four measured decode rates were:

Request	Decode rate
1	18.89 tok/s
2	33.27 tok/s
3	52.84 tok/s
4	54.73 tok/s

With four measurements, the median is the midpoint between the two central values:

(33.27 + 52.84) / 2 = 43.055

Rounded:

43.06 tok/s

The calculation is correct.

The stability is less convincing.

The fastest request was approximately 2.9 times faster than the slowest.

At 54.73 tok/s, the model felt extremely fast for an integrated GPU.

At 18.89 tok/s, it was slower than the ordinary Unsloth configuration.

The median describes the centre of those four observations.

It does not promise that the next request will run at 43 tok/s.

The draft model occasionally became very confident

Speculative draft acceptance for the specialized artifact ranged from:

0.69231–1.00000

Some requests accepted every logged draft proposal.

Those requests also appeared to produce the highest decode rates.

This makes speculative acceptance the most likely explanation for the variation.

But four measured requests are not enough to establish a reliable relationship.

At this point, I have a correlation, a plausible mechanism, and exactly enough data to become overconfident.

The next benchmark should record, for every request:

• Time to first token
• Decode throughput
• Total request time
• Draft acceptance
• Drafted token count
• Accepted token count
• Prompt-processing time
• Slot identifier
• Cache state
• Host CPU utilisation
• Guest CPU utilisation
• System temperature
• GPU utilisation
• Memory usage
• Other active host workloads

Then I can determine whether draft acceptance actually predicts performance rather than simply appearing nearby in the logs.

From 13 to 43 tokens per second

The most tempting comparison is:

LM Studio:                  13.01 tok/s
Specialized Unsloth model:  43.06 tok/s

That is approximately 3.31 times the decode throughput.

Median total request time also fell from:

10.23 seconds

to:

3.66 seconds

That is a reduction of approximately 64.2%.

But this was not one isolated change.

I changed:

1. The serving application
2. The speculative-decoding policy
3. The quantized model artifact

The more accurate sequence is:

Moving from LM Studio to Unsloth with the same Q4_K_S model increased median decode throughput from 13.01 to 22.24 tok/s.

Then:

Moving from Q4_K_S to Unsloth’s UD-Q4_K_XL artifact increased the observed median from 22.24 to 43.06 tok/s.

That explanation is longer.

It also survives contact with the benchmark data.

What this suggests

The experiment provides a few useful signals.

Unsloth’s defaults worked better here

With the same model artifact, Unsloth Studio was clearly faster for this short sequential workload during this test session.

Speculative decoding can make a large difference

The Radeon 780M did not become more powerful.

The server became better at avoiding unnecessary decode work.

The model artifact matters

Changing to Unsloth’s specialized dynamic quant almost doubled the median throughput inside Unsloth during this run.

Fast and stable are different qualities

The specialized artifact produced the best median result.

It also produced the widest range.

Depending on the workload, I may prefer a stable 22 tok/s over a configuration that alternates between 19 and 55 tok/s.

Or I may accept the variation because watching an integrated GPU briefly exceed 50 tok/s is entertaining.

The production decision remains unresolved.

The benchmark is appropriately named.

What this does not prove

I did not test whether the specialized model was equally capable.

There was no comparison of:

• Reasoning quality
• Factual accuracy
• Coding ability
• Instruction following
• Multilingual performance
• Tool use
• Long-context retention
• Perplexity
• Hallucination rate

A model producing text twice as quickly can also produce incorrect text twice as efficiently.

This benchmark measured inference performance.

Nothing more.

It also does not prove that:

• UD-Q4_K_XL is always faster
• 43.06 tok/s can be sustained
• Another prompt will produce the same speculative acceptance
• Concurrent requests will preserve the advantage
• Bare-metal performance will match the VM
• The VM matches bare-metal performance
• Another Radeon 780M system will reproduce the result
• Unsloth’s non-speculative path is faster than LM Studio
• My current benchmark procedure is fully stable
• Four measured requests are enough to characterize performance

The benchmark happened inside one VM on one UM890 Pro using one short prompt.

That is useful evidence.

It is not a universal law of local inference.

Building a benchmark I can trust

The next step is not only to test more models.

It is to improve the benchmark process itself.

I want one script that can:

1. Verify which server process is running
2. Record the exact server command
3. Record model filename and checksum
4. Confirm context and slot configuration
5. Clear or document cache state
6. Send warm-up requests
7. Run a larger measured sample
8. Save every raw response timing
9. Capture speculative-decoding statistics
10. Record host and guest resource utilisation
11. Produce p50, p90, and p95 results
12. Compare output quality separately
13. Store everything in a persistent benchmark directory

The goal is to make the next result reproducible without reconstructing the process from terminal history and optimism.

The next unnecessarily elaborate test

The next benchmark should include:

Configuration	Reason
LM Studio with Q4_K_S	Original baseline
Unsloth with Q4_K_S and speculation	Same-model optimized result
Unsloth with Q4_K_S without speculation	Backend control
Unsloth with UD-Q4_K_XL and speculation	Fastest current configuration
Unsloth with UD-Q4_K_XL without speculation	Separate the quant from speculation

Each configuration should run at least 20 measured requests.

I should also report:

• p50 latency
• p90 latency
• p95 latency
• Minimum throughput
• Maximum throughput
• Standard deviation
• Host CPU utilisation
• Guest CPU utilisation
• System temperature
• GPU utilisation
• Draft acceptance
• Memory consumption
• A small quality evaluation

I also need a bare-metal comparison.

That would help answer a question the current experiment cannot:

How much performance am I trading for the isolation and convenience of the VM?

This will turn a quick local experiment into a proper benchmarking project.

That is generally how these things happen.

First, I want to know whether one server feels faster.

Three hours later, I am planning latency percentiles, host-versus-guest monitoring, and debating whether my test prompt represents production traffic.

Current status

The UM890 Pro still has:

• A Ryzen 9 8945HS
• 64 GB of physical RAM
• No discrete GPU
• Several other jobs
• One VM with 8 CPU cores and 36 GB of RAM running local inference

LM Studio reached 13.01 tok/s.

Unsloth reached 22.24 tok/s with the same model.

The specialized dynamic Q4 artifact reached a 43.06 tok/s median.

Sometimes.

The specialized model is currently the most promising configuration I have tested on this machine.

It is also the least consistent.

The benchmark process is not yet stable enough for broad performance claims.

The next steps are:

• Repeat the runs across separate clean sessions
• Increase the number of measured requests
• Add host and guest monitoring
• Run a bare-metal comparison
• Test concurrent requests
• Compare output quality
• Package the procedure into one reproducible script

Until then, I can say that my integrated GPU generated more than 50 tokens per second at least twice.

Inside a virtual machine.

I simply cannot promise it will do it again.