一、开启IO-MMU

1. 修改grub文件

bash

nano /etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT项末尾添加以下内容:

text

quiet intel_iommu=on pci=assign-busses pcie_acs_override=downstream video=efifb:off,vesafb:off,simplefb:off iommu=pt

参数说明:

  • intel_iommu=on:开启Intel平台的IOMMU(AMD平台使用amd_iommu=on

  • iommu=pt:passthrough模式,可提高性能

  • pcie_acs_override=downstream:将同一Group中的设备分开直通

  • video=efifb:off:禁用efifb驱动,防止BAR 3内存保留错误

  • pci=assign-bussesiommu=pt:开启SR-IOV所需参数

2. 更新grub配置:

bash

update-grub

二、显卡直通

1. 加载直通内核模块

bash

nano /etc/modules

添加:

text

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

2. 屏蔽显卡驱动

Intel核显:

bash

echo "blacklist snd_hda_intel" >> /etc/modprobe.d/pve-blacklist.conf
echo "blacklist snd_hda_codec_hdmi" >> /etc/modprobe.d/pve-blacklist.conf
echo "blacklist i915" >> /etc/modprobe.d/pve-blacklist.conf

AMD显卡:

bash

echo "blacklist radeon" >> /etc/modprobe.d/pve-blacklist.conf
echo "blacklist amdgpu" >> /etc/modprobe.d/pve-blacklist.conf

NVIDIA显卡:

bash

echo "blacklist nouveau" >> /etc/modprobe.d/pve-blacklist.conf
echo "blacklist nvidia" >> /etc/modprobe.d/pve-blacklist.conf 
echo "blacklist nvidiafb" >> /etc/modprobe.d/pve-blacklist.conf

N卡额外配置:

bash

echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf

3. 更新内核并重启

bash

update-initramfs -k all -u
reboot

4. 检查模块加载

bash

lsmod | grep vfio

text

root@home:~# lsmod | grep vfio
vfio_pci               57344  0
vfio_virqfd            16384  1 vfio_pci
irqbypass              16384  2 vfio_pci,kvm
vfio_iommu_type1       36864  0
vfio                   36864  2 vfio_iommu_type1,vfio_pci

5. 绑定显卡到vfio-pci

查看PCI设备:

bash

lspci

text

root@home:~# lspci 
00:00.0 Host bridge: Intel Corporation Device 9b33 (rev 01)
00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 01)
00:02.0 VGA compatible controller: Intel Corporation CometLake-S GT2 [UHD Graphics 630] (rev 05)  #核显 00:02.0
00:14.0 USB controller: Intel Corporation Comet Lake PCH-V USB Controller
00:16.0 Communication controller: Intel Corporation Device a3ba
00:17.0 SATA controller: Intel Corporation 400 Series Chipset Family SATA AHCI Controller
00:1b.0 PCI bridge: Intel Corporation Device a3e9 (rev f0)
00:1c.0 PCI bridge: Intel Corporation Device a392 (rev f0)
00:1c.3 PCI bridge: Intel Corporation Device a393 (rev f0)
00:1c.4 PCI bridge: Intel Corporation Device a394 (rev f0)
00:1d.0 PCI bridge: Intel Corporation Device a398 (rev f0)
00:1f.0 ISA bridge: Intel Corporation Device a3c8
00:1f.2 Memory controller: Intel Corporation Memory controller
00:1f.3 Audio device: Intel Corporation Device a3f0                                               #板载声卡 00:1f.3
00:1f.4 SMBus: Intel Corporation Comet Lake PCH-V SMBus Host Controller
01:00.0 VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 730] (rev a1)            #独显 01:00.0
01:00.1 Audio device: NVIDIA Corporation GK208 HDMI/DP Audio Controller (rev a1)                  #独显声卡 01:00.1
04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
05:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
05:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
05:10.0 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)
05:10.2 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)
05:10.4 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)
05:10.6 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)
05:11.0 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)
05:11.2 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)
05:11.4 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)
05:11.6 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)
05:12.0 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)
05:12.2 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)
05:12.4 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)
05:12.6 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)
06:00.0 Non-Volatile memory controller: KIOXIA Corporation Device 0008 (rev 01)                    #nvme硬盘 06:00.0

bash

lspci -n

text

root@home:~# lspci -n 
00:00.0 0600: 8086:9b33 (rev 01)
00:01.0 0604: 8086:1901 (rev 01)
00:02.0 0300: 8086:9bc5 (rev 05)    #核显 8086:9bc5
00:14.0 0c03: 8086:a3af
00:16.0 0780: 8086:a3ba
00:17.0 0106: 8086:a382
00:1b.0 0604: 8086:a3e9 (rev f0)
00:1c.0 0604: 8086:a392 (rev f0)
00:1c.3 0604: 8086:a393 (rev f0)
00:1c.4 0604: 8086:a394 (rev f0)
00:1d.0 0604: 8086:a398 (rev f0)
00:1f.0 0601: 8086:a3c8
00:1f.2 0580: 8086:a3a1
00:1f.3 0403: 8086:a3f0             #板载声卡 8086:a3f0
00:1f.4 0c05: 8086:a3a3
01:00.0 0300: 10de:1f82 (rev a1)    #独显 10de:1f82
01:00.1 0403: 10de:10fa (rev a1)    #独显声卡 10de:10fa
04:00.0 0200: 10ec:8125 (rev 05)
05:00.0 0200: 8086:1528 (rev 01)
05:00.1 0200: 8086:1528 (rev 01)
05:10.0 0200: 8086:1515 (rev 01)
05:10.2 0200: 8086:1515 (rev 01)
05:10.4 0200: 8086:1515 (rev 01)
05:10.6 0200: 8086:1515 (rev 01)
05:11.0 0200: 8086:1515 (rev 01)
05:11.2 0200: 8086:1515 (rev 01)
05:11.4 0200: 8086:1515 (rev 01)
05:11.6 0200: 8086:1515 (rev 01)
05:12.0 0200: 8086:1515 (rev 01)
05:12.2 0200: 8086:1515 (rev 01)
05:12.4 0200: 8086:1515 (rev 01)
05:12.6 0200: 8086:1515 (rev 01)
06:00.0 0108: 1e0f:0008 (rev 01)    #nvme硬盘 1e0f:0008

绑定设备:

bash

echo "options vfio-pci ids=1e0f:0008,8086:9bc5,8086:a3f0,10de:1f82,10de:10fa disable_vga=1" > /etc/modprobe.d/vfio.conf

如果无法输出到外接显示器,取消disable_vga=1参数:

bash

echo "options vfio-pci ids=1e0f:0008,8086:9bc5,8086:a3f0,10de:1f82,10de:10fa" > /etc/modprobe.d/vfio.conf

6. 验证配置

bash

cat /etc/modprobe.d/pve-blacklist.conf
cat /etc/modprobe.d/vfio.conf

刷新配置:

bash

update-grub
update-initramfs -k all -u
reboot

检查配置是否成功:

bash

lspci -nnk

text

root@home:~# lspci -nnk
00:00.0 Host bridge [0600]: Intel Corporation Device [8086:9b33] (rev 01)
              DeviceName: Onboard - Other
            Subsystem: ASUSTeK Computer Inc. Device [1043:8694]
              Kernel driver in use: skl_uncore
00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 01)
            Kernel driver in use: pcieport
00:02.0 VGA compatible controller [0300]: Intel Corporation CometLake-S GT2 [UHD Graphics 630] [8086:9bc5] (rev 05)
            DeviceName: Onboard - Video
            Subsystem: ASUSTeK Computer Inc. Device [1043:8694]
            Kernel driver in use: vfio-pci
            Kernel modules: i915

7. 设置虚拟机配置

编辑虚拟机配置文件:

bash

nano /etc/pve/qemu-server/100.conf

添加:

text

args: -cpu host,kvm=off,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=NV43FIX

注意: 安装完独显驱动后需手动禁用默认集显,否则不会默认使用独显。

三、硬盘直通

1. 查看硬盘ID

bash

ls -l /dev/disk/by-id

2. 设置硬盘直通

bash

qm set 100 -scsi0 /dev/disk/by-id/ata-ST2000DM001-1ER164_Z4ZBXJS2

注意: Windows 虚拟机:数据盘首选 SCSI (VirtIO SCSI),群晖 (DSM) 虚拟机:首选 SATA

1. M.2 NVMe 固态硬盘:首选「PCIe 硬件直通」

既然是 NVMe 协议的盘,直接用 qm set 映射成 SCSI/SATA 是非常“浪费”性能的。

  • 推荐方案: PCIe 屏蔽并直通给虚拟机

  • 原因:

    • 极致性能: 虚拟机直接控制 NVMe 控制器,延迟最低,读写性能几乎与物理机无异。

    • 完整特性: 虚拟机里的 Windows 能直接读取该盘的 S.M.A.R.T 信息,并由 Windows 原生管理 NVMe 的电源管理和 TRIM。

  • 操作简述:

    1. 在 PVE 的硬件设置里,添加“PCI 设备”。

    2. 找到你的 NVMe 控制器(注意不是单个分区,是整个控制器)。

    3. 勾选“PCI-Express”、“所有功能 (All Functions)”和“ROM 栏”。

    4. 注意:如果你的 PVE 系统也装在这块盘上,则不能这样做。

2. 机械 HDD 硬盘:推荐「qm set 映射 + SCSI (VirtIO)」

机械硬盘的瓶颈在于物理磁头寻道,接口带宽(SATA3)远超机械盘上限,所以没必要纠结硬件直通。

  • 推荐方案: 使用 qm set <VMID> -scsiX /dev/disk/by-id/ata-xxxx 方式挂载。

  • 原因:

    • 管理方便: PVE 宿主机依然能监控硬盘状态,且不需要复杂的 IOMMU 分组设置。

    • VirtIO SCSI 的优势: 相比虚拟 SATA,SCSI 模式配合 VirtIO SCSI 驱动在高负载(多任务读写)下 CPU 占用更低,且支持磁盘丢弃(Discard)等优化。

  • 配置建议:

    • 在虚拟机硬件里,SCSI 控制器一定要选 VirtIO SCSI single

    • 在硬盘选项里,勾选 IO thread(IO 线程),这能显著提升机械盘在高负载时的系统响应速度。

3. 挂载已有分区的物理磁盘

bash

qm set 100 --sata1 /dev/sdb

4. 显示所有磁盘信息

bash

fdisk -ll

5. 虚拟磁盘直通挂载

bash

qm importdisk 101 /var/lib/vz/template/iso/****.img local-lvm

text

Successfully imported disk as 'unused0:local-lvm:vm-101-disk-0'

6. 删除直通设备

bash

qm set 101 -delete sata0

四、USB和声卡直通

直通声卡和 USB 控制器比直通 NVMe 盘要稍微复杂一点,因为这两个设备对中断信号(Interrupts)和电源管理非常敏感。

以下是针对这两类设备的勾选建议:

1. USB 控制器直通:最推荐的方式

直通整个 USB 控制器(PCIe 层级)比单纯直通某个 USB 端口要稳定得多,可以实现真正的“即插即用”。

  • PCI-Express:必勾。 USB 3.0/3.1 控制器依赖 PCIe 消息信号中断(MSI),不勾可能会导致设备识别延迟或传输速度极慢。

  • 所有功能 (All Functions):建议勾选。 许多主板的 USB 控制器带有一个“子控制器”(例如 XHCI),勾选此项能确保整个控制器及其子功能一起移交给虚拟机。

  • ROM 栏 (ROM Bar):建议勾选。 保证虚拟机在启动阶段就能初始化 USB 设备(如键盘鼠标)。

💡 小贴士: 直通 USB 控制器前,请务必确认你的 PVE 宿主机键鼠 没有插在这个控制器对应的接口上,否则直通开启的一瞬间,你将失去对 PVE 物理机的本地控制。


2. 声卡 (Audio) 直通:最挑剔的设备

板载声卡直通是 PVE 中比较容易出问题的部分,建议如下:

  • PCI-Express:必勾。 解决声音“爆音”、“拉长”或“卡顿”的关键。

  • 所有功能 (All Functions):强烈建议勾选。 * 原因: 现代主板的音频设备通常是“复合设备”。例如,Intel 的高清音频控制器可能包含音频输出和某些电源管理功能。如果只直通一半,Windows 驱动可能会显示“设备无法启动 (Code 10)”。

  • ROM 栏 (ROM Bar):默认勾选即可。


汇总表:直通参数建议

选项

USB 控制器

板载/独立声卡

理由

PCI-Express

勾选

勾选

启用原生 PCIe 特性,减少延迟和卡顿

All Functions

建议

必须

确保音频复合设备/多接口 USB 完整移交

ROM Bar

勾选

勾选

确保设备初始化正常

Primary GPU

不勾

不勾

仅直通显卡作为主显示输出时才勾选

⚠️ 避坑指南:如果直通后出现问题

  1. 声音爆音/断断续续:

    这是由于虚拟机时钟同步问题。你可能需要在虚拟机的 .conf 文件(/etc/pve/qemu-server/VMID.conf)中添加以下行来优化 CPU 响应:

    cpu: host,hidden=1,flags=+hypervisor

  2. IOMMU 分组报错:

    声卡和 USB 控制器经常被主板厂商划分在同一个 IOMMU Group 里。如果你只想直通其中一个却报错,可能需要开启 ACS Patch(在 GRUB 中添加 pcie_acs_override=downstream,multifunction),但这属于进阶操作,有一定的系统风险。

五、网卡SR-IOV

1. 查看网卡参数

查看所有网卡:

bash

lspci -nn|grep Eth

text

03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 06)
0b:00.0 Ethernet controller [0200]: Qualcomm Atheros Killer E220x Gigabit Ethernet Controller [1969:e091] (rev 10)
0c:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)

查看网卡信息:

bash

lshw -c network -businfo

text

Bus info          Device           Class          Description
=============================================================
pci@0000:03:00.0                   network        RTL8111/8168/8411 PCI Express Gig
pci@0000:0b:00.0  enp11s0          network        Killer E220x Gigabit Ethernet Con
pci@0000:0c:00.0  enp5s0f0f0       network        82599ES 10-Gigabit SFI/SFP+ Netwo
usb@2:6           enx000ec6711984  network        Ethernet interface
                  vmbr0            network        Ethernet interface
                  ztnfag5sip       network        Ethernet interface
                  tap888i0         network        Ethernet interface

查看SR-IOV开启状态:

bash

lspci -s 05:00.0 -vvv | grep Capabilities

text

pcilib: sysfs_read_vpd: read failed: Input/output error
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [70] MSI-X: Enable+ Count=64 Masked-
        Capabilities: [a0] Express (v2) Endpoint, MSI 00
        Capabilities: [e0] Vital Product Data
        Capabilities: [100 v1] Advanced Error Reporting
        Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-ba-bf-e6
        Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)

查看网口链路状态:

bash

ethtool enp5s0f0f0

text

Settings for enp5s0f0f0:
        Supported ports: [ FIBRE ]
        Supported link modes:   10000baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Auto-negotiation: off
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

查看网卡驱动版本:

bash

ethtool -i enp5s0f0

text

driver: ixgbe
version: 5.13.19-1-pve
firmware-version: 0x00012b2c
expansion-rom-version:
bus-info: 0000:0c:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

查看网卡支持的VF数量:

bash

cat /sys/bus/pci/devices/0000:0c:00.0/sriov_totalvfs

text

63

2. 创建开机脚本启用SR-IOV

bash

nano /etc/init.d/net-sriov

脚本内容:

bash

#!/bin/sh

# Copyright (C) 2011, 2012, 2016 Nicira, Inc.
# Licensed under the Apache License, Version 2.0

### BEGIN INIT INFO
# Provides:          openvswitch-switch
# Required-Start:    $network $named $remote_fs $syslog $openvswitch-switch
# Required-Stop:     $remote_fs $openvswitch-switch
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: SR-IOV initialization
# Description:       Initializing VM's network with sriov support
### END INIT INFO

start() {
    # 启用SR-IOV,将enp5s0f0虚拟12个网卡
    echo 12 > /sys/class/net/enp5s0f0/device/sriov_numvfs
    ip link set dev enp5s0f0 up
    
    # 设置VF的MAC地址
    ip link set dev enp5s0f0 vf 0 mac aa:bb:cc:dd:ee:f1
    ip link set dev enp5s0f0 vf 1 mac aa:bb:cc:dd:ee:f2
    ip link set dev enp5s0f0 vf 2 mac aa:bb:cc:dd:ee:f3
    ip link set dev enp5s0f0 vf 3 mac aa:bb:cc:dd:ee:f4
    ip link set dev enp5s0f0 vf 4 mac aa:bb:cc:dd:ee:f5
    ip link set dev enp5s0f0 vf 5 mac aa:bb:cc:dd:ee:f6
    ip link set dev enp5s0f0 vf 6 mac aa:bb:cc:dd:ee:f7
    ip link set dev enp5s0f0 vf 7 mac aa:bb:cc:dd:ee:f8
    ip link set dev enp5s0f0 vf 8 mac aa:bb:cc:dd:ee:f9
    ip link set dev enp5s0f0 vf 9 mac aa:bb:cc:dd:ee:a0
    ip link set dev enp5s0f0 vf 10 mac aa:bb:cc:dd:ee:a1
    ip link set dev enp5s0f0 vf 11 mac aa:bb:cc:dd:ee:a2
    
    # 初始化网络
    systemctl restart networking
}

stop() {
    [ "$READ_INTERFACES" != "no" ] && network_interfaces ifdown
    ovs_ctl stop
}

case $1 in
    start)
        start
        ;;
    stop|force-stop)
        stop
        ;;
    *)
        echo "Usage: $0 {start|stop}" >&2
        exit 1
        ;;
esac

exit 0

3. 启用SR-IOV脚本

bash

chmod +x /etc/init.d/net-sriov
systemctl enable net-sriov

4. 重启PVE

bash

reboot

重启后检查网络界面是否出现多个网卡,验证MAC地址设置:

bash

ip a

六、安装破解内核解决IOMMU分组问题

1. 下载破解内核

yfdoor/PVE-Kernel 下载破解内核。(如需新版本内核6.11.0-2可联系我)

2. 安装内核

进入内核文件目录:

bash

cd pve-kernel-5.13.18-1/
dpkg -i *.deb

text

root@pve2:~/PVE-Kernel/pve-kernel-5.13.18-1# dpkg -i *.deb
(Reading database ... 180164 files and directories currently installed.)
Preparing to unpack linux-tools-5.13_5.13.18-1_amd64.deb ...
Unpacking linux-tools-5.13 (5.13.18-1) over (5.13.14-1) ...
Preparing to unpack linux-tools-5.13-dbgsym_5.13.18-1_amd64.ddeb ...
Unpacking linux-tools-5.13-dbgsym (5.13.18-1) over (5.13.14-1) ...
Preparing to unpack pve-headers-5.13.18-1-pve_5.13.18-1_amd64.deb ...
Unpacking pve-headers-5.13.18-1-pve (5.13.18-1) over (5.13.18-1) ...
Selecting previously unselected package pve-kernel-5.13.18-1-pve.
Preparing to unpack pve-kernel-5.13.18-1-pve_5.13.18-1_amd64.deb ...
Unpacking pve-kernel-5.13.18-1-pve (5.13.18-1) ...
Preparing to unpack pve-kernel-libc-dev_5.13.18-1_amd64.deb ...
Unpacking pve-kernel-libc-dev (5.13.18-1) over (5.13.14-1) ...
Setting up linux-tools-5.13 (5.13.18-1) ...
Setting up linux-tools-5.13-dbgsym (5.13.18-1) ...
Setting up pve-headers-5.13.18-1-pve (5.13.18-1) ...
Setting up pve-kernel-5.13.18-1-pve (5.13.18-1) ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 5.13.18-1-pve /boot/vmlinuz-5.13.18-1-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 5.13.18-1-pve /boot/vmlinuz-5.13.18-1-pve
update-initramfs: Generating /boot/initrd.img-5.13.18-1-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
No /etc/kernel/proxmox-boot-uuids found, skipping ESP sync.
run-parts: executing /etc/kernel/postinst.d/proxmox-auto-removal 5.13.18-1-pve /boot/vmlinuz-5.13.18-1-pve
run-parts: executing /etc/kernel/postinst.d/zz-proxmox-boot 5.13.18-1-pve /boot/vmlinuz-5.13.18-1-pve
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
No /etc/kernel/proxmox-boot-uuids found, skipping ESP sync.
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 5.13.18-1-pve /boot/vmlinuz-5.13.18-1-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.13.18-1-pve
Found initrd image: /boot/initrd.img-5.13.18-1-pve
Found linux image: /boot/vmlinuz-5.11.22-5-pve
Found initrd image: /boot/initrd.img-5.11.22-5-pve
Found linux image: /boot/vmlinuz-5.4.124-1-pve
Found initrd image: /boot/initrd.img-5.4.124-1-pve
Found linux image: /boot/vmlinuz-5.4.106-1-pve
Found initrd image: /boot/initrd.img-5.4.106-1-pve
Found memtest86+ image: /boot/memtest86+.bin
Found memtest86+ multiboot image: /boot/memtest86+_multiboot.bin
Adding boot menu entry for EFI firmware configuration
done
Setting up pve-kernel-libc-dev (5.13.18-1) ...
Processing triggers for man-db (2.9.4-2) ...

3. 配置内核启动参数

编辑grub文件:

bash

nano /etc/default/grub

设置内核启动顺序:

bash

GRUB_DEFAULT="Advanced options for Proxmox VE GNU/Linux>Proxmox VE GNU/Linux, with Linux 5.13.18-1-pve"

更新配置:

bash

update-grub
update-initramfs -k all -u
reboot

4. 验证IOMMU分组

bash

for d in /sys/kernel/iommu_groups/*/devices/*; do
  n=${d#*/iommu_groups/*}; n=${n%%/*};
  printf 'IOMMU Group %s ' "$n";
  lspci -nns "${d##*/}";
done

text

IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:9b33] (rev 01)
IOMMU Group 10 00:1d.0 PCI bridge [0604]: Intel Corporation Device [8086:a398] (rev f0)
IOMMU Group 11 00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:a3c8]
IOMMU Group 11 00:1f.2 Memory controller [0580]: Intel Corporation Memory controller [8086:a3a1]
IOMMU Group 11 00:1f.3 Audio device [0403]: Intel Corporation Device [8086:a3f0]
IOMMU Group 11 00:1f.4 SMBus [0c05]: Intel Corporation Comet Lake PCH-V SMBus Host Controller [8086:a3a3]
IOMMU Group 12 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK208B [GeForce GT 730] [10de:1287] (rev a1)
IOMMU Group 12 01:00.1 Audio device [0403]: NVIDIA Corporation GK208 HDMI/DP Audio Controller [10de:0e0f] (rev a1)
IOMMU Group 13 04:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU Group 14 05:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 [8086:1528] (rev 01)
IOMMU Group 15 05:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 [8086:1528] (rev 01)
IOMMU Group 16 06:00.0 Non-Volatile memory controller [0108]: KIOXIA Corporation Device [1e0f:0008] (rev 01)
IOMMU Group 17 05:10.0 Ethernet controller [0200]: Intel Corporation X540 Ethernet Controller Virtual Function [8086:1515] (rev 01)
IOMMU Group 18 05:10.2 Ethernet controller [0200]: Intel Corporation X540 Ethernet Controller Virtual Function [8086:1515] (rev 01)
IOMMU Group 19 05:10.4 Ethernet controller [0200]: Intel Corporation X540 Ethernet Controller Virtual Function [8086:1515] (rev 01)
IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 01)
IOMMU Group 20 05:10.6 Ethernet controller [0200]: Intel Corporation X540 Ethernet Controller Virtual Function [8086:1515] (rev 01)
IOMMU Group 21 05:11.0 Ethernet controller [0200]: Intel Corporation X540 Ethernet Controller Virtual Function [8086:1515] (rev 01)
IOMMU Group 22 05:11.2 Ethernet controller [0200]: Intel Corporation X540 Ethernet Controller Virtual Function [8086:1515] (rev 01)
IOMMU Group 23 05:11.4 Ethernet controller [0200]: Intel Corporation X540 Ethernet Controller Virtual Function [8086:1515] (rev 01)
IOMMU Group 24 05:11.6 Ethernet controller [0200]: Intel Corporation X540 Ethernet Controller Virtual Function [8086:1515] (rev 01)
IOMMU Group 25 05:12.0 Ethernet controller [0200]: Intel Corporation X540 Ethernet Controller Virtual Function [8086:1515] (rev 01)
IOMMU Group 26 05:12.2 Ethernet controller [0200]: Intel Corporation X540 Ethernet Controller Virtual Function [8086:1515] (rev 01)
IOMMU Group 27 05:12.4 Ethernet controller [0200]: Intel Corporation X540 Ethernet Controller Virtual Function [8086:1515] (rev 01)
IOMMU Group 28 05:12.6 Ethernet controller [0200]: Intel Corporation X540 Ethernet Controller Virtual Function [8086:1515] (rev 01)
IOMMU Group 2 00:02.0 VGA compatible controller [0300]: Intel Corporation CometLake-S GT2 [UHD Graphics 630] [8086:9bc5] (rev 05)
IOMMU Group 3 00:14.0 USB controller [0c03]: Intel Corporation Comet Lake PCH-V USB Controller [8086:a3af]
IOMMU Group 4 00:16.0 Communication controller [0780]: Intel Corporation Device [8086:a3ba]
IOMMU Group 5 00:17.0 SATA controller [0106]: Intel Corporation 400 Series Chipset Family SATA AHCI Controller [8086:a382]
IOMMU Group 6 00:1b.0 PCI bridge [0604]: Intel Corporation Device [8086:a3e9] (rev f0)
IOMMU Group 7 00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:a392] (rev f0)
IOMMU Group 8 00:1c.3 PCI bridge [0604]: Intel Corporation Device [8086:a393] (rev f0)
IOMMU Group 9 00:1c.4 PCI bridge [0604]: Intel Corporation Device [8086:a394] (rev f0)

七、最后记录一下我自己的配置

硬件概览

  • CPU: Intel Core i9-10900T (ES) - 10核20线程

  • 内存: 64 GB

  • 虚拟化平台: Proxmox VE 9.x

虚拟机最终配置详情

1. Windows 11 虚拟机 (主力桌面,带硬件直通)

  • CPU: 12 vCPU

    • 限制: 8

    • 权重: 2000 (最高优先级)

    • 类型: host

    • NUMA: 未启用

  • 内存: 20480 MB (20 GB)

  • Ballooning 设备: 已禁用 ⚠️ (因直通独立显卡、声卡、USB控制器与硬盘,启用存在驱动冲突风险)

  • 允许 KSM: 否

2. 黑群晖 (DSM) 虚拟机 (NAS服务,带硬件直通)

  • CPU: 8 vCPU

    • 限制: 6

    • 权重: 1000 (高优先级)

    • 类型: host

  • 内存: 6144 MB (6 GB)

  • Ballooning 设备: 已禁用 ⚠️ (因直通核显与硬盘)

  • 允许 KSM: 是 (可合并与其它Linux系统的相同内存页)

3. 其他服务虚拟机 (JumpServer, HAOS等)

  • CPU: 按需分配 (通常2-4 vCPU)

    • 限制: 设置为等于或略低于vCPU数 (如 2, 1)

    • 权重: 100 (默认值,低优先级)

    • 类型: host

  • 内存: 按需分配,并启用 Ballooning 设备 设置动态范围 (如 最小2G / 最大4G)。

  • 允许 KSM: 是

4. LXC 容器 (多种轻量服务)

  • CPU: 通常 1 vCPU

    • 限制: 0.51

    • 权重: 100

  • 内存: 通常 1024 MB (1 GB)

  • Swap (缓存): 按需分配(防止过度使用宿主机交换空间)

核心优化策略与说明

  1. 内存管理原则

    • 直通设备,禁用Ballooning:任何直通了显卡、USB控制器等硬件的虚拟机,必须关闭内存气球驱动,这是保障稳定性的铁律。

    • 分配依据:内存分配值应基于虚拟机的 常驻工作集大小 并保留合理余量,而非盲目设定最大值。通过 qm monitor <VMID> 命令中的 free_mem 数据可精确判断。

  2. CPU资源管控

    • “限制” (硬顶):定义了虚拟机可消耗物理CPU资源的绝对上限。为核心服务(如Win11)设置足够高的限制(如8),以保证其性能基线。

    • “权重” (软优先级):定义了在物理CPU资源争抢时各虚拟机的调度优先级。为主力机设定极高权重(如2000),确保其在系统繁忙时优先获得算力。

    • 组合效果:此组合确保了系统空闲时资源可充分利用,满载时资源分配可控且优先保障关键服务。

  3. 其他关键设置

    • CPU类型:设置为 host,以获得最佳性能和兼容性。

    • NUMA:在消费级平台或单CPU插槽的服务器上,通常无需启用。

    • KSM:对多个相似操作系统(如Linux容器)开启,可节省内存。

配置后验证方法

  1. 内存健康度:

    bash

    free -h

    关注 available 内存是否充足,Swap 使用量是否稳定且无增长。

  2. CPU调度状态:

    bash

    htop

    观察物理核心负载是否均衡,是否存在因争抢导致的长期满负载核心。

  3. 虚拟机内部状态:

    • 在 Windows 11 中,可使用任务管理器观察内存和CPU使用是否正常。

    • 在 PVE 宿主机上,使用 qm status <VMID>pct status <CTID> 查看运行状态。

此配置已稳定运行,宿主机内存压力解除,整体响应流畅,为主力服务提供了确定性的资源保障,并为未来的负载留出了弹性空间。


注意: 本文所有操作均在Proxmox VE环境下测试通过,具体操作时请根据自身硬件配置进行相应调整。