Opened 3 years ago

Closed 3 years ago

#26 closed Task (Fixed)

Discovery: Improve zfs performance.

Reported by: D Delmar Davis Owned by: Joe Dumoulin
Priority: Important Milestone: Server Modernization Phase II
Component: Infrastructure Keywords: zfs performance lxd ansible
Cc: Joe Dumoulin

Description (last modified by D Delmar Davis)

Making ansible scripts for backups is demonstrating how poorly fast machines with lots of resources can perform.

Lets see if we can tune zfs to make hot copies of containers between systems live up to its expectations.

Change History (5)

comment:1 Changed 3 years ago by D Delmar Davis

Owner: changed from D Delmar Davis to Joe Dumoulin

Joe,

Do you have a 1G switch we can put on the admin lan?

root@kb2018:~# mii-tool enp4s0f1
enp4s0f1: negotiated 100baseTx-FD flow-control, link ok

I just realized that the current router is throttling the network between the two servers to 100baseT.

Don.
PS I will add links to a couple of sub $20 options if you don't have one on hand.

Last edited 3 years ago by D Delmar Davis (previous) (diff)

comment:2 Changed 3 years ago by D Delmar Davis

Description: modified (diff)

comment:3 Changed 3 years ago by D Delmar Davis

This actually looks perfect for the admin lan.

https://www.frys.com/product/6466002 $16 plus shipping....

comment:4 Changed 3 years ago by D Delmar Davis

Added cache disk to devel pool. Seems to be slightly faster.

comment:5 Changed 3 years ago by D Delmar Davis

Resolution: Fixed
Status: assignedclosed
  • replaced router ports with the switch listed above.
  • changed the MTU on the admin lan to 9000

Changing the MTU

  • modify netplan config and apply it (should but may not really work)
    # not really cloud-init.yaml. Create static addresses because like DUH.
    # disable cloud-init's network configuration capabilities, by writing a file
    # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
    # network: {config: disabled}
    network:
        version: 2
        renderer: networkd
        ethernets:
            enp3s0f0:
                addresses: []
                dhcp4: no
            enp3s0f1:
                addresses: []
                dhcp4: no
            enp4s0f0:
                addresses: []
                dhcp4: no
                optional: true
            enp4s0f1:
                dhcp4: no
                addresses: [192.168.31.159/24]
                gateway4: 192.168.31.1
                mtu: 9000
                nameservers:
                    search: [suspectdevices.com fromhell.com vpn]
                    addresses: [198.202.31.141]
        bridges:
           br0:
              dhcp4: no
              dhcp6: no
              interfaces:
                  - enp3s0f0
           br1:
              dhcp4: no
              dhcp6: no
              interfaces:
                  - enp3s0f1
    
  • for paranoias sake add the mtu config to rc.local
    root@kb2018:/etc/ansible# cat /etc/rc.local 
    #!/bin/bash
    ip link set enp4s0f1 mtu 9000
    
  • reboot and repeat for Bernie.

Resulting improvement

Network improvement shaves an hour and 8 minutes off of the total backup.
The majority of the remaining time is creating and copying the archives to usb connected external disk.

Backup time with small packets: 3:40:29
Backup time with jumbo packets: 2:32:46

Note: See TracTickets for help on using tickets.