Opened 5 weeks ago

Last modified 5 weeks ago

#85 assigned Service Improvement

Add error recovery to NightlySnapshots.

Reported by: D Delmar Davis Owned by: D Delmar Davis
Priority: Priority Milestone: Make Shit Happen / Own Your Shit.
Component: Development Keywords: Python lxd backups
Cc: Joe Dumoulin

Description

Because LXD has a mind of its own (grumble grumble) sometimes it puts zfs in a funny state.
The way the scripts are currently written any failure results in the remaining containers not being backed up.

A litte
try:

exception:

Is in order here.

Change History (2)

comment:1 Changed 5 weeks ago by D Delmar Davis

LXDAPIException: Create instance snapshot (mount source): Failed to run: zfs mount infra/containers/naomi: cannot mount 'infra/containers/naomi': filesystem already mounted

It is completely unintuitive that the way you resolve this issue is to mount the filesystem and take a snapshot.

root@kb2018:/etc/ansible# zfs mount infra/containers/naomi
root@kb2018:/etc/ansible# lxc snapshot naomi

comment:2 Changed 5 weeks ago by D Delmar Davis

Fixed the zfs/lxd container side.
When we have a successful nightly backup I will pull the code changes from the repo.

Note: See TracTickets for help on using tickets.