For some reason I thought this would be a good idea, something fun to play with anyway. But of course I wanted to make it complicated and make the drives network acessable.
Anyway, the virtual tape library itself was a breeze to setup, after mucking about for a month or so reading all the scsi specs and drawing lots of scribled diagrams on how I was going to make it work, I discovered that yet again I was trying to re-invent something that had already been done.
MHVTL – Virtual tape library with automatic volume changer
Software : http://sites.google.com/site/linuxvtl2/
User Forum: http://mhvtl.nimsa.us/
Implementation took about 5 minutes (excluding the 4hrs taken to backup the server before I started mucking about).
Before
[root@falcon ~]# lsscsi -g
[0:0:0:0] disk Maxtor Basics Desktop 0122 /dev/sdb /dev/sg3
[1:0:0:0] disk ATA ST3802110A 3.AA /dev/sda /dev/sg0
[2:0:0:0] cd/dvd SAMSUNG CD-ROM SC-152L C100 /dev/sr0 /dev/sg1
[2:0:1:0] cd/dvd CyberDrv CW068D CD-R/RW 120D /dev/sr1 /dev/sg2
And five minutes later, the after
[root@falcon log]# lsscsi -g
[0:0:0:0] disk Maxtor Basics Desktop 0122 /dev/sdb /dev/sg3
[1:0:0:0] disk ATA ST3802110A 3.AA /dev/sda /dev/sg0
[2:0:0:0] cd/dvd SAMSUNG CD-ROM SC-152L C100 /dev/sr0 /dev/sg1
[2:0:1:0] cd/dvd CyberDrv CW068D CD-R/RW 120D /dev/sr1 /dev/sg2
[3:0:0:0] mediumx SPECTRA PYTHON 550V /dev/sch0 /dev/sg12
[3:0:1:0] tape IBM ULT3580-TD4 550V /dev/st0 /dev/sg4
[3:0:2:0] tape IBM ULT3580-TD4 550V /dev/st1 /dev/sg5
[3:0:3:0] tape IBM ULT3580-TD4 550V /dev/st2 /dev/sg6
[3:0:4:0] tape IBM ULT3580-TD4 550V /dev/st3 /dev/sg7
[3:1:0:0] mediumx SPECTRA PYTHON 550V /dev/sch1 /dev/sg13
[3:1:1:0] tape IBM ULT3580-TD4 550V /dev/st4 /dev/sg8
[3:1:2:0] tape IBM ULT3580-TD4 550V /dev/st5 /dev/sg9
[3:1:3:0] tape IBM ULT3580-TD4 550V /dev/st6 /dev/sg10
[3:1:4:0] tape IBM ULT3580-TD4 550V /dev/st7 /dev/sg11
Then I hit major problems with making it all network acessable via iSCSI.
Lots of users have had success using SCST ( http://scst.sourceforge.net/ ) to provide iSCSI target support for the tape robot and virtual tape support, most network backup solutions seem to work with SCST, the MHVTL forum link above has lots of examples in the SCST discussion section showing screen shots backup tools detecting and using the drives (windows tools, TSM etc).
Of course SCST will not compile under the Fedora core 13 OS. And SCST is not available in the fedora repositories either.
But Fedora repositories did have scsi-target-utils. So I installed those and had a play. No tape support !. The fedora solution supports only block devices (disks), and while it allows VMs to boot from networked iSCSI devices (and may play with later) that is not what I was looking for at this time. If you are interested in booting your servers from iSCSI devices using standard fedora software try http://berrange.com/posts/2010/05/05/provisioning-kvm-virtual-machines-on-iscsi-the-hard-way-part-1-of-2/ which has a good short howto. I guess for RedHat in their virtual machine push disk support is the priority, but no good to me.
For what I was trying to do it looks like only SCST provides full iSCSI features including virtual tape library support for Linux. While it has patches for vanilla kernels (although the latest version I pulled down had patches for a kernel version that didn’t exist on kernel.org) I tried to compile it with the makefile flag CONFIG_SCST_STRICT_SERIALIZING set which should have negated any need for kernel patching…
…the Fedora kernel header libraries (or gcc libraries) are missing a function call required, frustratingly only the one, the only error in the make is
scst-1.0.1.1/src/scst_priv.h:348: error: implicit declaration of function ‘scsi_execute_async’
But thats a show stopper for me.
Although not strictly, I am going to install Ubuntu on my spare server to see if that works, if not I’ll try Mint which by all reports is a nice desktop to end up with at least.
Getting slightly frustrated with Fedora spins, I mean why are they writing their own iSCSI support when SCST is already way ahead (ahh, SCST do seem to have a commercial arm and have it installed/supported in commercial accounts, so redhat can’t use a working solution but must write their own in order to charge support for it. Maybe thats not true, but there is a working open source solution and redhat have decided to reinvent the wheel, they only have a few spokes in place at the moment, while scst goes rolling on).
I’ll install Ubuntu server on my spare box next weekend, and see if I can get SCST working.
Update: 2010/07/20. I may not have got scsi interfaces to it working, or automatic tape loading (that I assume is supposed to happen), but by manually loading virtual tapes on devices and doing a backup to the virtual device, then a restore from the virtual device, it all works !. Ok, there is a kernel oops for every backup and restore (backup pherhaps because it prompts for permission to overwrite the character device before using it which is not correct behaviour), but the restored files are (as far as diff can tell) identical to the backed up files. So all I have to do is figure out the remote scsi bits. Well actually also how to get it to autoload for any request as without that it’s pretty pointless.
