4. FAQ

4.1 Q: How does the system know about the blades on the network?

A: After insert initialization the EtherDrive blade emits a broadcast frame indicating its presence. In addition to this mechanism, the system will send out a query frame once a minute to discover any new blades.

4.2 Q: How do I see what blades the system knows about?

A: The file /dev/etherd/stat contains a list of devices the system considers valid. It also displays the status of the device (up or down). For example:

$ cat /dev/etherd/stat
/dev/etherd/e0.0       up
/dev/etherd/e0.1       up
/dev/etherd/e0.2       up
$

4.3 Q: How does the system know an EtherDrive blade has failed?

A: When a blade encounters a hard drive failure it will indicate so in the response to the failed request. The system will fail the device upon reception of the response. In addition, if an EtherDrive blade has not responded to a prior request within a default timeout (currently one minute) the system will fail the device.

4.4 Q: How do I take a blade out of the failed state?

A: Simply remove and reinsert the blade.

4.5 Q: How can I use LVM on my EtherDrive blade?

A: You can configure LVM2 to use EtherDrive block devices by editing lvm.conf, or you can create md devices from your aoe devices and tell LVM to use the md devices.

You can add this line to the "devices" block of your lvm.conf to configure LVM2 to use AoE block devices.

types = [ "aoe", 16 ]

4.6 Q: When I "modprobe aoe", it takes a long time. The systemseems to hang. What could be the problem?

A: If the new "hotplug" service is running on your system, it can slow things down and cause problems when the aoe module loads. We are working on compatibility with the new hotplug service. For now, it is probably best to disable it on your system. Usually the right commands look like this:

chkconfig hotplug off
/etc/init.d/hotplug stop

4.7 Q: Can I allow multiple Linux hosts to use a filesystem that is on my EtherDrive blades?

A: Yes, but you're now taking advantage of the flexibility of EtherDrive storage, using it like a SAN. Your software must be "cluster aware", like GFS. Otherwise, each host will assume it is the sole user of the filesystem and data corruption will result.

4.8 Q: How can I make a RAID of more than 27 EtherDrive blades?

A: For Linux Software RAID, the kernel limits the number of disks in one RAID to 27. However, you can easily overcome this limitation by creating another level of RAID.

For example, to create a RAID 0 of thirty disks on EtherDrive blades, you may create three ten-disk RAIDs (md1, md2, and md3) and then stripe across them (md0 is a stripe over md1, md2, and md3).

Here is an example raidtools configuration file that implements the above scenario for shelves 5, 6, and 7: multi-level RAID 0 configuration file. Non-trivial raidtab configuration files are easier to generate from a script than to create by hand.

The EtherDrive storage gives you a lot of freedom, so be creative.

4.9 Q: Why can't I make a large block device?

A: The 2.4 kernel is limited to 2 terabyte block devices, but some (carelessly-written) software may only be able to use 1 terabyte block devices. The version 2.6 Linux kernel supports very large block devices, but your 2.6 kernel must be configured with large block device support.

That means on 2.4-based systems, it is safest to limit the size of, e.g., Software RAID devices and LVM devices to 1 TB.