The Coraid Linux NAS HOWTO

Ed L. Cashin

2009-31-12

Table of Contents

Initial Configuration
Setting the Root User Password
Setting the Timezone
Limiting Login Access
Configure Ethernet MTU
Configure IP Networking
Setting the Hostname
Setting the Syslogging Receiver IP
Outgoing Mail
Email Alerts for SR Events
User and Group Information
Setting Up ATA over Ethernet
Setting Up Linux Software RAID
Using LVM to Manage Storage
Growing a Logical Volume
LVM Jargon
Creating a Filesystem
Growing a Filesystem
Repairing a Filesystem
Using NFS to Export Filesystems
Samba and CIFS for FS Export
Using Filesystems Quotas
Using Netconsole to See Kernel Messages
Sending Console Messages from Every CLN
Sending Console Messages outside the LAN
Using APT to Manage the CLN Software
Searching for Packages
Merging Configuration Files
Extra Virtual Memory for APT
Sweeping Upgrades
Getting Secure SSH Host Keys
Staying Informed
Mailing List
Changelog

Initial Configuration

Before using the Coraid Linux NAS (CLN), there is some site-specific information that needs to be configured. You can quickly store this configuration information on the CLN by using the commands and text editing steps described below.

For text editing, the CLN provides a traditional UNIX text editor, vi, as well as a "user friendly" editor called pico.

It's prudent to make a backup of an important file before editing it in place. You can use the convenient bkp script the CLN provides for this purpose.

makki:~# bkp /etc/hostname
bkp "/etc/hostname" --> "/etc/hostname.20051108"
makki:~# vi /etc/hostname

If you're really being careful you can examine your changes using the diff command. The old lines have a "-" at the left, and the new lines have a "+" at the left. Unchanged lines are shown with a space at the left in order to provide context.

makki:~# diff -u /etc/hosts.20051108 /etc/hosts
--- /etc/hosts.20051108 2005-11-08 10:31:46.000000000 -0500
+++ /etc/hosts  2005-11-08 10:32:04.000000000 -0500
@@ -1,5 +1,5 @@
 127.0.0.1      localhost.localdomain   localhost
-205.185.197.212        makki.coraid.com        makki
+192.168.1.20   foo.example.com foo
 # The following lines are desirable for IPv6 capable hosts
 ::1     ip6-localhost ip6-loopback
makki:~#

Setting the Root User Password

Every CLN comes with the root user account's password set to the word, "changeme". Unless only legitimate system administrators have physical or network access to the CLN, it's important to give the root user account a password right away.

Login to the CLN as the root user using the default password, "changeme", and then run the passwd command to change the root user's password to one you have chosen.

You type your new password and then type the same new password again. It is not displayed to the screen when you are typing, so you are typing blind when entering the new password.

makki:~# passwd
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
makki:~#

Setting the Timezone

The CLN uses NTP to set the system time when it boots. By setting the timezone for your location, your time will display correctly, and software will work more smoothly.

To find out the name for your timezone, you can use the interactive tzconfig tool. When prompted, just type in the appropriate answers for your location and hit enter.

Limiting Login Access

Before networking is configured, you can limit the IP addresses that are allowed to connect to the CLN by ssh. The only way to login to the CLN over the network is via the secure shell, ssh.

By editing the /etc/hosts.allow file, you can selectively provide access. The /etc/hosts.allow lines below only allow ssh connections from 205.185.197.207.

sshd: 205.185.197.207: allow
sshd: ALL: deny

The hosts.allow syntax is flexible enough to accomodate more complex access specifications. Run man hosts.allow to find out more.

Configure Ethernet MTU

The CLN will attempt to use a "jumbo" MTU (Maximum Transmission Unit) on all of its ethernet interfaces. If you are using a network switch that does not support jumbo frames (an MTU above 1500), you can override this behavior by editing the mtu.conf file. The simple configuration file syntax is explained in the comments at the top of the file.

bkp /etc/aoe/mtu.conf
vi /etc/aoe/mtu.conf

It is helpful to configure eth2 and eth3, the onboard ports, so that they do not use an MTU above 4200. Above this size the onboard ports may drop some packets, decreasing performance.

eth2 4200
eth3 4200

If you do not have that file, you are probably using an older version of the coraid-init package, and your CLN will not attempt to use jumbo frames. You can upgrade using the commands below, after you have completed the rest of the initial configuration steps listed in the following sections.

Even if your network switches support jumbo frames, they might need to be configured before this feature is available.

Configure IP Networking

The "front side" network of the CLN will be connected to your LAN, through which all the storage clients will contact the CLN. The "back" network will be used for ATA over Ethernet (AoE). The CLN is like a storage doorway, with NFS clients on the front side of the CLN and AoE storage devices behind the door.

For networking on the front side, you need an IP address that the clients will use to reach the CLN.

The CLN has onboard network interfaces that are free for administrative uses. When you view the back of the CLN, the PCI-X extension card appearing on the right contains eth0 on the right, and eth1 on the left. Figure 1 shows this view of the CLN's network ports.

CLN Network Ports

The port for the front network is eth0, and AoE storage will be accessed through eth1.

The examples in the following sections use this configuration.

(An alternative configuration would be to use eth0 and eth1 for the back network, with eth2 serving the front. This alternative provides higher AoE throughput and somewhat lower TCP/IP throughput, since the onboard ports cannot handle the 9000-MTU frames that the extension card ports can use. The important thing is not to mix ports of dissimilar performance characteristics.)

Setting the IP Address

To set the IP address, you use a text editor on two files. One is /etc/network/interfaces and the other is /etc/hosts. Replace the IP 205.185.197.212 with the IP address that you want your clients to use for accessing your NAS box.

You should not use 205.185.197.x addresses for your front side network. These are Coraid IP addresses, and you will need to use them on the back network if you are going to forward syslog messages from any SATA+RAID units. Syslog messages will be discussed below.

Also, edit /etc/hosts, again replacing 205.185.197.212 with your IP.

Setting the Network Information

Edit /etc/network/interfaces to reflect your IP network.

  • Replace 205.185.197.0 with the address of your network.
  • Replace 205.185.197.254 with the gateway you use to reach networks outside your LAN.
  • Replace 205.185.197.2 with the IP address of your primary name server.
  • Replace coraid.com with your local domain name.

Then restart networking with the command below.

/etc/init.d/networking restart

Edit /etc/resolv.conf, supplying your name server's address and the default domain.

  • Replace 205.185.197.2 with the IP address of your primary name server.
  • Replace coraid.com with your local domain name.

Nameless Hosts

For internal networks, it is sometimes the case that the name server specified in /etc/resolv.conf and /etc/network/interfaces doesn't know any names for certain hosts that will be connecting to the CLN.

Those hosts may experience delays when connecting to the CLN's services. You can put names in the /etc/hosts file for each such "nameless host" so that the CLN does not look up the name.

You can even generate names with a simple shell loop.

makki:~# for n in `seq 101 105`; do echo 192.168.2.$n h$n.coraid.com h$n; done
192.168.2.101 h101.coraid.com h101
192.168.2.102 h102.coraid.com h102
192.168.2.103 h103.coraid.com h103
192.168.2.104 h104.coraid.com h104
192.168.2.105 h105.coraid.com h105
makki:~# for n in `seq 101 105`; do echo 192.168.2.$n h$n.coraid.com h$n; done >> /etc/hosts

Setting the Hostname

Because the CLN is a server, we store the hostname on the CLN itself instead of looking it up via DNS.

  • Edit /etc/hostname, replacing "makki" with the hostname you have chosen for this CLN unit.
  • Edit /etc/hosts and /etc/mailname, replacing "makki" with your chosen hostname and "coraid.com" with your domain name.
  • Run the command hostname makki, using your own host name instead of "makki".

The hostname and domainname, joined by a period, form the "fully qualified domain name" (FQDN).

Setting the Syslogging Receiver IP

Note: If you decide to use syslog-ng, as described below in the section, Email Alerts for SR Events, you will not need to configure /etc/syslog.conf as described in this section. It is, however, instructive to read this section in any case.

The CLN uses flash storage locally, so to minimize writes to the flash medium, we can send its system log information to a remote host. The syslog daemon on the remote host should be run in a mode such that it will accept the syslog messages coming from the CLN.

Configuring the CLN to send its syslog information to a remote host is easy. In the echo command below, replace "kokone.coraid.com" with the host that will receive the syslog messages.

makki:~# bkp /etc/syslog.conf
bkp "/etc/syslog.conf" --> "/etc/syslog.conf.20051108"
makki:~# echo '*.* @kokone.coraid.com' > /etc/syslog.conf
makki:~# /etc/init.d/sysklogd restart
Restarting system log daemon: syslogd.
makki:~#

You can test this setup using logger.

makki:~# logger "testing"

Lines like this should show up on the remote host running syslog -r or the equivalent.

Nov  8 16:26:51 makki.coraid.com syslogd 1.4.1#17: restart.
Nov  8 16:26:57 makki.coraid.com ecashin: testing

(It says "ecashin" because during testing I logged into the CLN as that user before using the su command to become root.)

Forwarding SR Messages via Syslog

You can skip this section for now if you're just getting started.

You can configure the CLN to forward messages from one or more SATA+RAID (SR) units.

The SR appliances do not, in general, perform IP networking, but they do generate syslog messages in the form of UDP packets. These UDP packets have a source IP of 205.185.197.30 by default. Even if all the SR units you have use this default, you can still tell which SR unit on a given AoE network generated a message, because the shelf address is contained in the syslog message.

If your CLN is forwarding its own syslog messages to a remote host, it can also send messages from SR units as well. Here are the changes to make.

First, tell the syslog daemon on the CLN to listen to the network for messages from other hosts, and to forward those messages.

You can change the SYSLOGD options in /etc/init.d/sysklogd from an empty string to this:

SYSLOGD="-h -r"

Remember to restart the syslog daemon after editing the startup script.

/etc/init.d/sysklogd restart

Next, assign an IP to the CLN's network interface on the back network by editing /etc/network/interfaces. The purpose of this IP is only to help the kernel receive syslog UDP packets from the SR units, so it should be an IP on the same network as the source IP for the SR syslog messages. An example would be to make the CLN's eth1 be 205.185.197.1, so that it can receive the syslog messages with the default source IP 205.185.197.30.

# the back network interface
auto eth1
iface eth1 inet static
        address 205.185.197.1
        netmask 255.255.255.0
        network 205.185.197.0
        broadcast 205.185.197.255

In this example you are using 205.185.197.1 (or another IP in that network) as your own CLN's eth1 IP, so that it can receive packets with a source IP of 205.185.197.30.

Outgoing Mail

The CLN has only a stripped-down mail transport agent (MTA), enough to get mail off the CLN and onto a mail server. The primary reason for performing this easy step now is that the package management system, APT, sometimes sends email about installed packages.

To configure outgoing mail, edit the /etc/ssmtp/ssmtp.conf file, making the following changes:

  • Replace makki.coraid.com with the FQDN (hostname + dot + dommainname) of your CLN.
  • In mailhub=kokone, replace kokone with the hostname of your mail server.

Email Alerts for SR Events

The CLN comes with a traditional syslog daemon (discussed in Setting the Syslogging Receiver IP), but if you would like to receive email alerts based on the syslog messages that the CLN receives from your SR units, you can install and configure a more advanced syslog daemon. Here we discuss the use of syslog-ng.

It is safe to skip this section if you are getting started with your CLN, or if you have no interest in receiving emails containing messages on the SR.

First, ensure that you have a recent enough coraid-scripts package. It should be at least version 1.4. If it isn't recent enough, you can upgrade the package as shown below.

makki:~# dpkg -l | grep coraid-scripts
ii  coraid-scripts         1.2                   Helpful scripts for the CLN
makki:~# apt-get update       # repeat if necessary
makki:~# apt-get install coraid-scripts

Next replace the existing syslog daemon with syslog-ng.

makki:~# apt-get install syslog-ng
Reading package lists... Done
Building dependency tree... Done
The following packages will be REMOVED:
  klogd sysklogd
The following NEW packages will be installed:
  syslog-ng
0 upgraded, 1 newly installed, 2 to remove and 17 not upgraded.
Need to get 199kB of archives.
After unpacking 225kB of additional disk space will be used.
Do you want to continue [Y/n]?

Hit enter to accept the default response and initiate the replacement.

Once syslog-ng is installed, it needs to be configured. You can backup the distributed configuration file first.

makki:~# bkp /etc/syslog-ng/syslog-ng.conf

Below is an example syslog-ng.conf that will send email to "bogus@example.com". To receive email alerts, you should change that address to that of the intended recipient.

You will also need to replace "kokone.coraid.com" with the name or IP address of the host where you want the CLN's syslog messages to wind up.

# example /etc/syslog-ng/syslog-ng.conf for email alerts
options {
      # five seconds
      flush_timeout(5000);
      use_dns(no);
      log_msg_size(1024);
      stats_freq(0);
};
source local {
        internal();
        unix-stream("/dev/log");
        file("/proc/kmsg" log_prefix("kernel: "));
};
source remote {       udp(); };
filter sr_critical {
      match("shelf_") and (
              match("fail") or
              match("offline") or
              match("abort") or
              match("cknowledge by running online")
      );
};
filter emerg { level(emerg); };
destination loghost {
      udp("kokone.coraid.com" port (514));
};
destination users { usertty("*"); };
destination email_alert {
      program("/opt/bin/sendalert bogus@example.com");
};
log {
      source(local);
      source(remote);
      destination(loghost);
};
log {
      source(remote);
      filter(sr_critical);
      destination(email_alert);
};
log { source(local); filter(emerg); destination(users); };

You can modify the configuration to suit your needs by consulting the syslog-ng documentation and examples. You can see the contents of the syslog-ng package (or any package you like) by using the dpkg command.

makki:~# dpkg -L syslog-ng | less

User and Group Information

No special configuration is needed on the CLN for users and groups.

With NFS, all access is provided based on the numerical identifier for the user, called a "UID", and a numerical group identifier ("GID"). The users and groups need only exist on the NFS clients. You don't need to create the users and groups on the CLN.

If you have multiple NFS clients sharing a filesystem, then the UIDs and GIDs for the users and groups on all the NFS clients should match. There are many options for managing users and groups on NFS clients (NIS+, LDAP, etc.), but user and group information does not need to be stored on the CLN.


www.coraid.com