UNIX Unleashed, System Administrator's Edition - 27 - Administering FTP Services by Salim Douba File Transfer Protocol (FTP, RFC 959) service is probably among the earliest (if not the earliest) data exchange and transfer services that were first introduced on the Internet. When first introduced, as early as 1971, FTP assumed a primitive nature (RFC 114) that was used to send test traffic on the ARPANET. Quickly thereafter, FTP grew into one of the two principal network applications (the other was TELNET) that run on a TCP/IP stack. Among other reasons, FTP's popularity stems from its independence from any specific platform. Implementations of both the FTP server and client can be found on many platforms including UNIX, DOS, Windows NT, and Novell's NetWare network operating system, to name a few. As a service available on UNIX platforms, FTP presents system administrators with additional concerns. Among the principal concerns are setting up the FTP service, its availability, and security. Security becomes a doubly burdening concern should the network on which the FTP server is brought up be connected to the Internet. This chapter addresses these issues to varying degrees of detail, thus empowering the administrator with both the knowledge base and the skills required to properly provide FTP services, be it on a physically isolated local area network (LAN) or on an Internet-connected LAN. The skillful administration of FTP services, however, requires a reasonable foundation in some of the concepts governing the FTP protocol and operations. For this reason, this chapter is roughly divided into two components. The first four sections deal with the underlying concepts and operational issues of FTP, while the remainder of the chapter deals with practical issues and applications. In particular, the chapter covers the following: An overview of the FTP protocol How to set up an FTP Server How to set up an Anonymous FTP Server How to administer access privileges to FTP services How to implement third-party data transfer How to implement unattended FTP data transfers How to troubleshoot FTP-related problems As in other chapters, the coverage of the material included here takes into consideration the different variants of UNIX that the book covers. While the FTP governing concepts and mechanisms are, to a large extent, identical on all platforms (and not only UNIX), there are subtle differences in the way that some aspects of the service applications are implemented. In due course, those differences will be highlighted. Overview of FTP Protocol and Service The File Transfer Protocol is intended for the transfer of files between two hosts across the network. Furthermore, hosts engaging in an FTP transfer session need not be similar in nature. Both the hardware and operating systems running on the involved hosts can be anything but similar--a characteristic feature that made FTP server and client implementations widely available on platforms other than UNIX. Using a DOS FTP client, for example, a user can (subject to security restrictions) easily exchange files with a UNIX host running FTP service. The DOS user may also engage two other servers in the exchange of data without being concerned about the nature or identity of these machines. While one could be an IBM AIX UNIX host, the other could be a mainframe or a Windows NT server. This later ability to engage two servers in the exchange of data from a client running on a third machine is known as third-party (or proxy) transfer (more on this later in the chapter). Among the other features that FTP supports are its capability to handle both ASCII and EBCDIC (Extended Binary Coded Decimal Interchange Code) character sets. Equally important is FTP's support of ASCII and binary transfers. Unlike UUCP, which requires that any data other than ASCII, be uuencoded at the source and uudecoded at the destination (thus wasting both CPU processing power and communications bandwidth), FTP submits data on as is basis, never attempting to modifying or interpreting it--a big plus. FTP Connections FTP runs at the application level of the OSI (Open System Interconnection) model and depends on the Transmission Control Protocol (TCP, see Chapter 20, "Networking," for more on TCP) for the reliable delivery of data among hosts on the network. As such, FTP behaves as a strict application without being concerned with the architectural underpinnings of the supporting communications architecture. It relies on TCP for doing what it takes to ensure the establishment of the necessary connections and the error-free delivery of data. Whenever an FTP client (in UNIX, the client typically known as ftp) engages in an FTP session with a server process (controlled by what is commonly referred to as ftpd or the in.ftpd daemon in UNIX), TCP establishes two connections on behalf of both entities. These connections are known as the control connection and the data connection (see Figure 27.1). Figure 27.1. FTP service protocol establishes two connections, a data connection and a control connection The data connection is used exclusively for the transfer of user data, whereas the control connection is used exclusively for the exchange of internal FTP commands and responses governing the session. Control Connection The control connection is established by the client while invoking the ftp command by opening a TCP connection to the desired server. No user data is exchanged over the control connection. It is used exclusively for the exchange of internal FTP commands and responses that govern the exchange of user data over a separate data connection. Here is an example of invoking an FTP session with host tenor: # ftp tenor Assuming that host tenor is running the UNIX operating system, if configured properly to host FTP services, it responds by establishing the control connection using port 21 (default FTP control port as defined in the /etc/services file). Responses to FTP requests are handled by the ftpd daemon on the UNIX host. When the control connection is open, the user authentication process takes place before even establishing a data connection. The user is required to provide a valid user login name and password to the FTP server. After successful completion of the authentication process, both the server and the client engage in negotiating the parameters required to govern the exchange of data. For example, the data type is set during this process (for instance, ASCII versus EBCDIC), and transfer mode (for instance, STREAM). Data Connection Upon successful negotiation of the transfer parameters, both the client (ftp) and server (ftpd) processes proceed to establishing the data connection using port 20 (set in the /etc/services file). The data connection is full duplex, allowing simultaneous transfer of data in both directions. It is important to know that the data connection may not be maintained throughout the session. There are circumstances when the data connection is closed by the end of a file transfer, or upon detection of an irrecoverable error condition. As long as the control connection is maintained, the session is considered live. Both sides can negotiate and establish a new data connection whenever the need arises for the resumption of file transfers. More on Connections Two more things you ought to know before quitting the topic on connections: 1. While the default ports (also known as well know ports) for the control port and data port are 21 and 20, respectively, the FTP server (ftpd daemon) may be configured to listen on other ports for connection and for the exchange of data. 2. Unless overriden, the port assigned to the data connection is one less than the control port. This is demonstrated by the default ports assigned to them (21 is the control port, and 20 is the data port number). While ports 21 and 20 are the norm, you will be surprised to know that some implementations of FTP rely on the use of other ports. In cases like these you are required to fine tune any FTP-based applications that run on your UNIX host so they can interact successfully with computers hosting such implementations of FTP. NOTE: Using alternate ports, rather than the default (control port 21) can be achieved by simply editing the /etc/service file to replace the port number in the FTP entry to the desired port number. (See the "Troubleshooting FTP" section for more on this topic.) Reliability of FTP FTP relies on TCP for the delivery of information between hosts. As discussed in Chapter 20, TCP is a reliable, connection oriented, transport protocol. FTP's reliability stems from the reliability of TCP. When engaged in the transfer of data, FTP relies on TCP to handle the sequencing of exchanged data segments and recovery from erroneous events during the delivery process. Should the transfer process be inadvertently interrupted, however, it is FTP's responsibility to recover from the interruption and resume the transfer process without necessarily retransmitting the entire file. FTP achieves this by employing a checkpointing mechanism that breaks the file undergoing transmission into logical blocks of data that are individually acknowledged upon successful delivery. A block can be transmitted in one TCP segment or multiple TCP segments. While TCP concerns itself with the error-free delivery of individual segments, FTP concerns itself with the successful transmission of all segments comprising the individual block. Consequently, whenever the file transfer is restarted upon recovery from an abrupt interruption, FTP retransmits data starting from the beginning of the interrupted block only and until completion; thus achieving optimal performance and minimal impact on available CPU power and transmission bandwidth. Operational Characteristics of FTP FTP is a real-time process. This means that an FTP session is established as soon as the FTP application is invoked. If invoked in the foreground, a user or program must suspend other action and wait the completion of the session before proceeding. Because the speed of the transfer is affected by many factors, including the size of the file and the capacity of the available bandwidth, the time it takes for completion may vary considerably from one network environment to another, and during different times of the day. For this reason, FTP was developed to be a time-insensitive application. If you hate to wait prolonged periods of time until you regain control of your terminal, you might want to consider running FTP sessions in an unattended mode of operation. In this mode, ftp (the client) can be invoked in the background and made to process a session in accordance with an ftp script that you make available to it. See the "Unattended FTP Transfers" section later in this chapter. Another operational feature of FTP on UNIX platforms is the provision for scripting. Instead of interactively establishing a session, a user or system administrator can script the entire session and submit it to the ftp command upon invocation. This approach requires that the user know the exact details of the proceedings of the intended session. Later in the chapter (see the "Unattended FTP Transfers" section), both this feature and unattended session are explained and applied to solve a real-life situation. Scripting is not a requirement of the FTP standard; rather, it is a feature that some platform-specific implementations provide, while others ignore it completely. Sample FTP Session In this section, the proceedings of an FTP session are illustrated by invoking one. The purpose is to foster some of the concepts learned above and to prepare the ground for the material covered in the upcoming section. In its simplest form, ftp can be used to invoke a file transfer session with a target host: # ftp targethost targethost can be the IP address or the name of the remote host. As a secure service, ftp prompts you for both your login name and your password. Unless you are authenticated, ftp aborts requests to establish the file transfer session. Later in the chapter, FTP security access control and authentication are discussed in detail. In the following example, a user who is logged in to host jade uses ftp to establish a session with host tenor. # ftp tenor Connected to tenor. 220 tenor FTP server (UNIX(r) System V Release 4.0) ready. Name (tenor:root): sam 331 Password required for sam. Password: 230 User sam logged in. ftp> The prompt ftp>, which in this case is displayed after the user sam authenticated successfully, is FTP's and serves as an indication of readiness on FTP's part to execute any of the ftp user commands. Before going any further, it is worth noting that while the FTP session is being established with the remote system, processing of any of the commands the user might issue is done locally, not remotely as in the case of TELNET. Hence, the ftp prompt is, in fact, local to the machine from which the user invoked the session, and pertains to the local ftp command. Upon invocation of the ftp command, a so-called User-PI (user process interpreter) is created. The User-PI acts very much like a UNIX shell in that it accepts user issued ftp commands and submits them to the FTP protocol for subsequent handling. To find out what user commands ftp supports, enter help at the prompt as shown here: ftp> help Commands may be abbreviated. They are ! debug mget pwd status $ dir mkdir quit struct account disconnect mls quote system append form mode recv sunique ascii get modtime reget tenex bell glob mput rstatus trace binary hash newer rhelp type bye help nmap rename user case idle nlist reset umask cd image ntrans restart verbose cdup lcd open rmdir ? chmod ls prompt runique close macdef proxy send cr mdelete sendport site delete mdir put size ftp> Use get to transfer a file from the remote host to your station, and use put to send a file to the remote host. Use mget (multiple get) and mput (multiple put) for the transfer of more than one file at a time. The best place to go to for explanation of the above commands is the man pages of your UNIX operating system. However, it is worth noting that not all commands are implemented in all of the UNIX variants of FTP. Therefore, unless implemented on both the local and the remote host, a command is not valid for execution. Instead, an error response is generated notifying the user of failure, as the following example demonstrates: ftp> rstatus 502 STAT command not implemented. ftp> rstatus is typically used to query the FTP server about the status of its end of the connection. In the preceding example, the remote end (the FTP server) indicated that the STAT (an internal FTP protocol command) is not implemented. How to tell whether the message pertains to the server and not to client capabilities? Simple. Whenever the response is preceded with a three-digit code such as 502 in the example, it means that the command was recognized by the local host (how this code is interpreted is the subject of the next section). Consequently, the message pertains to an issue on the remote end (the server). If the command is not supported locally, a message similar to the following is issued: ftp> rstatus ?Invalid command ftp> Notice how no code precedes the message. Whenever a user ftp command is issued, the User-PI determines whether the command is supported by the local implementation of the protocol before submitting it to FTP (the protocol) for processing. If not supported, the command is rejected outrightly by the local process and is subsequently declined access to the protocol itself. There are two sets of FTP commands, referred to as user ftp commands, and internal ftp commands. User ftp commands are those that the user is capable of issuing at the ftp prompt. A list of supported user commands is available upon entering the help command at the prompt. Internal commands are, on the other hand, those that are peculiar to the protocol per se. These are commands that the FTP protocol, not the ftp command, supports. There is a relationship between the two command sets. As the example illustrates, the user command rstatus is mapped to the internal command STAT. Once successfully invoked, an ftp command may trigger a series of internal FTP commands and related responses. FTP Internal Commands and Responses The FTP protocol supports a wide range of commands designed to provide full protocol functionality and services from one end of the connection to the other. Table 27.1 contains a detailed listing of all commands that the RFC 959 standard recommends. I refer to the RFC 959 standard because some of those commands are not necessarily implemented on all UNIX platforms, and, consequently, their man pages might fail to reference all the commands cited below. RFC 959 specifies a minimum set of commands that require implementation for the ftp application to be branded compliant. Other than the specified minimum, vendors are left the freedom to adopt whatever other features are recommended by the RFC. Table 27.1. FTP internal commands. Command Description ABOR Abort previous command and any associated transfer of data. ACCT User account ID; specified as an argument to the command. ALLO Allocate storage for forthcoming file transfer. Size is specified as command argument. APPE Append incoming data to an existing file; specified as command argument. CDUP Change directory up one level (to parent directory). CWD Change working directory on remote server; path specified as command argument. DELE Delete file; filename specified as command argument. HELP Send information regarding status of server implementation. LIST Transfer directory listing. MKD Make directory; directory path is specified as command argument. MODE Set transfer mode. One of three: stream, block, or compressed. NLST Transfer directory listing. NOOP Do nothing. If server is alive, it responds with OK. PASS User password. PASV Requests server to listen on a data port other than the default one. PORT Causes the server to change to a different data port than the currently used one. PWD Print working directory. QUIT Terminate the connection. REIN Reinitialize connection while allowing transfers in progress to complete. REST Restart marker. Causes the transfer to resume after being disrupted from a specified data checkpoint in the file. RETR Retrieve file specified in argument field. RMD Remove directory specified in argument field. RNFR Rename From; specifies the old pathname of the file that is to be renamed. This command is normally followed by a rename to (RNTO) command specifying the new filename. RNTO Rename to command. SITE Sends site-specific parameters to remote end. Such parameters are normally essential to file transfer but not sufficiently universal to be included as commands in the protocol. SMNT Mount file system. STAT Return status of service. STOR Accept and store data. STOU Accept and store data under a unique name. STRU Specifies the structure of the file pending transmission. The specified structure can be File, Record, or Page. The default is File. SYST Return type of operating system. TYPE Specifies type of data pending transmission. One of three: ASCII, EBCDIC, or BINARY. USER User ID Most of the internal commands are made up of four ASCII character sequences, some of which requires the use of arguments. The choice of encoding the commands using ASCII characters has the advantage of allowing expert system administrators to actually telnet the server at the FTP control port and execute most of the internal commands themselves--a capability that might prove useful in a troubleshooting exercise. ASCII encoding of commands also conveniently enables users to observe the command flow and make sense of it without having to employ specialized protocol analysis tools--let alone the advanced level of expertise required to operate such tools. To turn on the capability of tracing the flow of internal FTP commands, the user must enter the debug command at the ftp prompt before attempting any operation. This is demonstrated in the following example: # ftp tenor Connected to tenor. 220 tenor FTP server (UNIX(r) System V Release 4.0) ready. Name (tenor:root): 331 Password required for root. Password: 230 User root logged in. ftp> debug Debugging on (debug=1). ftp> put testfile local: testfile remote: testfile ---> PORT 100,0,0,2,4,103 200 PORT command successful. ---> STOR testfile 150 ASCII data connection for testfile (100.0.0.2,1127). 226 Transfer complete. 13415 bytes sent in 0.57 seconds (23 Kbytes/s) ftp> quit ---> QUIT 221 Goodbye. # Notice how the internal commands are all represented using a sequence of ASCII uppercase characters. FTP Responses In addition to the internal commands, the FTP protocol supports a considerable range of responses. Responses are expressed using three-decimal digit notation. The place of each digit bears a certain significance to the interpretation of the response being examined. In the example, notice how every internal command is followed by a three-digit response. The responses to FTP commands are designed to ensure the synchronization of requests and actions in the process of file transfer. They also enable the user process (User-PI) to follow up on the status of the server. Every command is required to generate at least one response. In addition, some commands are required to occur in a certain sequence (for example, every RNFR should be followed immediately by an RNTO). A failure to produce an expected sequence suggests the existence of abnormalities either in the transmission process or in the software implementation. An FTP reply consists of a three-digit number. The three-digit number is also accompanied by some text that either explains the reply (226 Transfer complete) or provides information pertaining to the reply itself. The textual information may vary from one FTP implementation to another. The three-digit code is standardized and cannot be changed. Each of the three digits of the reply has a special significance. The first digit denotes whether the response conveys good, bad, or incomplete operational status. Table 27.2 summarizes the values that the most significant digit of the response can take and the significance of each value: Table 27.2. Values of most significant digit in FTP response code and their corresponding meanings. Value Description 1 Requested action has been initiated; expect a reply before sending another action. 2 The action has completed successfully. User can send a new (and unrelated) action. 3 The action has been accepted; the requested action is being suspended until further information is received from user. 4 The action was not accepted due to a transient error. The requested action may be resubmitted for completion. 5 The action was not accepted due to permanent error. User may not resubmit request the same action. The middle digit provides more details about the response in question. Table 27.3 provides information about the values that this digit can take and the associated meaning. Table 27.3. Values of middle digit in FTP response code and their corresponding meanings. Value Description 0 The reply refers to a syntax error. 1 Signify a reply to a request for information, such as status of connection. 2 The reply refers to data and control connection management function. 3 Reply pertains to login and authentication actions. 4 Not used. 5 Reply conveying the status of the server file system. The third digit provides even finer details about the function categories specified in the middle digit. Listing all applicable values is beyond the scope of this chapter. Interested readers may reference RFP 959 for further details. FTP Third-Party (Proxy) Transfers There are situations when a user wants to transfer files between two FTP servers, neither of which is the local host that is attended to by the user. Figure 27.2 illustrates the situation. As shown in the diagram, a user at host jade may want to transfer files from host tenor to host alto. Of course, one way to doing this is to transfer files from host tenor to host jade and from host jade to host alto--thus doubling the network traffic arising from the desired transfer as well as unduly engaging the resources (CPU and disk) of host jade, not to mention the degradation in response time. Figure 27.2. Third-party transfer using the proxy FTP command. FTP service protocol provides a transfer mechanism that alleviates these performance bottlenecks. This mechanism is called third-party transfer or proxy transfer. Using this mechanism, the user at host jade can initiate control connections to both servers: hosts tenor and alto. The connection invoked first is the primary control connection, and the second is the secondary control connection. As shown in Figure 27.2, because the user logged in to the FTP service on host tenor first, this connection is the primary one, and the second to host alto is the secondary connection. When both connections are established, the user can set up a data connection between the two servers. In this fashion, control information is communicated over the control connections while data is transferred between the servers over the data connection without being routed through host jade. Setting up a third-party transfer session is a matter of invoking ftp's proxy command at the ftp prompt after a primary FTP connection is established. In Figure 27.2, the user logs in to host tenor first, thus establishing a primary connection with that host. To invoke a secondary connection with host alto, the user should invoke the proxy open command at the ftp prompt (to obtain a list of all the commands that can be invoked on a secondary channel, enter proxy ?). The following output of a sample FTP session illustrates the proceedings of a third-party transfer that was invoked from host jade in Figure 52.1: # ftp tenor Connected to tenor. 220 tenor FTP server (UNIX(r) System V Release 4.0) ready. Name (tenor:root): 331 Password required for root. Password: 230 User root logged in. ftp> proxy open alto Connected to alto 220 alto FTP server (Version 5.60 #1) ready. 331 Password required for root. 230 User root logged in. Remote system type is UNIX. Using binary mode to transfer files. ftp> pwd tenor:257 "/" is current directory. ftp> get testfile local: testfile remote: testfile tenor:200 PORT command successful. tenor:150 ASCII data connection for testfile (100.0.0.2,1156) (12870 bytes). tenor:226 ASCII Transfer complete. 13415 bytes received in 0.75 seconds (17 Kbytes/s) ftp> quit tenor:221 Goodbye. alto Goodbye. # It's important to remember that some of the user ftp commands behave differently under a proxy transfer. For example, get and mget commands cause files to be transferred from the primary to the secondary, while put and mput cause file transfers from the secondary to the primary. So be careful with the use of these commands; there is the likelihood that if the same filename exists on both servers, one file might erroneously overwrite the other. To avoid this possibility, you should enable saving files under unique filenames using the sunique and runique ftp commands. For example, to enable storing files under unique names on the secondary channel you should issue the command as shown here: ftp> proxy sunique Store unique on. ftp> Third-party transfers can be conveniently used to carry out unattended, scheduled file transfers among different hosts from a central location. Using ftp scripts, the system administrator can describe the actions that the ftp client ought to take, as well as instructing it to log the output pertaining to all invoked sessions in a central log file. The script can then be scheduled (using cron) to run unattended at a time when the traffic is low on the network, typically, at night. CAUTION: Not all server implementations support third-party transfers. Check with the vendor to verify that FTP internal command PASV is supported; if it isn't, a secondary connection cannot be used to initiate third-party transfers. A possible workaround is conditional on having PASV support on at least one of the involved servers, in which case, you invoke the primary connection with the server lacking the PASV support, and the secondary with the server that has it. Administering FTP The three main areas of FTP administration involve the following tasks: Setting up the service Setting up user environment Setting up secure anonymous FTP service Setting up the FTP Service As explained in Chapter 20, aside from the basic TCP/IP services, such as routing and domain name services, all other services, including FTP, are started on an as-needed basis. On UNIX platforms, the FTP service is implemented in the ftpd daemon. This deamon, among others (such as, telnetd for TELNET services), is started by the superserver daemon inetd. Not until inetd receives a request for the services of ftpd is the latter started and handed that request. To ensure that inetd is indeed listening on behalf ftpd, you should ensure that the /etc/inetd.conf superserver configuration file has an entry for the ftpd daemon and that that entry is enabled. To enable an entry in the inetd.conf file, you simply uncomment it by removing the # character in front of it. Following is the listing of /etc/inetd.conf as it exists on one of the UNIX hosts: # @(#)inetd.conf 5.2 Lachman System V STREAMS TCP source # # System V STREAMS TCP - Release 4.0 # # Copyright 1990 Interactive Systems Corporation,(ISC) # All Rights Reserved. # # Copyright 1987, 1988, 1989 Lachman Associates, Incorporated (LAI) # All Rights Reserved. # # SCCS IDENTIFICATION ftp stream tcp nowait NOLUID /etc/ftpd ftpd telnet stream tcp nowait NOLUID /etc/telnetd telnetd shell stream tcp nowait NOLUID /etc/rshd rshd login stream tcp nowait NOLUID /etc/rlogind rlogind exec stream tcp nowait NOLUID /etc/rexecd rexecd finger stream tcp nowait nouser /etc/fingerd fingerd #uucp stream tcp nowait NOLUID /etc/uucpd uucpd # Enabling this allows public read files to be accessed via TFTP. #tftp dgram udp wait nouser /etc/tftpd tftpd # This is the more secure method, since only files from /tftpboot can # be accessed via TFTP. This must be root in order to do the chroot # to /tftpboot. /tftpboot must be created by hand. #tftp dgram udp wait root /etc/tftpd tftpd -s /tftpboot comsat dgram udp wait root /etc/comsat comsat ntalk dgram udp wait root /etc/talkd talkd #bootps dgram udp wait root /etc/bootpd bootpd echo stream tcp nowait root internal discard stream tcp nowait root internal chargen stream tcp nowait root internal daytime stream tcp nowait root internal time stream tcp nowait root internal echo dgram udp wait root internal discard dgram udp wait root internal chargen dgram udp wait root internal daytime dgram udp wait root internal time dgram udp wait root internal smtp stream tcp nowait mmdf /usr/mmdf/chans/smtpd smtpd /usr/mmdf/chans/smtpsrvr smtp According to this listing, the entry pertaining to the ftpd daemon is enabled. Consequently, inetd is expected to be listening on behalf of ftpd on the FTP control port. The inetd daemon knows which control port to listen on by referencing the /etc/services file, which contains simple entries depicting the name of each service and the TCP port it's required to be using. This file is referenced only when inetd is started. Consequently, making changes to the /etc/services file (or to the /etc/inetd.conf) won't take effect unless the inetd is restarted by rebooting the host or by sending inetd a SIGHUP signal. CAUTION: It is vitally important that /etc/services is not deleted or changed in any way by inexperienced users. Tampering with this file can lead to trouble trying to connect to network services. As a system administrator, you are well advised to make a backup copy of it--just in case something happens to it. Administering FTP Users By default, anyone who has a valid user account on a UNIX host on which the ftpd daemon is enabled is capable of starting an FTP session with the server and transferring files from and to the server (subject to file system permissions). A system administrator can prevent a user from logging in to the FTP service by adding the user's login name in the /etc/ftpusers file. This is a simple text file containing a list of the account names (one line per login name) that are not allowed FTP access to the server. For security reasons, it is suggested that the file contain login names root, and uucp. If the /etc/ftpusers file does not exist, users with valid login accounts will be able to log in to the system using FTP. Even if a user's login name is not in the /etc/ftpusers file, and the user authenticates, she may still be prevented from accessing the FTP service. This condition occurs if the login attempt is being made from a shell that is not included in the /etc/shells file. Access to FTP service is granted, subject to successful authentication to users running any of the shells listed in the /etc/shells file. Here is an example of the contents of this file: # @(#)shells 4.2 Lachman System V STREAMS TCP source # SCCS IDENTIFICATION /bin/csh /bin/sh /bin/ksh /usr/local/bin/ksh /usr/bin/scosh Check your shells! Some UNIX systems (for example Solaris 2.5), by default, have no /etc/shells file, a condition that leaves a door open to hackers to come through to tamper with the system. The nonexistence of the /etc/shells file enables users access from any shell. Always make sure that the file exists. If it doesn't, create it, and include in it only the shells that you want to allow users to use when accessing your system. Autologin and .netrc Inexperienced users can make it necessary for you (the system administrator) to automate the login process to the FTP server. Users who need to frequently access the service may find the routine of entering their username and password annoying. Fortunately, ftp provides an autologin functionality that automates the entire process. This is achieved by creating a file called .netrc in the user's home directory. The .netrc file contain both login and ftp initialization parameters to be used by an autologin procedure. Before you create the file, however, you should first test the desired setting on the command line. Multiple entries can be included in the .netrc file, one per remote host, specifying the autologin authentication information and the pertinent initialization parameters. Following is a partial depiction of the syntax of the .netrc file entries: machine hostname | default login login_id password password macdef macfilename In this syntax machine hostname Specifies the name or the IP address of the remote host you want to establish an FTP session with. default Matches any machine name not included in any of the .netrc entries. Only one default entry can be included, and it must show as the last entry in the file. Login login_id Specifies the user's login ID on the remote host password password Specifies the user's password. Note that, for security reasons, if this token is present, ftp requires that the file be protected from being read by anyone other than the user. The permissions on .netrc should, consequently, be set to 600. Failure to meet this condition aborts any attempt to autologin to the remote host. Macdef macfilename Specifies the name of a user-defined macro, defining all the initialization parameters. Refer to the man pages for more information. In the following two examples, the first entry autologs user adel to host tenor and establishes an ftp session on her behalf with that host. The second entry allows adel to log in to other servers with the specified default account specifications: machine tenor login adel password letmein default login adel pasword jamsession To establish a session with tenor, adel must enter this: # ftp tenor Connected to tenor. 220 tenor FTP server (UNIX(r) System V Release 4.0) ready. 331 Password required for adel. 230 User adel logged in. ftp> Notice how the user is not required to enter the login ID or the password. To establish a session with any other host, adel must have an account maintained on all the servers she wants to log in to with same login ID (adel) and password (jamsession). Anonymous FTP With the phenomenal growth in the popularity of the Internet, many organizations began offering a wide range of Internet services, including the distribution of software, information, technical support, and other services. Anonymous FTP access was among the earliest methods used to offer such services on the Internet. Even when users resort to web browsers to download files from the Internet, the browser invokes an FTP connection on behalf of the user, and logs him in to the server in anonymous capacity before the download process is actually started. Anonymous FTP access provides users who do not have an account with the FTP server the ability to establish an FTP session with that server. Users use anonymous for user name, and their E-mail address for the password. A user who logs in anonymously is misled to believe that he is at the root of the file system. This restriction provides an extra level of security, protecting the rest of the file system from use by anonymous users. Most UNIX variants included in the ftpd daemon have a feature that forces the daemon to change the root of the file system to the home directory of a special account. The account name is ftp. Assuming that the home directory of the ftp account is /home/ftp, the ftpd daemon uses chroot to change the root of the file system to /home/ftp for that user. Setting Up Anonymous FTP Service Setting up an FTP server for anonymous access involves creating the ftp user account and creating a mini-file system under the home directory of ftp. The directory must contain the necessary file system resources that render anonymous access functional. In general, following are the required steps for providing anonymous FTP access: Create the user ftp in /etc/passwd. Make the user belong to a new group (group ftp). The user's home directory becomes the root of the file system that anonymous users see. Once the ftp account is created, anonymous FTP access is enabled on the server. Following is a depiction of the ftp account entry in the /etc/passwd file: ftp:x:5000:5000:Anonymous FTP:/home/ftp:/noshell Notice that the specified shell is noshell. Since noshell does not exist, security of the ftp account is being reinforced by preventing the use of this account for login and subsequent tampering with the file system. There should be no need for anybody including root to try accessing the host using the ftp account. Create home directory and assign its ownership to root with the same group as that to which the ftp account is made to belong (to GID 5000 in the example entry). The directory must not be writable by anyone. For security, assign the permissions 555 to the ~ftp directory. Create the ~ftp/bin directory. Assign ownership to root, and permissions should be set to disallow anyone from writing to the directory. In this directory, the /usr/bin/ls command must be present to enable users to list its contents. The permissions on /usr/bin/ls should be set to 111. Create ~ftp/etc. This directory must contain a stripped version of the /etc/passwd file, /etc/group, and /etc/netconfig. These files must be present for the ls command to work satisfactorily. Set permissions on the ~ftp directory to 444. Create the ~ftp/dev directory and assign its ownership to root. This directory must contain both special files dev/tcp and /dev/tcp. Both files must be created using the mknod command. The major and minor numbers corresponding to each file can be obtained by using the "ls -lL" command. Failure to include any of ~ftp/dev/tcp, ~ftp/dev/zero or ~ftp/etc/netconfig may cause the error message "Can't create data socket (0.0.0.0,20): Bad file number" upon attempting to transfer files from the server. Also the permissions on ~ftp/dev/tcp and ~ftp/dev/zero should be set to 666. Otherwise, passive ftp requests might fail, resulting with permission denied error messages. Create the ~ftp/usr/lib directory and assign its ownership to root. This directory should contain a copy of all of the following shared libraries from /usr/lib: ld.so* libc.so* libdl.so* libintl.so* libw.so* libnsl.so* libsocket.so* nss_nis.so* nss_nisplus.so* nss_dns.so* nss_files.so* straddr.so* Create the directory ~ftp/pub, assign its ownership to ftp and set its permissions to 555. This is the directory where public access is allowed via the anonymous login. This is where you want to put files for public distribution The above steps are generic and apply to most variants of FTP implementation on UNIX platforms. Solaris Anonymous FTP Setup The ftp man pages included in Solaris 2.x include a shell script that auto-creates the entire setup of the anonymous FTP server. TIP: If you are interested in a highly configurable and secure FTP daemon, you might want to give the Wuarchive FTP 2.4 daemon a try. You can download a copy of it from ftp.uu.net server. It is in the /networking/ftp/wuarchive-ftpd directory. The directory includes implementation for both BSD and SVR4 variants of UNIX. The archive file contains documentation that explains all of the daemon's features, how to set it up and configure it properly. How Secure is the Server After completing the setup and verifying the success of logging in to it anonymously, it is prudent to verify that you did not leave behind any security loopholes that may lead to unwelcome intrusion (and, consequently, the potential loss of your job). Following are the minimum recommended checks you should make: Log in to the server as anonymous user and verify that the account cannot be used to create files or directories in any of the directories you created as part of the setup. Particular attention should be given to the home directory of the ftp account. If created, the .rhosts file grants instant access to hackers from across the Internet. Check to see whether the ftp command SITE CHMOD is implemented. If it is, it can be at the heart of serious security breaches. Similarly, check for the SITE EXEC command. The way to do this check is by telnetting to the host at port 21(this is FTP's default port). If you can, and are able to execute the SITE EXEC command or SITE CHMOD command, you'd be better off replacing the ftpd daemon with a more secure one (see the preceding Tip). Make sure that the home directory is not owned by ftp. Remember, lots of UNIX security depends on the contents of the users' home directories. If they are owned by ftp, you are potentially allowing the smart hacker unlimited access if he successfully changes its permissions to 777 (see the remark on SITE CHMOD in the preceding bulleted item). Make sure no files or subdirectories under ftp's home directory are owned by ftp to prevent an intruder from replacing them using virus-contaminated files. Unattended FTP Transfers There are times when the need arises to transfer many large files using FTP. If you do it manually, you must login to the target server and attend to the transfer process on a file-by-file basis. Depending on the size of the files, you might end up spending considerable time just waiting for each transfer to complete before initiating the next one. Add to this scenario the requirement to have the transfers carried out after hours, which is usually the case, and you find yourself in an unpleasant situation. Well, not to worry, because most of the ftp (client) implementations that exist on the UNIX platforms include support for unattended operation. Better still, these ftp clients can take input from script files containing details of the action requests you want executed to bring about the desired results. Here are the required steps to implementing unattended ftp sessions: Using vi, or your preferred text editor, create a file containing all the ftp commands that are required to invoke the desired actions. Here is an example of what a simple file might contain: open tenor user maya apassword binary cd /incoming put myfile bye This file specifies all the actions necessary for the transfer of the file myfile to the /incoming directory on host tenor. Notice how the file starts with an open command specifying the host name of the target server. Next, the user name (maya) and password (apassword) are sent to host tenor for authentication using the ftp user command. Assuming that all goes well, the transfer mode is set to binary, a change directory command (cd) is issued, and the put command transfers the file myfile to host tenor. Upon completion of the transfer, the script instructs the ftp client to drop the connection using the bye command. Because the order in which the ftp commands are listed is vital to the success of the unattended operation, it makes sense to rehearse the commands following the exact order depicted in the file. This way you can assess its accuracy and take corrective measures, if needed, before trusting the script to production. Next, verify the automated operation of the script. To invoke ftp and let it execute the contents of the script without human intervention, enter the following on the command line: # ftp -vin < scriptname where -v Stands for verbose mode. As implied, this switch turns on verbosity. It helps you follow on the "conversation" taking place between the FTP server and your client. Hence, it is convenient for troubleshooting the script. -n Turns off the autologin feature. If this switch is not included, ftp attempts autologin based on any .netrc file it might find in your home directory. The login initialization parameters contained in that file might cause some conflicts leading to abortion of the session. -i Turns off interactive prompting during multiple file transfers--a time-saving feature. scriptname Is the name of the file that contains the ftp script. Following is a simple script called ftpscript: open tenor user root apassword cd /usr/sam put * ls * ftplist bye and the following is the output resulting from its execution: # ftp -vin < ftpscript Connected to tenor. 220 tenor FTP server (UNIX(r) System V Release 4.0) ready. 331 Password required for root. 230 User root logged in. 250 CWD command successful. local: budget.rpt remote: budget.rpt 200 PORT command successful. 150 ASCII data connection for budget.rpt (100.0.0.2,1239). 226 Transfer complete. 746 bytes sent in 0.03 seconds (24 Kbytes/s) 200 PORT command successful. 150 ASCII data connection for /bin/ls (100.0.0.2,1241) (0 bytes). 226 ASCII Transfer complete. 221 Goodbye. # This script included something that the previous one did not include: the ls /usr/root/ftplist command. Notice that this command is included right after the mput command. This command helps verify whether the files intended for transfer were indeed transferred to the destination host. Because this is meant to be an unattended operation, the filename (/usr/root/ftplist) is included as an argument for the ls command, forcing it to log the returned listing to the specified file, and making the information conveniently available for subsequent verification. After you rehearse the script, as suggested, and trust it, you can use either at or cron tools to schedule it for execution at any desired time. You can omit the -v switch if desired, which turns verbosity off so that the command looks like this in the cron entry: ftp -in < ftpscript Preferably, however, you'd rather have verbosity on, and redirect its output to a log file. This way, should the session fail to meet your expectation, you can look up the session details as logged in the file for further analysis while trying to narrow in on the causes underlying the failure. Here is the command for meeting the logging requirement: ftp -inv < ftpscript > logfile Using the ability to schedule unattended FTP file transfers, and the third-party transfer capability together, a system administrator can virtually perform unattended transfers not only between her server and another machine but between any two hosts on the network. Troubleshooting FTP As a system administrator, you are required to respond to and solve user and system related problems including those resulting from the use of FTP. There is a wide range of reasons that might cause an FTP session to fail; anything from user configuration error and network failure to problems resulting from system failure or defective software. Hardware and network-related problems are dealt with in detail in Chapter 20. In this section, only issues directly related to FTP are considered. Following are suggested areas of investigation that might prove worthwhile in sorting out FTP failures: If a user reports a problem connecting, the first thing that you might want to do is attend to the user's workstation (if possible) and ask that user to go through the motions of logging in. Verify that the user is following the proper login procedure. If not, educate the user on how to do it. If, even after the user follows the proper procedures, he is still having problems connecting to a server, try to assess whether the failure is a local configuration issue on the client workstation, a local hardware problem, or a network problem. If the user is getting a Connection refused type of error, the problem is likely to be a logical one. There are several reasons for this error to arise: 1. If the user relies on the $HOME/.netrc file to autologin to the server, check the contents of the file and ask the user to verify the information pertinent to the login process. Invalid information can lead to the server's refusal to connect a user. If all is fine with the contents of the .netrc file, check its permissions. The permissions should be set so that no one but the owner is allowed to read or write its contents. Failure to meet this requirement can result in failure to log in to an FTP server. 2. Verify that the user has a valid account on the remote server. If not, or if the account is expired, you might refer her to the administrator of the host in question for subsequent resolution. 3. Another thing to look at to explain a valid account's failure to FTP the server is the /etc/ftpusers file. If it exists and the account name is included in the file, the ftpd daemon prevents the user from logging in. Again, recommend that the user bring up the issue with the administrator of the host in question for resolution. 4. Users failing to use one of the shells specified in the /etc/shells file are bound to fail to connect via FTP. Check the /etc/shells file on the target host and take the required action to remedy the situation. 5. Although rare, it is possible that somehow the port on which the ftpd daemon is listening to connections is changed to other than the default (21). Should this happen, ftp clients defaulting to port 21 when making connection requests will be refused entry to the server. Check the server's /etc/services file to verify the port number on which inetd is listening on behalf of ftpd for connection request. This can be accomplished using the following command: # grep ftp /etc/services ftp 21/tcp Ö. If the port number is different from 21, this is where the trouble lies. Contact the system administrator for that host and try to find out why this is so. The port number might have been changed maliciously, or might have been required for other reasons. Depending on the reason, the port might have to be changed back to the default or left at its new value. In the latter case, the user should be advised to try to ftp to that host using the existing port. Assuming that the port number is 420, the ftp command should look like this: # ftp hostname 420 If users complain about frequent abrupt disconnection of FTP sessions, you might want to increase the timeout value set for the ftpd daemon. By default, most daemons self-configure on startup to 15 minutes. Congested network conditions might lead to server timeouts, resulting in disruption to ongoing user sessions. Try increasing the timeout value on the server. Be careful not to set it too high; otherwise, users with idling FTP sessions end up needlessly hogging FTP connections at the expense of other users waiting for their release. To change the timeout value on the server, edit the /etc/inetd.conf file, and adjust the entry affecting the ftpd daemon to include the -ttimeout command parameter as shown in this example: ftp stream tcp nowait NOLUID /etc/ftpd -t1800 ftpd According to this entry the timeout value has been increased to 1800 sec. (30 min.). After making the change, send the inetd daemon a SIGHUP signal to reinitialize it so that the new change takes effect. If a user complains about the inability to transfer multiple files using mput along with wild characters in the filename specification, verify that she has filename expansion turned on. Filename expansion can be toggled using the ftp glob command as shown below: ftp> glob Globbing turned off ftp> glob Globbing turned on ftp> Users complaining about failure to transfer files reliably might have ASCII transfer type turned on instead of BINARY type when sending or receiving non-ASCII files. Verify the transfer type and take corrective action (including educating the user) if need be. One thing you might want to try to resolve some of the more complex problems is to turn on debugging upon invocation of the FTP session. You can do this upon invoking the ftp command by using the -d switch # ftp -d hostname or after invoking ftp using the debug command ftp> debug Debugging turned on. ftp> The debug mode allows you to follow up on the progress of the FTP session. Whenever enabled, it displays session data indicating the direction of the exchange and the ftp messages being exchanged. As shown in the following example, -[ra] indicates that the local host is sending, whereas a three-digit code indicates that the remote host is sending: # ftp tenor Connected to tenor. 220 tenor FTP server (UNIX(r) System V Release 4.0) ready. 331 Password required for root. 230 User root logged in. ftp> debug Debugging on (debug=1). ftp> cd /usr/sam ---> CWD /usr/sam 250 CWD command successful. ftp> ls ---> PORT 100,0,0,2,5,6 200 PORT command successful. ---> LIST 150 ASCII data connection for /bin/ls (100.0.0.2,1286) (0 bytes). total 6 -rw-rw-rw- 1 root other 55 Oct 19 15:55 ftpscript -rw-rw-rw- 1 root other 721 Oct 19 15:55 proxy -rw-rw-rw- 1 root other 10153 Oct 17 19:42 sendAriel -rw-rw-rw- 1 root other 204 Oct 19 15:55 tenor.ftp -rw-rw-rw- 1 root other 519 Oct 19 15:55 tenor.help -rw-rw-rw- 1 root other 12870 Oct 19 15:55 testfile 226 ASCII Transfer complete. ftp> get testfile local: testfile remote: testfile ---> PORT 100,0,0,2,5,8 200 PORT command successful. ---> RETR testfile 150 ASCII data connection for testfile (100.0.0.2,1288) (12870 bytes). 226 ASCII Transfer complete. 13415 bytes received in 0.58 seconds (23 Kbytes/s) ftp> quit ---> QUIT 221 Goodbye. # Server-related problems can result from software bugs or poor implementation. Similar to the client, the ftpd daemon comes with a -d switch, which can be optionally used to turn on debugging. Once turned on, debugging data are logged to the /usr/adm/syslog file. Equally possible is the ability to log details pertaining to individual FTP sessions. This is achieved by specifying the -l switch in the entry for ftpd in the /etc/inetd.conf file as shown below: ftp stream tcp nowait NOLUID /etc/ftpd -l ftpd Be careful not to leave debugging or logging turned on for too long. The /usr/adm/syslog file can grow to an astronomical size over a short period of time due to the amount of details both modes generate. Turn them off as soon as you finish troubleshooting. Summary FTP service protocol is the probably the most widely used method of downloading and uploading files to UNIX and other hosts on the Internet. Even Web browsers use FTP as the underlying mechanism for transferring files when requested. Consequently, system administrators are faced more than ever with the challenging task of providing reliable and secure across-the-Internet FTP services. The associated tasks involve setting up FTP services for local access as well as setting up anonymous FTP service for the Internet community of users. Setting up such services requires a solid understanding of how the FTP protocol works. Equally involved is developing the required skills to troubleshoot FTP services in the most efficient way. This chapter shows how to set up FTP services, including anonymous access. It also includes tips on verifying the security of the service. Both third-party transfers and unattended transfers are explained and illustrated. Finally, common FTP problems and ways of troubleshooting them are discussed. Most implementations of FTP servers and clients are similar in features and capabilities. For details of the implementation, check the vendor's documentation. A fair assessment of the implementation falls short of achieving the desired goal if not done in the light of a thorough reading of the standard.

©Copyright, Macmillan Computer Publishing. All rights reserved.