Questions and Answers about Serial Communications and UUCP ********************************************************** My serial connections are losing characters =========================================== Here are three possibilities. First, check the NCLIST kernel parameter. This governs how many CLIST structures are allocated; these are used to buffer input and output. If this is too low, then at times of high serial I/O demands, your system will run out of CLIST structures and start discarding characters. Note that there is a limit as to how many CLIST structures may be allocated to an individual process, regardless of how many are available systemwide. This is done to prevent one misbehaving program from monopolizing all of the CLIST structures. Look for the tunable parameter TTHOG (only available in newer Unix systems), which controls this limit. The next possibility is that you may have an old UART (8250 or 16450). See the next answer for more info. And you may also have flow control problems. The devices at either end of a serial cable must agree on what flow control is being used or else you can end up with data loss, unexplained pauses in the data stream, etc. See the man pages for stty and termio for more information on the available settings. ____________________________________________________________________________ What do the terms UART, 8250, 16450 and 16550 mean? =================================================== UART means Universal Asynchronous Receiver/Transmitter. This is a chip which receives and transmits data serially; each serial port you have will use one, though it is possible that several may be integrated into one chip. 8250, 16450 and 16550 are all common types of UARTs. The 8250 is an old chip which cannot run at high speed. The 16450 is similar to the 8250 except that it supports data communications at higher speeds. Both of these chips generate an interrupt for every character that is sent or received, which basically tells the CPU either "Here is some data for you" or "Feed me!" This is all very well, except that at high speed, the number of interrupts (nearly 4000 per port per second at 38 400 bps) can overwhelm a CPU, bringing system performance way down. Also, if the CPU is busy servicing another interrupt at the time, the serial port's interrupt may not be serviced in time, which will cause a character to be lost. The 16550 is pin-compatible with the 16450 and, by default, runs in 16450 mode. This makes it compatible with software which is not 16550-aware. If your software is 16550-aware, it can turn on a special mode in which the 16550 buffers all data with 16-byte internal buffers. This not only allows the CPU to deal with far more bytes at a time, increasing efficiency, but also means that if the CPU can't service the interrupt before the next character comes in, there's still space in the buffer for it. 16550 support was introduced in Xenix 2.3.4, ODT 1.1 and Unix 3.2.2. If you have these, or later, versions, your operating system will automatically detect 16550-equipped ports and will enable their buffering. A third-party serial driver called FAS includes 16550 awareness in its feature set; you may wish to investigate this as well. FAS can be found at ftp://ftp.fu-berlin.de/pub/unix/driver/fas/. Note that the above is not really applicable to intelligent multiport serial cards. While these cards may well use 16550s, it is the processor on the serial card which is responsible for dealing with the serial ports it controls, and the main CPU has nothing to do with the UARTs. ____________________________________________________________________________ How do I adjust my 16550's trigger level? ========================================= The answer is different for Unix and Xenix, but much of the information is the same, so it's been grouped together here. We will deal with the common information first, and the specific details for Xenix and Unix. By default, Xenix and Unix set the 16550's trigger level to 14. This means that once fourteen characters have been received, the 16550 will generate an interrupt (it will also generate an interrupt if the buffer is not full but serial data flow has stopped, so the system doesn't always have to wait until the trigger level is reached). This give the system two character times in which to begin to clear the buffer; at high speed on a highly loaded system, this may not be enough, and you may still lose characters even though you have a 16550. On the other hand, this value should generally be set as high as possible to reduce the number of interrupts generated; servicing an interrupt is quite costly in terms of CPU time. There is an array in the kernel called sio_fifoctl[]. It is a 16-byte array with control values for different minor numbers. To find which array element will be used for a particular serial port, AND the minor number of the port with 0x0F (for example, /dev/tty2A has a minor number of 136 and /dev/tty2a of 8; either one ANDed with 0x0F yields 8, so sio_fifoctl[8] controls this port). There are four different values you may wish to use for the entries in sio_fifoctl[]. A value of 0x0F sets the trigger value to 1; 0x4F sets it to 4; 0x8F sets it to 8; 0xCF sets it to 14 (these values are determined by the 16550 itself, not by SCO, and other values will not set the trigger level to intermediate values). In Xenix, this parameter is set by patching the disk image of the kernel (/xenix) using adb (the info on how to find adb is elsewhere in this FAQ). The following is a sample adb session to change the trigger level of /dev/tty2A to 8 from 14 (the line numbers in parentheses are for the explanation below); the asterisks are adb's prompt and should not be typed in: 1. cp /xenix /xenix.save 2. adb -w /xenix - 3. * sio_fifoctl+8/x 4. sio_fifoctl+0x8: 0xcfcf 5. * sio_fifoctl+0x8/w 0xcf8f 6. sio_fifoctl+0x8: 0xcfcf= 0xcf8f 7. * $q Line 1 makes a backup, and line 2 runs adb in write mode. Line 3 tells adb to print the current value of sio_fifoctl[8]. Line 4 is adb's reply, which includes two bytes from this array (the rightmost one is the value for sio_fifoctl[8], and the leftmost is for sio_fifoctl[9]). You must look at these carefully, as one half will have to be changed while the other will have to be left alone. In line 5, we write 0xCF8F into this location; note that the value for sio_fifoctl[9] is left unchanged at 0xCF. Line 6 is adb's reply giving the old and new values. Line 7 quits adb. For Unix, there is a table at the end of the text file /etc/conf/pack.d/sio/space.c which gives the same array. It is formatted in the same manner (to find the appropriate value, AND the minor device number with 0x0F). If you run Unix, check the man page for the sar command to see if you have the -g option to check for serial I/O overruns. If so, try running it. If you see overruns, this indicates that your trigger level is set too high and the system doesn't have adequate time to service the 16550. The cure is to turn the trigger level down one notch and try again. The information for Xenix in this answer is taken from the comp.unix.xenix.sco FAQ, maintained by Chip Rosenthal. The copyright for that document reads: This_collection_is_Copyright_1992-1994,_Unicom_Systems_Development,_Inc._All rights_reserved._Permission_granted_to_reproduce_and_distribute_this document_provided_this_notice_remains_intact_and_any_changes_to_the document_are_clearly_marked._We_have_tried_to_review_all_information,_but cannot_guarantee_it_for_any_particular_purpose._We_do_not_offer_any warranties_or_representations,_nor_do_we_accept_any_liability_for_any damage_resulting_from_the_use_or_misuse_of_information_or_procedures_in this_document._ ____________________________________________________________________________ I can transfer small files via UUCP but large files won't go ============================================================ I can transfer small files via UUCP but large ones go really slowly =================================================================== This may indicate a flow control problem. uucico, the program that actually performs UUCP transfers, turns off all hardware and software flow control on the port it is using (prior to 3.2v4.0, which allows you to specify what it should use). If your modem's buffers are too small, then the stream of data involved in a large file transfer may overflow them, causing transmission errors. This may just cause your throughput to go way down, or it may result in a total inability to transfer larger files. Run uucico with debugging (you may find /usr/lib/uucp/uutry to be useful) and watch for "alarm" messages. If you see these, it's an indication that some characters are likely being dropped during transmission. If you are using a serial port on a multiport serial card such as one from Equinox or Digi, your board may have shipped with a utility that lets you permanently set flow control on your port. Another alternative is to reduce uucico's window size. Under 3.2v4.0, uucico does not turn off flow control; it is left at whatever setting the dialer used. If you are using a compiled dialer and have the source and the development system, you can write in whatever stty settings you desire. Under 3.2v4.1, if you are using atdialer, you can specify stty settings in /etc/default/atdial*. Note that it has been reported that cu, even under 3.2v4.x, turns on XON/XOFF flow control and, in doing so, disables hardware flow control. In 3.2v4.x, a new stty setting has been added to perform bidirectional RTS/CTS flow control. This is CRTSFL. CTSFLOW and RTSFLOW do not perform proper bidirectional flow control; they allow the modem to signal the computer to stop sending, but not vice versa. Depending on what modem you have, you may need CRTSFL if your system can't accept characters as quickly as your modem can produce them. ____________________________________________________________________________ How do I get better UUCP throughput? ==================================== You may or may not be able to. Here are some rules of thumb which may or may apply to your situation: If you and the remote site both use Telebit modems, enable UUCP spoofing. This enables the modems to maintain the UUCP protocol between them, and means that your CPU's response time to individual UUCP packets is not as critical. Some sites can achieve higher throughput with a Telebit link than with a direct connection at the same baud rate. If you and the remote site are both running versions of UUCP which allow you to specify different protocols (SCO Unix 3.2v4.0 and above, for example), look through your release notes to find the highest-performing protocol that's applicable to your situation. For example, the standard g protocol has its own error detection and correction logic. If you are running over a guaranteed error-free link, this is unnecessary overhead, and eliminating it by selecting a protocol that relies on an error-free link can speed up transfers. Be careful that you don't specify such a protocol for a non-error-free link, or else your transmissions will occasionally be corrupted. Note that just because you are using error-correcting modems does not guarantee that your data will be error-free; there is a possibility of lost characters and flow control problems in the serial ports at both ends of the connection. While the link between the modems may well be error-free, the end-to-end link from one uucico to another may not. You should usually use g or G for modem links, and reserve t, e, f, etc. for network connections. If you are running over a link that uses MNP, V.42 or V.42bis, set your window size as high as possible to ensure that the modems have as much data to work with as possible. For further tips on high-speed modems, see the section on maximizing serial throughput. For information on changing uucico's window size, see below. If the site with which you are communicating sometimes drops packets, try adjusting your packet delay to a lower number. If you are operating over a link with high latency (i.e. it takes a long time for a packet to get from one site to the other - an example would be a satellite link), you may need to increase your packet delay. For more details, see below. See the note below on how to get the UUCP Internals FAQ. ____________________________________________________________________________ How do I change uucico's window size? ===================================== For OpenServer Release 5, see the man page for Configuration. For earlier versions, read on. You need to have adb or _fst to do this. See the information on getting these, which can be found in section 1. First, make a backup copy of /usr/lib/uucp/uucico! This is very important. Also, note its ownership and permissions. To change the window size to 7, run the following command from the Bourne shell: adb -w /usr/lib/uucp/uucico - << EOF $d _windows/w 7 $q EOF When you have done this, reset the ownership and permissions to what they were originally; this process will change them on you. Note that specifying a window size greater than seven may break uucico, as the g protocol is designed for a window size of no more than seven. Also, the documentation and/or scopatches that ship with some SCO products list the second line of the above as "%d". This, at least on the copy of adb I have, will fail; $d is correct. If you are using Unix 3.2v4, leave the underscore off the symbol name. On Xenix 2.3.4, you may be tempted to use the scopatch command to make this change. Don't bother; there are at least three bugs in the short script which does this work, and it's just as easy to do it manually as it is to fix the script. ____________________________________________________________________________ I increased my window size but nothing changed ============================================== When UUCP is negotiating a connection, each side will tell the other what window size to use when sending. Therefore, if your window size is 7 and the remote uses 3, you will be sending files using a window size of 3, but the other side may send with a window size of up to 7. Note that the UUCP on the other side may not support the window size you specify, and may send with a smaller window than you requested. While this should not cause problems, it may provide lower performance than you'd like. If your connection does not have data compression, error correction, or long transmission delays, and the two sites involved have sufficient CPU power to respond quickly to incoming packets, a window size of 2 or 3 should be sufficient to achieve streaming data flow. In this case, a larger window size will not provide any benefit. If you have a Telebit modem, see the Telebit notes below. ____________________________________________________________________________ I increased my window size and UUCP broke! ========================================== If your modem, or the modem on a remote site, has small data buffers, a large window size may cause the modem's buffers to overrun. If the site that broke polls your system, you may wish to create a separate copy of uucico with a smaller window size, and specify it as the login shell in /etc/passwd. If you are the polling site, you may have to restore your original uucico. Alternatively, you could try a window size of 4, then 5, then 6, then 7, and see at what point UUCP stops working. You will then know the maximum window size you can use. You may also wish to ask the sysadmin at the remote site if there is anything they can do, such as forcing flow control to be used on their modem, that may remedy the problem. You may, however, have no choice but to return to using a window size of 3. If you have a Telebit modem, see the notes below relating to Telebit modems. ____________________________________________________________________________ UUCP frequently has to resend packets ===================================== If you are using a modem with error correction and/or data compression, or if you are making a long-distance connection, there are delays inherent in your connection which may cause UUCP to timeout and resend packets. You may need to increase the packet timeout. See also the section above on problems in which UUCP can transfer small files but not larger ones. ____________________________________________________________________________ How do I change uucico's packet timeout? ======================================== For OpenServer Release 5, see the man page for Configuration. For earlier versions, read on. This is very similar to changing the window size; once again, you will need to have a copy of adb, and you must first make a backup copy of uucico. Then run the following Bourne shell command: adb -w /usr/lib/uucp/uucico - << EOF $d _pktime/w 5 $q EOF Replace 5 with the desired number of seconds. Note that you should set this parameter to the smallest value that works reliably. If you specify too large a value, it will reduce throughput if you encounter a bad line or other conditions that require packets to be resent. If you are running Unix 3.2v4, leave the underscore out of the symbol. Once again, restore the permissions and ownership once you're done. ____________________________________________________________________________ What special considerations are there regarding my Telebit modem? ================================================================= If you are using Telebit UUCP spoofing (S111=30), the modem spoofing firmware will only negotiate a 3-window connection, so changing the window size to 7 will not do anything for spoofed connections. It will, however, be effective for non-spoofed connections (i.e. if you connect to a site which is not using a Telebit modem). Some people have reported problems with Telebit modems and UUCP-g spoofing when they change their window and/or packet sizes. Apparently, Telebits can handle requests for increased window sizes without error (though they will only use a maximum window size of 3), but will not work correctly with large packets (probably above the SCO default of 64 bytes). There is a bug in version LA 5.00W of the Telebit Worldblazer firmware that causes uucico to fail in startup when the answering uucico is set for windows=7. ____________________________________________________________________________ What are all the V.something codes, MNP, HST, etc.? =================================================== Here are the most common data modem signalling standards. Note that the descriptions are not complete technical descriptions, and only cover the major feature of the standard (i.e. no discussion is made of fallback data rates etc.) o Bell 103 - North American 0-300 bps o Bell 212A - North American 1200 bps o V.22 - International 1200 bps; not generally used in North America o V.22bis - 2400 bps o V.32 - 4800 & 9600 bps o V.32bis - 4800 - 14 400 bps o V.32terbo - Vendor standard (no formal recognition), 16.8 & 19.2 kbps o V.FC - Vendor standard (no formal recognition), up to 28.8 kbps o V.34 - International standard, up to 28.8 kbps; this has been extended to 33.6 kbps o HST - USRobotics' proprietary High-Speed Transfer, 9600 - 16 800 bps o PEP - Telebit's proprietary standard; also Turbo PEP o X2 - USRobotics' asymmetrical high-speed modulation; up to 56 kbps downstream (limited to 53.3 kbps by legislation), V.34 upstream. See http://x2.usr.com/ o K56flex - same idea as X2, different implementation There are also numerous standard for error correction and data transmission. MNP (Microcom Networking Protocol) is a family of protocols with various levels. MNP levels 1 through 4 denote error correction schemes of increasing sophistication. MNP level 4 is the most common MNP error correction level; while there are higher levels, they are not terribly widespread. You may never see your modem report an MNP level 4 connection if you have data compression enabled; it will report level 5 instead. MNP level 5 is usually considered a 2:1 compressor, meaning that it will generally compress your data by up to a factor of 2 (though it can exceed this on some data and not reach it on other data). Note that it does not check to ensure that it can actually compress the data; if you send precompressed data through it, it will actually increase the amount of data which must be transferred between modems, thereby decreasing throughput. V.42 is another error correction standard; unlike MNP, it is non-proprietary. Its primary error-control protocol is called LAP-M. It also includes a provision for falling back to MNP level 4 error correction should the remote modem not support V.42. V.42bis is the corresponding data compression standard. It is both more efficient than MNP level 5, providing up to a 4:1 rate (even higher on particularly repetitive data), and more intelligent, in that it can recognize whether or not it can compress data and send the data uncompressed if this makes the most sense. For further information on modems, see the newsgroup comp.dcom.modems. ____________________________________________________________________________ How do I maximize serial throughput? ==================================== This answer assumes that you have an error-correcting and/or data-compressing modem. The rule of thumb here is to feed data to the modem as fast as you can. The more data it has to work with, the more efficiently it can compress it. Not only that, but there is a more-or-less fixed overhead involved in bundling data into packets, which is how the modems transmit it, so when error correction and/or data compression is in use, you can reduce this overhead to a minimum by ensuring that the modem has as much data as possible to put into each packet. The usual rule of thumb is to feed the modem four times faster than the fastest connection rate it supports. If you have a V.32 modem, which supports connections of up to 9600 bps, you should communicate with it at 38 400 bps (assuming your hardware supports this rate). If you are using error correction but not data compression, the next speed up (19 200 bps, in this case) is generally sufficient. The modem will have an internal data buffer of some size; it could be as large as a couple of kilobytes on better modems. If you are sending it data faster than it can transmit it (which you should be), this buffer will fill up over time. You will need to have handshaking between the modem and the computer so that the modem can signal to the computer to stop sending when the buffer is nearly full, and to start sending again once there is room in the buffer. This can be done in software (usually using the ASCII XON and XOFF characters), or in hardware (usually using the RS-232 CTS and RTS lines). The use of software handshaking requires less wires between the computer and the modem, but will interfere with the transmission of binary data. The use of hardware handshaking requires additional wiring, but will not interfere with binary data. There are advocates of both methods; I personally prefer hardware handshaking. Whichever method you choose, you must make sure that you configure both the modem and the computer (via stty settings) to use the same protocol. ____________________________________________________________________________ My data-compressing modem doesn't work much faster than my old modem ==================================================================== There are a couple of possible reasons for this. Firstly, you may be transferring data that has already been compressed and is therefore not compressible (compressed news batches, for example). If your modem connects to the remote site using V.42bis data compression, this will not adversely affect throughput. If, however, you are using an MNP level 5 connection, you will actually lose throughput when sending compressed data. If the majority of your transfers are already compressed, you should disable MNP level 5 data compression. Note that using MNP level 3 or 4 error correction, or V.42 error correction, without any compression will increase throughput regardless of the type of data being transmitted. Also, due to the way data compression and error correction in modems work, there are delays involved while the two modems agree on whether or not each packet of data was received correctly. If possible, you and the remote site should try to increase uucico's window size; this will cause more data to be transmitted at once, which will not only improve the efficiency of your modem's data compression, but will also often mean that the response to a previous packet arrives before the window has expired. I'm afraid a complete discussion of packetization and sliding window protocols is beyond the scope of this FAQ; just trust me on this one if you don't quite follow me. Further discussion can sometimes be found in the newsgroup comp.mail.uucp. ____________________________________________________________________________ A BSD-based machine can't connect to mine via UUCP ================================================== BSD-based UUCP sends data with even parity; SCO expects no parity. Add "" P_ZERO to the chat script in the BSD machine's Systems or L.sys file. For example: xnxbox Any uucp 19200 5553333 "" P_ZERO "" \r in:--in: nuucp ____________________________________________________________________________ Where can I get more information on UUCP protocols? =================================================== There is a FAQ posted to the news group comp.mail.uucp which goes into a good deal of detail on the format of UUCP files, the handshake process at the start of a connection, what some of the debugging output from uucico -x? means, and the internals of numerous UUCP protocols including the common g protocol and several others. If you do not get this group, you can find the FAQ at ftp://rtfm.mit.edu/pub/usenet-by-hierarchy/comp/mail/uucp/UUCP_Internals_Frequently_Asked_Questions. If you do not have anonymous ftp access, send email to mail-server@rtfm.mit.edu; include the words "help" and "index" in separate lines in the body of your message. ____________________________________________________________________________ I edited my gettydefs and things broke ====================================== You can check gettydefs using /etc/getty -c /etc/gettydefs; this command will print out everything you could possibly want to know, and then some. Common mistakes include not leaving a blank line between entries or having a cycle of settings but forgetting to close the loop (e.g. if 1 fails, use 2; if 2 fails, use 3; if 3 fails, you forgot to tell it to go back to 1). ____________________________________________________________________________ I can't get shared dial-in and dial-out working =============================================== Here are a few things to keep in mind. Under Unix, the entries in /etc/inittab and /usr/lib/uucp/Devices must match exactly; if one says "ttyi3" while the other says "/dev/ttyi3", getty won't consider them to be the same port, and your modem will not be sent the undialer string. Serial ports to be used by UUCP (or even just initialized with an undialer) should generally be owned by user uucp and group uucp. If you set it this way but it keeps being put back to something else, chances are you're suffering from the problem mentioned above. If you have several entries in /usr/lib/uucp/Devices for a port (specifying different dialers for different speeds or whatever), only the first one will actually be used to reset the port. Make sure the first one listed is the one you want - that's usually marked ACU, but local customs may vary. ____________________________________________________________________________ What are some common settings for my modem? =========================================== For the real answers, read your modem's manual. If you want to discuss modems, see comp.dcom.modems. Having said that, here are some common setup strings for various features on many modems; see your manual to determine which one(s) apply to your modem. Note that not all modems will accept the following commands; even the manual for one modem is too large to fit, let alone all possible modems. o Locked DTE rate - this causes your modem to always talk to your serial port at the same rate, regardless of the data transfer rate between modems. Try AT\J0, AT&Qn (n is a number which depends on other settings as well), or AT&B1. o Hardware flow control - this uses out-of-band signalling on the RTS and CTS lines to perform flow control and will not interfere with the transmission of binary data, as software flow control will. Try AT&K3 or AT&E4. o Software flow control - particularly on interactive (cu) connections, you may wish to use software flow control in some cases. This is usually done with the same command as hardware flow control, but with a different number (e.g. AT&K4). One disadvantage is that if you get garbage characters (due to line noise, modem disconnection, etc.), you will sometimes get a flow control character as part of the garbage; the system doesn't know it's garbage and so it may appear to hang the port. o Correct use of carrier detect - the DCD line is used to indicate to the computer that the modem has established carrier. This setting is almost universally done with AT&C1. o Error control - this feature, if negotiated between modems, causes the modems to take care of retransmitting data when errors crop up on the phone line. Try AT&Qn (again, n depends on other settings), AT\Nn (ditto), or AT&E1. See the discussion elsewhere in this FAQ to help determine what, if any, error control you should use. Generally, though not always, you will want to enable all error control methods your modem supports, in order to be able to negotiate connections with other modems, but may need adjustment in some circumstances. o Data compression - this feature, if negotiated between modems, causes the modems to attempt to compress the data before sending, improving throughput. Try AT&Qn (n depends on other settings) and AT%C1; some modems also use registers such as S36 to fine-tune this. See the discussion elsewhere in this FAQ for more information on when you might or might not want to use data compression. o Correct use of DTR - the DTR line, when dropped by the computer, should cause the modem to hang up (and, according to some opinions, reset entirely). This feature is almost universally done with AT&Dn, but the supported values of n may vary from modem to modem. Try AT&D2 to hang up and AT&D3 to hang up and reset. o Writing settings - Two cautions first. First, you do NOT want to do this every time you initialize the modem. The EEPROMs used in most modems to store this information have a lifetime of something like 10 000 writes. At 30 reinitializations a day, you'll burn out your EEPROM in around a year. Second, if you use this to do the initial setup of the modem, document your settings somewhere so that if your modem does die, you'll have the settings you used somewhere (I'd recommend in a comment in your /usr/lib/uucp/Dialers file). See AT&W; some modems have several settings available, written to with AT&W0, AT&W1, etc. See AT&Y on some modems to determine which settings are used upon reset, and ATZn for resetting the modem. o To Echo or Not To Echo - you can configure your modem to echo commands back, or not to echo commands back, with ATE1 and ATE0, respectively. You can control the sending of result messages with ATQ1 and ATQ0; some modems also use ATQ2 to disable result codes on incoming connections. o Enabling busy detection - ATXn controls both the range of result codes the modem sends (such as enabling the display of connect speed rather than simply CONNECT) and what phone company signals the modem detects (e.g. busy, dialtone). Most people use ATX4; check your manual for other options. o Auto-answer - use ATS0=n, where n is the number of rings to allow to pass before answering. ATS0=0 disables auto-answer. You may wish to disable auto-answer while calling out or at other times when you don't wish the modem to answer. o Escape code - most modems use a pause, followed by three plus signs, followed by a pause, as a signal from the computer that the modem should go into command mode while on-line. This may cause problems in some cases. To disable this escape sequence, use ATS2=128.