From: vince@victrola.wa.com (Vince Skahan) Reply-To: vince@victrola.wa.com (Vince Skahan) Followup-To: poster Newsgroups: comp.os.linux.announce,comp.os.linux.admin,comp.answers,news.answers Subject: Linux News HOWTO Approved: linux-announce@tc.cornell.edu (Matt Welsh) Archive-name: linux/howto/news Last-modified: 26 January 1994 The Linux News HOWTO by Vince Skahan, v1.6, Last Modified 26 January 1994 --------------------------------------------------------------------------- This document describes the setup and care+feeding of USENET News under Linux. You need to read this if you plan to post or read USENET news either locally on your site or between your site and other sites. You probably do *not* need to read this document if don't plan to provide USENET news as a feature of your system. 0. Introduction The intent of this document is to answer some of the questions and comments that appear to meet the definition of 'frequently asked questions' about USENET News software under Linux in general, and the version in the Linux SLS distribution in particular. This document and the corresponding Mail and UUCP 'HOWTO' documents collectively supersede the UUCP-NEWS-MAIL-FAQ that has previously been posted to comp.os.linux.announce. 0.1 New versions of this document New versions of this document will be periodically posted to comp.os.linux.announce, comp.answers, and news.answers. They will also be added to the various anonymous ftp sites who archive such information including sunsite.unc.edu:/pub/Linux/docs/HOWTO. 0.2 Feedback I am interested in any feedback, positive or negative, regarding the content of this document via e-mail. Definitely contact me if you find errors or obvious omissions. I read, but do not necessarily respond to, all e-mail I receive. Requests for enhancements will be considered and acted upon based on that day's combination of available time, merit of the request, and daily blood pressure :-) Flames will quietly go to /dev/null so don't bother. Feedback concerning the actual format of the document should go to the HOWTO coordinator - Matt Welsh (mdw@sunsite.unc.edu). 0.3 Copyright Information The News-HOWTO is copyrighted (c)1994 Vince Skahan. A verbatim copy may be reproduced or distributed in any medium physical or electronic without permission of the author. Translations are similarly permitted without express permission if it includes a notice on who translated it. Short quotes may be used without prior consent by the author. Derivative work and partial distributions of the News-HOWTO must be accompanied with either a verbatim copy of this file or a pointer to the verbatim copy. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions. In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs. We further want that ALL information provided in the HOWTOS is disseminated. If you have questions, please contact Matt Welsh, the Linux HOWTO coordinator, at mdw@sunsite.unc.edu, or +1 607 256 7372. 0.4 Standard Disclaimer Of course, I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk. 0.5 Other sources of information USENET: ======= news.admin.misc General topics of network news administration. news.admin.policy Policy issues of USENET. news.admin.technical Maintaining network news. (Moderated) news.software.b Discussion about B-news-compatible software. news.software.nn Discussion about the "nn" news reader package. news.software.nntp The Network News Transfer Protocol. news.software.readers Software used to read network news. news.sysadmin Comments directed to system administrators. news.announce.newusers Explanatory postings for new users. (Moderated) news.newusers.questions Q & A for users new to the Usenet. Books: ====== The following is a non-inclusive set of books that will help... 'Managing UUCP and USENET' published by O'Reilly+Associates is in my opinion the best book out there for figuring out the programs and protocols involved in being a USENET site. 'Unix Communications' published by The Waite Group contains a nice description of all the pieces (and more) and how they fit together. 'Practical Unix Security' published by O'Reilly+Associates has a nice discussion of how to secure UUCP in general. 'The Internet Complete Reference' from Osborne is a fine reference book that explains the various services available on Internet and is a great "one-stop-shopping" source for information on news, mail, and various other Internet resources. 0.6 Where *NOT* to look for help There is nothing 'special' about configuring and running USENET news under Linux (any more). Accordingly, you almost certainly do *NOT* want to be posting generic news-related questions to the comp.os.linux.* newsgroups. Unless your posting is truly Linux-specific (ie, "please tell me what patches are needed to run INN with the bash1.12 in SLS v1.03) you should be asking your questions in the newsgroups mentioned above. Let me repeat that. There is virtually no reason to post anything news-related in the comp.os.linux hierarchy any more. There are existing newsgroups in the news.* hierarchy to handle *ALL* your questions. 1.0 Hardware Requirements There are no specific hardware requirements for USENET News under Linux. The only requirement of any type is sufficient disk space to hold the software itself, the threads database(s), and the amount of news you wish to keep on the system. Figure on a minimum of 10 MB of disk space for starters. 2.0 Getting USENET News software All the software referenced in this 'HOWTO' is available on the usual Internet anonymous ftp sites. Looking in /news on ftp.uu.net is usually a good way to start. The newspak-2.0.tar.z distribution contains config files and readme files related to building uucp, news, and mail software under Linux from the various freely-available sources. It can usually be found on sunsite.unc.edu in the directory /pub/Linux/system/Mail. 3.0 News Transport Software There are two main sets of news 'transport' software for *nix these days, Cnews and INN. The old 'Bnews' has been declared officially dead and unsupported by its authors. News 'transport' is defined here to be the software that works behind the scenes to post and propagate the news articles as well as making the articles available for the newsreaders to access. You can set your paths to anything you like, as long as UUCP has the absolute path to rnews in the Permissions file and as long as you have your newsreaders configured so that they can find 'inews' and 'mail'. Important - you're asking for trouble if you try to intermix Cnews and INN. Pick one or the other. It's ok to add the NNTP 'Reference Release' into Cnews since they're intended to play well together. 3.1 Cnews Performance Release The current de-facto standard news software is Cnews. It has been around for a number of years, I first saw it sometime around 1988. Cnews's main benefit is its maturity. It runs on about every *nix you can find and there are literally thousands of systems running it worldwide. Its main disadvantage is that it seems to have been intended for uucp-over-modem connections between sites and as such requires the addition of NNTP software to handle realtime Internet feeds and reading. In addition, it requires external threading packages for the various newsreaders, although the NOV package can now be hooked in to help somewhat and it's rumored to have native NOV support in the upcoming 'cleanup' release. Regardless, the beginning USENET admin should probably run Cnews first since it's so stable, well documented, and has many thousands of experienced administrators who can answer questions. The newspak-2.0.tar.z distribution on sunsite contains working config files for Cnews under Linux as well as a couple line patch you'll need to make to 'doexplode' to get around some problems with bash1.12. bash1.13 is now available on the various Linux archive sites that should help. 3.1.1 Installing Cnews Installing the Performance Release of Cnews is absolutely a 'rtfm' project. Just grab the sources, extract them, and follow the instructions. The build.def in newspak was generated by running 'build' the first time and simply looking up the answers by checking out the /usr/include files to get the right answers. When you start the actual compilation, it'll blow up a couple of times. In all cases, you'll need to simply comment out a couple of atoi() and atol() macros that gcc doesn't like when your doit.bin compilation blows up. Also, you need the following tiny change to doexplode to get around some bash1.12 bugs or else you won't feed anything downstream... [...from Steve Robbins - steve@nyongwa.cam.org ...] if [ ! -f $f ] ; then continue; fi # add this line # case "$f" in # comment me out # "out.master/[0-9]*") break ;; # comment me out # esac # comment me out Henry Spencer of Cnews fame says that the code in doexplode that is currently causing problems with bash is on the list to be revised in a future release. It is uncertain at this time if the need for this patch goes away if you switch to the beta bash1.13 now available for Linux. 3.1.2 Configuring Cnews At the very least, you need to edit the following files that all should be in /usr/local/lib/news: active - the active file batchparms - batch parameters explist - article expiration setup mailname - name in headers for mailed replies mailpaths - path to mail moderated postings to organization - your 'org' sys - control what you take and feed whoami - your hostname for the Path: line 3.1.3 Maintaining a Cnews Site First, a significant rule of thumb is to not mess with files by hand that have utilities that configure them. In particular don't set up newsfeeds (run 'addfeed'instead) and don't mess with your active file (run 'addgroup'). When in doubt, read and re-read the docs in the source distribution. Everything else can be done via cron. My crontab for 'news' looks like the following: # take the compressed batches that came in from other systems 20 * * * * /usenet/sw/news/bin/input/newsrun # batch 'em up to go out 0 * * * * /usenet/sw/news/bin/batch/sendbatches myfeedsite # expire C-news 59 0 * * * /usenet/sw/news/bin/expire/doexpire # monitor stuff and report if needed 10 5 * * * /usenet/sw/news/bin/newsdaily 00 5 * * * /usenet/sw/new/bin/newswatch # turn processing of incoming news batches off 6:30AM - 4:00 PM 30 6 * * * /usenet/sw/news/bin/input/newsrunning off 00 16 * * * /usenet/sw/news/bin/input/newsrunning on 3.2 InterNetNews (INN) INN is the newcomer on the scene, but it's gaining popularity as it matures. Its main benefit is speed and the fact that it contains an integrated nntp package. Its main drawback is that it's new and that it doesn't necessarily install and run flawlessly on the many 'standard *nixes' yet. In addition, it operates by having a daemon (the innd) always running plus potentially a overchan daemon to do threading. The tradeoff seems to be memory vs. speed. New USENET admins should probably not try INN until they have experience with either B-news or Cnews. While it's fast and reliable under Linux, it's virtually undocumented for the beginning news administrator (though in practice it's rather simple to run once you figure it out). INN is very particular about its permissions. Don't mess with them. 3.2.1 Installing INN [... from Arjan de Vet - devet@info.win.tue.nl ...] I've made a patch + config kit for INN 1.4 to get it to run on Linux. It can be found at: ftp.win.tue.nl:/pub/linux/ports/inn-1.4-linux.tar.gz INN depends heavily on a good /bin/sh substitute. I use a beta version of bash 1.13 that is now available for Linux on the normal archive sites. bash 1.12 gives some small problems with newgroups not being handled correctly (maybe some other problems too, I don't remember). 3.2.2 Configuring INN Basically follow Arjan's instructions and you'll be all set. Here's the summary of what to do: - In config.data, make sure you have "HAVE_UNIX_DOMAIN DONT" - Add the hostname of the system running innd to hosts.nntp (for a uucp-only site, that's your sitename...) - Make sure you do not have a line for nntp in /etc/inetd.conf - Make sure that you have innshellvars say "HAVE_UUSTAT DO" rather than the 'DONT' in his example config.data if you have uustat from the Taylor UUCP package installed. If you have this defined wrong, it'll result in no outgoing news getting batched. If you run INN, *definitely* define the recommended syslogd stuff because it is very, very helpful. There is a spectacular (!!!!) FAQ for INN that comes out monthly. Look on rtfm.mit.edu for it. You'll be glad you did. 3.2.3 Maintaining a INN Site I've found that there's essentially zero care-and-feeding of a Linux INN site other than having a working 'cron'. Basically you want a crontab that looks something like the following: # daily maint, also expire the .overview database and articles 1 0 * * * /usenet/sw/inn/bin/news.daily expireover delayrm < /dev/null # send 'em out 5 * * * * /usenet/sw/inn/lib/send-uucp [ if you switch to bash1.13, the '< /dev/null' above is not needed ] 3.3 Other News Transport Agents The following is a non-inclusive list of other news transport software known to work under Linux: dynafeed, nntp1.5.11, slurp1.05 4.0 News Readers There is no 'one true newsreader'. As a result, there are many well-known newsreaders that port easily to Linux in particular. At this writing, 'tin', 'trn', and 'nn' are in the SLS distribution of Linux. When picking a newsreader, you basically want to find something that is easy to use, very configurable by the user, with threading and kill files (to select interesting articles or make the non-interesting ones not appear at all). You can set your paths to anything you like as long as all the newsreaders can find 'inews' from your Cnews or INN installation and a 'mail' program to send mail replies to posts. This section will talk briefly about several of the most popular ones. Before you ask, I use 'nn' for lots of reasons :-) 4.1 Tin Tin is a threaded newsreader generally intended to be easy for new users. It supports kill files and NOV threading. If you're running INN, it will read NOV .overview files by default and not write index files. To compile Tin under linux, basically just edit the makefile to set the locations of the software (especially the location of inews) and type 'make linux'. There are no patches required for tin under Linux. For threading, you can basically just say 'tin -u' to update the index files. To enable the ability to read via NNTP, compile with 'NNTP_ABLE' defined. This will result in a file called 'tin' for local and one called 'rtin' for NNTP reading. 'tin -r' will also get the same behavior as 'rtin'. Iain Lea recommends the following crontab entry and says that you need to do a 'make daemon' to make tind. # thread the database 35 * * * * /usenet/bin/tind -u 4.2 Trn/Mthreads trn is a threaded derivative of the 'rn' newsreader. trn3.3 has the nice ability to select either the 'mthreads' (trn's threading package) or NOV (threader from INN) threading. To compile it, just run Configure and take the calculated defaults. You might need to have lib4.4.1 and bash-1.13 (there is a beta now available on the various Linux archive sites) to successfully run Configure. You'll probably need both bash1.13 and libs4.4.1 to get the new Configure to run properly. The newspak-2.0.tar.z distribution on sunsite contains working config files for trn under Linux. It's probably unwise to try to edit a trn config.sh by hand unless you're doing something *VERY* simple like changing the paths to fit your tastes. If you do so, you'll need to run 'Configure -S' before you 'make depend', 'make', and 'make install'. Although 'Configure' fails generally under Linux with bash1.12, 'Configure -S' work fine so if you take the newspak config.sh as a starting point, you'll be very close. Compiling for NNTP reading is as simple as answering 'yes' when Configure asks you if you want to do so (assuming Configure runs ok on your system). A future release of newspak will include a config.sh for NNTP reading as well as the existing one for local reading for those of us who are still 'bash-impaired' :-) There are dozens of command line switches for trn to get all kinds of behavior. Read the 'trn' man page for details. I use a nice feature to set all the switches easily: - add all the settings in a file called ~/.trnrc - export TRNINIT="~/.trnrc" A future newspak will have my .trnrc file as an example. trn3.2 and above has support for NOV or mthreads threading that's user-selectable. Accordingly, I recommend building the software to allow both threading mechanisms (it's a question in Configure). To pick one at runtime, try 'trn -Zo' for NOV and 'trn -Zt' for mthreads. To build the mthreads database, do something like the following in the 'news' crontab: # thread the trn database 35 * * * * /usenet/bin/mthreads all 4.3 NN The newspak-2.0.tar.z distribution on sunsite contains working config files for nn6.4.18 that you can drop into place and type 'make' under Linux. When you're done compiling, you need to do the following: - run the 'inst' program to install things. (install everything) - initialize the database - fire up nnmaster See the nn docs for details. Compiling, configuring, and running nn under Linux is no different than running nn on any other *nix with the exception that you probably want to run nnmaster as a cron entry rather than as a daemon. If you run it as a daemon under linux, it doesn't tend to wake up properly (the net effect of running it from cron is the same anyway). Support of 'nn' is as simple as the following crontab entries: # run nnmaster to collect 'nn' stuff 25 * * * * /usenet/sw/nn/lib/nnmaster # expire the nn database 0 4 * * * /usenet/sw/nn/bin/nnadmin =EYW # stash a copy of the active file for 'nngoback' and keep last 7 0 3 * * * /usenet/sw/nn/lib/back_act 7 4.4 Other newsreaders The following is a non-inclusive list of newsreaders said to install and run under Linux: tass, xrn, gnus 5.0 Acknowledgements The following people have helped in the assembly of the information (and experience) that helped make this document possible: Ed Carp, Steve Robbins, Ian Taylor, Greg Naber, Matt Welsh, Iain Lea, Arjan de Vet If I forgot anybody, my apologies... #============================================================================= 6.0 Frequently Asked Questions about USENET (in c.o.l.* anyway...) 6.1 Why can't I post to moderated groups ? Probably because the newsreader is trying to call /bin/mail to send the mail and it doesn't like it. Replace the /bin/mail in SLS with the port of mailx-5.3b.tar.z from the pub/Linux/system/Mail dir on sunsite.unc.edu and make it mode 2755 and group mail (like 'elm') and you'll be all set. Another possibility is that you have a moderated newsgroup set up on your local system as not-moderated and somebody upstream is quietly deleting the article (some system's software, not a person). Make sure you run a 'checkgroups' every now and then when the checkgroups article rolls by in news.admin every few weeks. 6.2 Why do I have problems that appear to be permission-related ? Because until very recently, SLS didn't have a 'news' username or group in the default passwd/group files. All the stuff in /usr/local/lib/news should be news.news except /usr/local/lib/news/setnewsids which should be setuid root. You can use whatever UID and GID you want for 'news'. 6.3 Why can I post articles locally, but they don't get fed downstream ? Probably because you didn't call newsrun from cron. Maybe because you edited your sys file and messed it up. There are examples of each in the SLS /usr/local/lib/news directory. Maybe because you don't have a /usr/spool/news/out.going tree or something. Do not create newsgroups or feeds by editing the active or sys files. Use the utilities in /usr/local/lib/news/bin/maint to do it. 6.4 Why doesn't SLS have the 'Performance Release' of C-news? Because it's optional. Because it has problems under Linux in some places. Because the one in SLS is more stable and plenty fast enough on any type of reasonable system (IMHO). If you need any more convincing, here's a few excerpts from the C-news README.new file: "This is the 20 Feb 1993 Performance Release of C News, a minor update to the 2 May 1992 P.R. that just fixes some installation problems and a couple of annoying bugs"... "This release is more or less a halfway step to what we've been calling the "cleanup release"... "People who are happy with the older C News might want to wait for the cleanup release, which is still coming although behind schedule"... Mainly because at the time SLS1.01 came out, the compilers and libs didn't like the Performance Release too much. 6.5 Why doesn't SLS have nntp ? Because it's supposed to be plug-n-play under Linux and because I didn't want to make SLS's news stuff doubly big by having to maintain both nntp and non-nntp versions of the newsreaders and news transport programs. Also because it compiles in localized information that is not overridable at runtime via a config file. 6.6 Why does doexpire (or relaynews or...) say "severe space shortage" when there's lots of room ? Because it can't read /etc/mtab. Make it mode 644. This happens when you unmount a mounted filesystem by root with a umask that doesn't permit world-read of files owned by root. 6.7 Why does everything look normal, but posting doesn't happen ? Older versions of Linux had a 'broken' sed that C-news was prone to blow up. In particular, if you've installed over an old SLS, be sure to check /bin and /usr/bin to be certain you have only one copy of sed and that it's a modern one. 6.8 Can I hook the new news overview (NOV) stuff into trn/tin/nn to replace the various independent thread databases ? Yes. trn3.3 supports it now. The next version of tin will. You can hook it into the Performance Release of Cnews and various other readers as well... The beta copy of nn6.5 I have here has NOV support and it works just dandy under Linux. I run it here and have nn running without having to run nnmaster (!). Look on uniwa.uwa.edu for a copy of it. 6.9 Why can't we have a binary distribution of NNTP ? Because significant local-only information is compiled-in and cannot be determined auto-magically at runtime. 6.10 How do I set up NNTP to allow read/post across the network while *not* storing any news articles or databases locally. [... mdw@sunsite.unc.edu (Matt Welsh) ...] 1) Grab the 'reference implementation' of nntp and a copy or rn from your local archive site. If you connect to ftp.uu.net you'd grab: nntp.1.5.11.tar.Z from ftp.uu.net:/news/nntp rn-4.4.pl3.tar.Z from ftp.uu.net:/news/rn 2) compile nntp as follows: copy common/conf.h.dist to common/conf.h. Edit common/conf.h to set certain options: The only ones I set were: DOMAIN: undefine it (i.e. change the line to #undef DOMAIN). REALDOMAIN: Define this. It looks up the domain using the libraries. SERVER_FILE: Set this to the name of the file which will contain the hostname of the news server (i.e. the machine you'll read and post news through). I use "/usr/local/lib/news/server". PASSFILE: If your news server requires authorization (i.e. some kind of username/password) to post, set this to the name of the file which contains the username and password (described below). I use "/usr/local/lib/news/nntppass". I decided to keep all of the other news stuff in /usr/local/lib/news. So I set all of the rest of the pathnames in the file (i.e. ACTIVE_FILE, NEWSGROUPS_FILE, etc.) to use /usr/local/lib/news. Many of these files are only used by the NNTP server, not the client, but to be safe I changed them all to point to the right directory. You can of course use the default pathnames; just make sure you create the directory accordingly. 3) Create the user "usenet" if you haven't already. The inews program runs as this user. All you need is a userid; you don't need a home directory or shell or anything for the user. Just plop the following line into your /etc/passwd: usenet:*:13:1::/: Make sure you set the userid ("13", above) to something unique. The group can be anything; I use "daemon" (gid 1). 4) Create the SERVER_FILE, above. For example, my news server is "wonton.tc.cornell.edu", so I created the file /usr/local/lib/news/server which contained one line: wonton.tc.cornell.edu 5) Create the PASSFILE. This file contains lines of the form Let's say that your news server (the one in SERVER_FILE, above) is "shoop.vpizza.com", and to post on that machine you need to be authorized as the user "news" with a password of "floof". Thus, in the PASSFILE (I use /usr/local/lib/news/nntppass), you need the line shoop.vpizza.com news floof 6) Make this file secure! The inews program runs as the user "usenet", so make this news directory owned by that user and the nntppass file as well. chown usenet /usr/local/lib/news chmod 755 /usr/local/lib/news chown usenet /usr/local/lib/news/nntppass chmod 600 /usr/local/lib/news/nntppass So nobody else can read this file. No, the passwords in it are not encrypted. 7) Go back to the nntp.1.5.11 source directory; issue "make client". At this point you'll build the NNTP version of inews, which is the only software used by the NNTP client. When I built inews, there was a bug in the library which caused the function uname() in uname.c to call itself eternally. This should be gone now; however, if inews seems to hang and your system starts slowing down *a lot* you should rename the function "uname()" in uname.c to something like "my_uname()", and change the calls to it (in inews.c) to call my_uname() instead. Mail me if you run into this problem. [...VDS note - this means mail to Matt...not me :-) ...] 8) Issue "make install_client". This will install the inews stuff. Also make the link /usr/local/lib/news/inews -> /usr/local/bin/inews Now you should be able to happily post (by hand). Try something like the following: $ inews -h << EOF Newsgroups: misc.test From: me@foo.bar.com Subject: Testing Reply-To: my-real-address@wherever.edu This is a test. EOF If this works, inews should post the article. You'll know because test-responders on misc.test will reply to the address on the Reply-To line, above. Please don't do test postings on real groups, like c.o.l. :) 6.11 How do I build 'rrn' as a nntp-based newsreader/poster ? [... mdw@sunsite.unc.edu (Matt Welsh) ...] 1. cd to the rn source directory and do "./Configure". For almost all the questions, take the defaults (it's very smart). Make sure you have a link to "cpp" (usually in /usr/lib/gcc-lib/i386-linux/2.3.3) in /lib, so rn can find it. - Don't panic if it says you're on a NeXT or a USG system. Just go with the flow... - Answer "y" to "Do you expect to run these scripts and binaries on multiple machines?" - For your domain name: if your system is, say, "foo.bar.com", answer with just "bar.com" (the domain, not the entire hostname). - Answer "usenet" for the name of the user which is the news admin. - For "Where is your news library?" use the directory where the above NNTP files are kept: I use /usr/local/lib/news. - Man page source is in /usr/man/man1. - Answer "y" to "Does your /etc/passwd file keep full names in Berkeley/V7 format?" - Answer "gcc" to the name of the compiler which forces resolution. - Answer "y" to "Do you want to build the NNTP version of rn (rrn)?" - For the name of the news server file: use the same name you used when building inews, above (for me, /usr/local/lib/news/server). Don't run makedepend yet! Go to the next step: 2) Edit config.h. Change the "#define ROOTID ..." line to #define ROOTID 0 if it isn't already. 3) In rrn-4.4.pl3, there is a small bug with SIGEMT: it doesn't exist on Linux. rn used SIGEMT to check if a process is still running; however, under POSIX.1 we can instead just send the process a signal 0. Edit init.c. On line 243, note the following: if (kill(processnum, SIGEMT)) { Change this to if (kill(processnum, 0)) { You may wish to #ifdef it out instead; it's up to you. Don't just define SIGEMT to be zero; this will break the sigignore() call in final.c. 4) Issue "makedepend > makedepend.out". 5) Issue "make". Rn should compile happily. 6) Issue "make install". You're all set. Keep in mind that rn does in fact eat a lot of memory (as does any newsreader), handling the newsgroups file. So when you first run rn, it might be a little slow building your .newsrc and checking for new groups. If you're impatient, copy a .newsrc from somewhere else and use that.