ipac Linux 2.0 and 2.2 ip accounting package

ipac

Linux 2.0 and 2.2 ip accounting package

[version, download] [news] [faq] [mailing list] [readme file] [changes file] [home]

What is ipac / Overview

ipac is an ip accounting package for linux. It collects, summarizes and nicly displays ip accounting data. The output of ipac can be a simple ascii table, an ascii graph or even images with graphs, showing traffic progression. ipac can be used for ip traffic analysis and for accounting purposes. For further information, read the current README which follows. At the bottom of the page is the latest version of the file CHANGES.

Version, Download

The current version is 1.10 (2000/08/07)
You can download ipac (~ 60 KByte) via http from Sourceforge (which also has an archive of recent versions). Or from here.

New major features in version 1.10 since 1.05:

support for .png image generation with newer GD librarys (1.06)
smaller changes and new features (see CHANGES) (1.06)
rewrite of fetchipac in C and introducing of 'accuracy' to ipacsum (1.07)
several bug fixes in ipacsum and fetchipac (1.08, 1.09)

Update from version 1.06 or older is recommended; for ipchains users (Linux kernel 2.2.x) update from version 1.08 or older is recommended. If you have problems with ipacsum which are mentioned in the CHANGES file in section 1.09 or 1.10, update to 1.10.

There is a development release of ipac 2, too. Go to Sourceforge to find it.

News

ipac development stagnation (2002/01/23)

Unfortunatelly, ipac development has stagnated since the last release due to limited time I have for it. It is even hard to reach me by email on ipac related matters. I still hope that I will have time for it in the future again, but I don't know if so and when.

ipac does not support 2.4 kernels and I have no idea when it will. There is however a branch named ipac-ng; the author claims that it does support ipchains (kernel 2.4). However, I have nothing to do with that.

ipac 2 development: First development release (2000/07/14)

The first development release of ipac is out. Check the sourceforge page for details.

ipac 2 development (2000/06/06)

Currently, there is a next major version of ipac, 2, under development. ipac 2 will have a general "storage backend API" interface. Storage backends can be "plugged in" at build time. They have to provide certain functionality like opening their database, storing records, deleting records or listing timestamps. ipac 2 will come with two standard storage backends: The old "plain file" backend which is the one used in ipac 1.x, and a new "gdbm" backend which uses gdbm files to store data and which is much faster than the plain file backend.

I hope that once an ipac 2 (beta) version is released, people who are interested in this may start to develop more backends. In the first row, this might be SQL based backends like mysql. One of the main goals would be to centralize accounting information and to be able to add traffic from different routers within the same rule. Probably the biggest problem to solve for SQL backends would be to handle error conditions such as unreachable database servers.

The current state of ipac 2 is still alpha, but shortly it will be beta. The main task for the moment is to write documentation, especially for the storage backend API; as soon as this is done, I will release the beta of ipac 2 and put the cvs tree on Sourceforge. I expect this to happen within the next two or three weeks.

Sourceforge (2000/05/31)

ipac now uses Sourceforge (same with ssl) for development support. Sourceforge is a good place for patches (upload and download) and bug reports. In future, the Sourceforge cvs repository will be used, too.

Mailing list

There is a mailing list about ipac. The list is intended for announcements, discussions, suggestions, patches etc. - everything that relates to ipac. To subscribe to this mailing list, send mail to ipac-l-request@daneben.de with the word subscribe in the body of the message. The list is really low-traffic.

Link back to my home page

README

  IPAC Version 1.10
  (c) 1997 - 2000 Moritz Both
  For copyright notice see at the bottom of this file

WHAT IS IT?

ipac is a package which is designed to gather, summarize and nicely
output the IP accounting data. ipac make summaries and graphs as ascii
text and/or gif images with graphs.

ipac...
  - is for Linux
  - runs on top of the ipfwadm or ipchains tool
  - needs certain kernel parts compiled in


HOW DOES IT WORK?

  ipac consists of two scripts (shell and perl) and one C program:

  - ipacset reads a configuration file and sets up ip accounting
  for the kernel using ipfwadm or ipchains

  - fetchipac, executed from cron once in a while, reads
  the current ip accounting data assembled by the kernel
  and writes it to a new file

  - ipacsum summarizes the data from a set of files and, optionally,
  replaces these files by one. It displays the values as a simple
  table containing the sums, as png graph pictures or as ascii graph
  pictures.


UPDATE

If you are updating from an old version of ipac, read the file UPDATE.


2.1.* OR NEWER KERNEL

If you update your kernel from a 2.0 to 2.1 or newer version, beware that
you will suddenly need the ipchains (version 1.3.8 or newer), 
awk and mktemp tools. If these all
existed when you installed ipac, there is no need for re-installing; however,
that is the unlikely case. You should probably re-install it anyway to
be on the safe side.

Once ipac was installed with all the tools in place, you can switch kernels
as you like without re-installing. Put differently, if the programs
ipfwadm, ipchains, awk and mktemp are in the PATH when installing, 
you are fine for every kernel. (As long as it is Linux.)



INSTALLATION / PRECONDITIONS

ipac runs at least under Linux kernel 2.0.29, 2.0.33, 2.0.35, 2.2.3 and
2.2.5.  It should run on any kernel above. 

You need perl 5.

If you want to use ipacsum to create images, you need a perl library
called "GD". If GD is not installed and you run ipacsum to make images,
it will exit with an error. GD can be found at http://www.cpan.org/CPAN.html
- follow the link to the perl module list and look for GD. After downloading,
you must install GD as described within the package.

The type of images ipacsum makes depends on the version of the perl GD
library you have. If you use GD version 1.19 or older, you will be 
generating gif images. If you use GD version 1.20 or later, the image 
format will be png. png is preferred because there is no copyright / patent
hassle.


PRECONDITIONS FOR 2.0.* kernels

You need a kernel which was compiled with the configuration option 
CONFIG_IP_ACCT set to y. You also need the front end to ip firewall and
ip accounting, that is, the tool 'ipfwadm'. I used version 2.3.0.

PRECONDITIONS FOR 2.1.* and 2.2.* kernels

You need a kernel which was compiled with the configuration option 
CONFIG_IP_FIREWALL set to y. You also need the front end to ip firewall and
ip accounting, that is, the tool 'ipchains'. I used version 1.3.8, and
it was reported that older versions do not work.

Finally, you will need awk and mktemp.

* Beware: firewall packet filter scripts may interfere with ipacset when
* using 2.1.* or 2.2.* kernels! If you have a script that sets up a packet 
* filter, read the ipacset man page, section BUGS!



INSTALLATION / OVERVIEW

To install:
  - Edit the file 'config'.
  - Type 'make'.
  - As root, type 'make install'.
  - Create the file 'ipac.conf' and execute ipacset (see below).
  - Put fetchipac into cron (see below).
  - Put ipacset into a startup file to set ip accounting after reboots
    (see below)
  - Make sure that the accounting data files are cleaned up properly
    (see below).



INSTALLATION / CONFIG FILE, RUNNING IPACSET

The ipac.conf file is '/etc/ipac.conf' if you don't change this path
in config. ipac.conf controls what data is collected. Each line
which begins with a '#' is ignored. All the other lines have the format

  Name of rule|direction|interface|protocol|source|destination

  where
  Name of rule          Any string to identify this rule
  direction             'in' or 'out' or 'both'
  interface             ip number or interface name or empty
  protocol              'tcp' or 'udp' or 'icmp' or 'all'
  source                \
  destination           both as described in ipfwadm(8), or empty
 
In the summaries, the 'Name of rule' string identifies the counter.
Both the source and destination must be in ipfwadm syntax -
consult the man page.

The interface can be named (for example, eth0) or its IP number can be given.
Linux IP accounting always counts at one interface or at any interface. The
direction means in or out of this interface, or both directions.

For a more complete explanation of this file, see the man page of
ipacset(8).

* You must run the ipacset script after changing the 
* ipac.conf file every time for the changes to take effect!

An example ipac.conf file comes with the distribution.



INSTALLATION / FETCHING ACCOUNTING DATA FROM KERNEL: FETCHIPAC IN CRON

In order to collect the accounting data, you must put a line into a
crontab to call fetchipac on a regular basis. The more often you call 
fetchipac, the less data is lost in case of a crash or reboot. It 
is unharmful to call fetchipac any time. I suggest to call it every 
15 minutes.  For example, put this into your /etc/crontab file:

# Save IP accounting info every five minutes.
*/15 * * * * root /usr/local/bin/fetchipac



INSTALLATION / AFTER REBOOTS

Naturally, the kernel forgets about the ip accounting on reboots.
To reset the ip accounting properly, you should put a line into a
startup file to call ipacset. For example, in my /etc/rc.d/rc.local
file, I put this:

# Switch on ip accounting
/usr/local/bin/ipacset



READING IP ACCOUNTING SUMMARIES

To get summaries, use ipacsum. Without arguments, ipacsum will print a
sum for every rule in ipac.conf. It will evaluate every file it finds in
the ip accounting data directory, thus, all data ever gathered by 
fetchipac will be used.

ipacsum outputs a nicely formatted overview of all accounting rules
which were in effect during the given period. The rules are identified
by their names from the ipac.conf file. If a rule was added or
deleted during that time, it is nevertheless shown.

Other capabilities of ipacsum include generation of graph images, filter output
by rule name and setting time frame of files to be evaluated.

For a complete description of ipacsum, read the man page ipacsum(8).
A small help screen will be displayed with ipacsum --help.



CLEANING UP

fetchipac generates a single file every time it runs. The more often
fetchipac runs, the more files you get and the more exact will be
your accounting info. For example, if you run fetchipac every five
minutes, ipacsum will be able to display accurate data for every
five minute period.

Every time ipacsum runs, you can let it make a summary file for
all the files read to replace them. This will decrease the needed
disk space and the time ipacsum needs to calculate sums for this
period. You lose accuracy, though, since all data files are
summarized into one, meaning there will be no more information
when exactly the traffic occurred, but only the sum for the whole
period.

In general, it makes sense to periodically summarize the info
for a past period. For example, you could run these cron jobs
cleanups:

 - run fetchipac every 15 minutes
 - every hour, summarize the files of the hour 48 hours ago
 - every day, summarize the files of the day 7 days ago
 - every week, summarize the files of the week 11 weeks ago
 - every year, summarize the files of the year 2 years ago

With this scheme, you can have the data of the ip traffic with
15-minute-accuracy for the past two days. For the last week, you
still can tell at which day the traffic passed your machine. Keeping
the daily files for 14 weeks, you have a daily overview of the last
three months. After that, you keep weekly files only. After two years,
you sum up the data of the year into one file.

The daily cron jobs could look like this:

# Summarize ip accounting info:
# every day, sum up the data of 7 days ago into one file.
1 0 * * * root /usr/local/bin/ipacsum -r -t "the day 7 days ago" >/dev/null
# every hour, sum up the data of 48 hours ago into one file
2 * * * * root /usr/local/bin/ipacsum -r -t "the hour 48 hours ago" >/dev/null
# every week, sum up the data of the week 11 weeks ago into one file
3 0 * * 0 root /usr/local/bin/ipacsum -r -t "the week 11 weeks ago" >/dev/null
# every year, sum up the data of the year 2 years ago into one file
4 0 1 2 * root /usr/local/bin/ipacsum -r -t "the year 2 years ago" >/dev/null



FURTHER DOCUMENTATION

Read the man pages - ipacset(8), ipacsum(8) and fetchipac(8).



CONTRIBUTIONS

The directory contrib/ contains stuff that does not directly belong to 
ipac but is related. Further (well-documented and... "nice") 
contributions to the directory are welcome!

Thanks to all who contributed with patches, comments or suggestions!



UPDATES, BUG REPORTS, WHERE TO GET

For new versions of ipac, look at 
http://www.comlink.apc.org/~moritz/ipac.html

There is a mailing list about ipac, for discussion, patches, suggestions
and announcements. To subscribe, send mail to <ipac-l-request@daneben.de>
with the word "subscribe" (without the quotes) in the mail body.

If you find a bug, please send me a report or a diff. See at the
bottom of this file for the email address.


COPYRIGHT

  Copyright (C) 1997 - 2000 Moritz Both
 
  This program is free software; you can redistribute it and/or modify
  it under the terms of the GNU General Public License as published by
  the Free Software Foundation; either version 2 of the License, or
  (at your option) any later version.
 
  This program is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.
 
  You should have received a copy of the GNU General Public License
  along with this program; if not, write to the Free Software
  Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 
  The author can be reached via email: moritz@daneben.de, or by
  snail mail: Moritz Both, Im Moore 26, 30167 Hannover,
              Germany. Phone: +49-511-1610129

 

$Id: README,v 1.32 2000/06/06 10:01:51 moritz Exp $
EOF

CHANGES

$Id: CHANGES,v 1.50 2000/06/06 10:01:49 moritz Exp $

V0.9 - initial revision

V0.91:
Added time zone arithmetics
fixed bug in ipacsum: no input files and -r
added CHANGES file
corrected documentation

V0.93:
Improved -t time frame syntax
Added graph support
Added -f filter option

V0.94:
Fix: no longer produce readdir() warning
-s and -e can take relative times, too
minor fixes

V0.95:
Added official distribution site
Added -d option to ipacsum
Avoided warnings

V0.96:
- ipacsum:
        better handle corrupt data file content
- ipacset + fetchipac:
        be more robust about missing permissions or RUNFILE
        don't call fetchipac on initial call at boot time
        handle all lock file cases correctly

V0.97:
- subst.c: fixed compile error under libc6 / RedHat 5.1
- ipacsum: fixed (nearly) endless loop when no start time was given and -g

V0.98 (98/07/14):
- changed default lock file directory from /tmp to /var/lock (security)
- ipacset: discovered ipfwadm may add more than one line to the kernel
  tables with one call - this happens when a host name resolves to
  more than one ip address. As a result, all gathered data were 
  messed up. Fixed ipacset to handle this properly.
These changes to ipacset were done by Friedrich Lobenstock 
<fl@fl.priv.at> [FL], thanks!:
  - Fixed a bug if there exist empty lines in /etc/ipac.conf
  - Added some basic syntax checking (empty parameters)
  - Added command line option '-D' for debugging
  - Not really added support for bidirectional rules.
  Just say 'io'(or 'both' as in ipfwadm(8)) instead of
  'in' or 'out' in /etc/ipac.conf
- README: Added hint to mailing list
- support for long option names

V0.99 (98/07/29)
- added man pages
- corrected many spelling errors (discovered ispell). Changed "--intervall"
  option of ipacsum to "--interval" :-). (Old version still works)
- added gif image creation (all options beginning with --gif)
- added --version

V1.00 
- improved ipacset error handling
- fixed order of week days in ipacsum
- added average value feature for gif pictures
- --gif-caption-in-index / show max and average values in html index file
- --show-run-progression / display a progression indicator while running
- added "Resolution:" in index.html
- support for empty interface spec in ipac.conf for "any interface"
- --gif-asis, HTTP Expires: header lines and HTML META tags
- can now define two or more rules with the same name. ipac shows the
  sum in its outputs

V1.01
- fixed 'Illegal division by zero at /usr/local/bin/ipacsum line 763.'
- minor sub-pixel fix in gif creation (probably never visible)

V1.02 (1999/04/14)
- ipacsum: --gif-average-dots
- Linux 2.1.* and up kernel / ipchains compatibility
- --gif-caption-in-index shows total traffic (for whole gif image)

V1.03 (1999/05/10)
- fixed bugs in ipacset and fetchipac which appeared with some shells
- added handling of cases when something deletes some or all of ipac's
  settings in the kernel (ipacset --fix-chains, fetchipac corrupted
  kernel settings detection, "BUGS" section in ipacset(8))

V1.04 (1999/05/31)
- bug fix concerning ipchains option --fix-chains (still had problems with
  firewall scripts)
- bug fix for /bin/sh compatibility in ipacset

V1.05 (1999/08/06)
- bug fix ipacsum "this week", "last week", "the week n weeks ago"
- bug fix no "in" rule resulting in "out" rules not counting

V1.06 (2000/01/24)
- added support for .png images while being backward compatible to old .gif
  images... if you have the GD library. Renamed all options '--gif-*'
  to '--png-*' in the documentation; the old versions still work.
- --gif-average-curve now "smoother"
- --fixed-quantity
- --png-total
- made pathsearch more compatible (hints from Thomas <thomasz@hostmaster.org>)
- added contrib/ directory with ipac.cgi by Dan Swan <swan_daniel@hotmail.com>

V1.07 (2000/02/25)
- added debug code to ipacsum which can be stripped off on install by an
  awk script in Makefile
- introduced 'accuracy' to ipac. ipacsum's output is now The Correct Numbers,
  I believe. All older versions didn't have a real concept about 
  calculating the correct numbers at the beginning and the end of a
  given time frame, resulting in incorrect values at the edges of a
  given time frame. 
  
  Specifically, ipac would take each data file it found in the data file
  directory which was created within the given time frame; if there was
  no data file, it would take no data file; and sum them all up. If you
  had one data file for the whole day, created at 23:58 h, it would give
  you a good result if you asked about the whole day, but asked about
  0:00 h (start time) to 23:50 h (end time), it wouldn't find any data
  and display nothing. Asked about 23:50 (start time) to 23:59 (end
  time), it would give you the whole day's data.

  As of this version, ipacsum really evaluates the correct values by
  considering all data from the given time frame plus the data from
  directly before and after the time frame. If the data for a given time
  frame is not specific enough, ipacsum computes an average value. Given
  the example above, asked about the whole day's data, ipacsum would
  consider the data from 23:58, but also the time of the data before this
  day (possibly with a time stamp of the day before, 23:58), and the data
  of the next file. So the values would be (a little) more accurate.
  Asked about the time until 23:50, the values would probably decrease a
  little. And asked about the time from 23:50 to 23:59, it also would
  only display this nine minutes' share of the collected values.

  Also, --replace now works in a conceptual
  meaningful way. The generated file's name is now derived from
  the time at the end of the time frame (before: from the newest file
  that was read).

- rewrote fetchipac in C. It is now more robust, more verbose in case of
  errors and it *could* be extended to store its data into a database.
  (Also, it creates much smaller data files and uses much less 
  system resources. And it does no longer use programs like
  ipchains, ipfwadm, awk and so on.)

V1.08 (2000/04/29)
- ipacsum bug fix --fixed-quantity and values >2^31 (showed negative numbers)
- ipacsum bug fix dont try to read data files which are directories or which
  have an invalid file name
- ipacsum bug fix dont reproduce old, deleted rules and old rule order on 
  --replace runs. dont create empty meaningless files on --replace runs
  which may confuse ipacsum, saying "* = data incomplete..."
- ipacsum bug fix --gif-total (option wasn't recognized)
- ipacsum add --filename-prefix option
- added 'ipac_and_mrtg' to contrib directory

V1.09 (2000/06/06)
- fetchipac: bug fix keep rule order in ipchains systems in any case
- ipacsum: bug fix: don't make time frame in images longer than it should be
  in infrequent cases
- ipacsum: bug fix: (again) dont make first value in images too high in rare
  cases

V1.10 (2000/08/07)
- added Suse rpm directory under contrib/
- ipacsum: --png-header-in-index option
- ipacsum: corrected documentation and behavior concerning --png-index
- ipacsum: --png-start, --png-end, --png-charset
- ipacsum: x axis labels are now aligned to month and year borders
- ipacsum: bug fix illegal division at line 1206/1185

Neuigkeiten der Nuller Jahre

Tags: linux, ipac