Version 0.3pre1 --------------- Detailed measuring system for wide range of IPs ----------------------------------------------- "fast" setup ------------ 1. copy this sample in /var/lib/FWcnt/etc/config (replace 192.168.0.0 with your IP) ---------------- cut ---------------- cntLINK eth0 cntNET 192.168.0.0 ---------------- cut ---------------- 2. run /usr/lib/FWcnt/init_FWcnt 3. run /usr/lib/FWcnt/generate 4. show http://YOUR_WEB/cgi-bin/mrtg-rrd.wrapper.cgi Part 1: ******* Kernel counters in iptables is used for counting paket and byte count od each measurments. Specification & features ------------------------- 1. each IP has own In and Out counter 2. each subnet of 8,16,32,64,128,256 IP has own In and Out counter 3. paket traverse only minimum rules in firewall max rules = 1(C range)+2(Ip/25)+4(Ip/26)+8(Ip/27)+ 16(subchains = Ip/28)+2(Ip/29)+16 = 49 rules one direction For reverse direction one more rule must be added. Full C bidirectional counting is counted by max 50 rules For each C class one rule more must be traversed for one direction for each C range exists this structure: match full ---+ rule subnet 0-127 C + rule subnet 128-255 range + rule subnet 0 - 63 + rule subnet 64-127 + rule subnet 128-191 + rule subnet 192-255 + rule subnet 0-31 + rule subnet 32-63 + . + . + rule subnet 224-255 + subchain 0-15 ----+ rule subnet 0-7 | + rule subnet 8-15 | + rule 0 | + rule 1 | + . | + . | + rule 15 + subchain 16-32 ----+ rule subnet 16-23 | + rule subnet 24-31 | + rule 16 | + rule 17 | + . | + . | + rule 31 . . + subchain 240-255 ----+ rule subnet 240-247 + rule subnet 248-255 + rule 240 + rule 241 + . + . + rule 255 Chain FWcnt is managed by programs init_FWcnt, get_FWcnt and clean_FWcnt. clean_FWcnt erase all rules and subchains in FWcnt chain get_FWcnt read counters from rules in FWcnt chain init_FWcnt set up rules for one C ranges of IPs for measurments Example: to set up measurments for one C range 192.168.0.0 init_FWcnt 192.168.0 Without argumet this command read configuration file and set up counters for all nets defined by keywork "cntNET" in config file (see below). Initialization in FW: iptables -N FWcnt #base chain for measurments iptables -I INPUT -j FWcnt #count input iptables -I OUTPUT -j FWcnt #count output iptables -I FORWARD -j FWcnt #count forward getting counters: get_FWcnt return 4 values per line .. paket count, bytes count, source IP destination IP Part 2 ****** counter database ---------------- Text database used by mrtg is slow. RRD database is used for each counter rule. cron job read all counters in FWcnt chain by calling "get_FWcnt". Output of "get_FWcnt" is passed to binary fw2rrd (from source fw2rrd.c) For each counter from FWcnt chain reverse counter is searched. If database file does not exist, then fw2rrd create one for each counter. fw2rrd must be running in working directory. Only iptables command must run with root privileges, database can be run by different user. example cron job: 0-55/5 * * * * root if [ -x /usr/lib/FWcnt/get_FWcnt ]; then \ (env LANG=C /usr/lib/FWcnt/get_FWcnt |su --shell=/bin/bash \ -c '(cd /var/lib/FWcnt/FW_rrd_database/; \ /usr/lib/FWcnt/fw2rrd' Debian-ippl);fi Counter name is derived vrom IP number. For each counter pair exists two rrd files, one for bytes count and one for pakets count. Name convencion: IP1.IP2.IP3.IP4_NM-b.rrd bytes for net IP1.IP2.IP3.IP4_NM IP1.IP2.IP3.IP4_NM-p.rrd pakets for net IP1.IP2.IP3.IP4_NM IP1..IP4 number 0..255 NM netmask bits (0 ..32) -p.rrd fixed part of name for pakets counter -b.rrd fixed part of name for bytes counter "Target" name of counter is filename without .rrd suffix Warning, "_NM" is optional, for measuring one IP "_NM" can be "_32" or this part of name absent. example: 192.168.0.1-b.rrd byte counter for one IP 192.168.0.1 192.168.0.1_32-b.rrd byte counter for one IP 192.168.0.1 but !!WARNING!! this counters is different from previous 192.168.0.0_16-p.rrd pakets counter for "B" class net 192.168.0.0 Part3 ***** ethernet device measurment -------------------------- data for each ethernet device can be found in /proc/net/dev. Not all devices is "counted". Only devices found in config file is used for measurment. FWcnt2rrd first run fw2rrd, next check configuration file and for each device create/update counter file. example config file (onli part for devices) --------------- cut -------------- cntLINK eth0 cntLINK vlan2 --------------- cut -------------- counter files with names: eth0-p.rrd eth0-b.rrd vlan2-p.rrd vlan2-b.rrd Visualization of counters by mrtg. ---------------------------------- mrtg-rrd is used to visualize rrd files. Original mrtg-rrd does not use multiple config files, wrapper is created for adapt mrtg-rrd to count C class networks. All needed config files for mrtg-rrd is automaticali generated by "generate" command. "generate" first creates one page with graphs for ethernet devices (based on config file) and next graph for summary of one C class. below graph for C class is subsummary graphs for /28 subnets (default) or subnets of C class defined in config file by directive "mrtgNET". Both variant of netmask can be used. Example config: ---------------------------- cut ------------------------- #eth interfaces cntLINK eth2 cntLINK eth0 # firewall counters for C class nets cntNET 192.168.40 cntNET 192.168.42.0 # mrtgNET 192.168.40.0 24 mrtgNET 192.168.42.0 25 mrtgNET 192.168.42.128 28 mrtgNET 192.168.42.144 29 mrtgNET 192.168.42.152 29 mrtgNET 192.168.42.160 27 mrtgNET 192.168.42.195 255.255.255.192 ---------------------------- cut ------------------------- Keywork "cntNET" is used for definition of counter for "init_FWcnt" command. obsollete files from previous development. ------------------------------------------ cnt2rrd [filename] read filename or stdin if no filename is specified. Format of input file is designed for iptables -L output but target must be RETURN. Example input file: ----------------------- cut --------------------------- Chain CNTin (1 references) pkts bytes target prot opt in out source destination 302978 50655932 RETURN all -- * * 0.0.0.0/0 62.168.101.165 Chain CNTout (1 references) pkts bytes target prot opt in out source destination 2 211 RETURN all -- * * 62.168.101.165 0.0.0.0/0 ----------------------- cut --------------------------- more chains can be concatenated in input and chains may consist of couple of rules for input and output. As "download" must be source set to 0.0.0.0/0. for upload must be destination set to 0.0.0.0/0. Protocol must be "all", opt must be "--" and in and out must be "-". For each couple of rule (one for down and one for upload) two rrd files are updated. Name od rrd file is derived from IP (or ip and netmask). For IP 62.168.101.165 for example is output rrd filename 62.168.101.165a.rrd and 62.168.101.165b.rrd. First file is updated by bytes counters from chain and seconf from pakets counter. If IP if rule match subnet (etc 172.26.0.0/24) file is named 172.26.0.0_24a.rrd (all "/" is replaced by "_"). Special IP ranges etc may cause problems.. WARNING ! rrd files creation is done in working directory.