twitter |   ||  email | PGP --> harshal @ harshdevx.comharshdevx:~#
harshdevX:~#r00t  |  analyt1cs   |  st1lls  |  ab0ut m3
updating...
generating stats with your firewall | part - 1
sometime back last year i setup my firewall. i was a fan of dd-wrt but then moved to a machine that can do the same...a linux machine with two nics. set up iptables that was the first time i ever worked on the it. to be honest initially i thought iptables are nightmare and i really wanna stay away from them. but gradually i started getting a sense of it making way for the packets. as my mentor rightly says think like a packet.

so first things first you need a machine with two nics, operating system of choice mine is linux (headless preferably = less overheads). the first nic behaves as lan and the second as wan interface. lets dive in. the first piece of code below will basically collect stats from your kernel log every hour (manually insert this as a cron job to run hourly). what it does is it creates a file as scanip_new.log from kernel log for current time "minus" 1. then we use linux diff tool to generate difference between the new log and old log. the difference is recorded in a temp file scanip_diff.log. this file either you can process it on the same machine or push it to a different machine to start counting the scans. now note that there may be a better way to do this so please and please do let me know and i will be more than happy to correct myself... we begin with collecting the kernel log.

#1 FROM=`date +%H -d "-1 hour"`
#2 if [ ! -f /tmp/scanip_old.log ]; then touch /tmp/scanip_old.log; fi
#3 cat /var/log/kern.log > /tmp/scanip_new.log
#4 diff --suppress-common-lines /tmp/scanip_new.log /tmp/scanip_old.log | strings -n 30 | cut -c 3- > /tmp/scanip_diff.log
#5 cat /tmp/scanip_diff.log | grep -E "SCANNER-IN" | gawk '/'"$FROM"':[0-59]/ && !/:'"$FROM"':[0-59]/' > /tmp/scanip.log
#6 #add lines 7-8 if you are doing the processing on different machines
#7 #----------------------------------------------------------------------
#8 scp -i /root/keyfile /tmp/scanip.log username@hostname://home/username/your-working-folder/
#9 ssh -i /root/keyfile username@hostname "/home/username/your-working-folder/sync_scanips.sh"
#10 #----------------------------------------------------------------------
#11 mv /tmp/scanip_new.log /tmp/scanip_old.log
now lets create the database.

#1 sudo apt-get install php5-sqlite sqlite3 geoip-bin
#2 touch analytics.db
#3 BEGIN TRANSACTION;
#4 CREATE TABLE "service" (
#5 `servicename` TEXT CHECK(50),
#6 `serviceport` INTEGER,
#7 `proto` TEXT
#8 );
#9 CREATE TABLE `scancount` (
#10 `date` TEXT,
#11 `count` INTEGER CHECK(11)
#12 );
#13 CREATE TABLE `kernellog` (
#14 `date` TEXT,
#15 `time` TEXT,
#16 `interface` TEXT CHECK(10),
#17 `src` TEXT CHECK(15),
#18 `dstno` INTEGER CHECK(11),
#19 `len` INTEGER CHECK(11),
#20 `ttl` INTEGER CHECK(11),
#21 `proto` TEXT CHECK(50),
#22 `sport` INTEGER,
#23 `dport` INTEGER,
#24 `flag` INTEGER
#25 );
#26 CREATE TABLE "geoiplookup" (
#27 `src` TEXT CHECK(15),
#28 `country` TEXT CHECK(50),
#29 PRIMARY KEY(src)
#30 );
#31 CREATE TABLE "dstip" (
#32 `dstno` TEXT,
#33 `dst` TEXT CHECK(15),
#34 PRIMARY KEY(dstno)
#35 );
#36 COMMIT;
download shell script and save as .sh | download schema
Part -2

concept and design: harshdevX