0

Episode 130 – One Dead Cow

Posted by Tony on July 13, 2014 in Show-mp3, Show-ogg |
Play

http://smlr.us

Downloads:

MP3 format (for Freedom Haters!)
OGG format (for Freedom Lovers!)
Total Running Time: 1:31:35

Un-edited Live session – http://youtu.be/wo_auoFACec

Contact Us:

show (at) smlr.us or the Contact us page

Summary

Kernel News: Mat
Time:
Distro Talk: Tony
Time:
Tech News:
Time:
Toolbox
Time:
Listener Feedback
Time:
Outtro Music
Time:


Intro:

Tony Bemus, Mat Enders, and Mary Tomich
Sound bites by Mike Tanner

Kernel News: Mat

Time:
mainline: 3.16-rc4
stable: 3.15.5 2014-07-09
longterm: 3.14.12 2014-07-09
longterm: 3.12.24 2014-07-04
longterm: 3.10.48 2014-07-09
longterm: 3.4.98 2014-07-09
longterm: 3.2.61 2014-07-11


Distro Talk: Tony

Time:

Distrowatch.com

  • 7-1 – antiX 14.2 “MX”
  • 7-1 – Zentyal 3.5
  • 7-2 – Clonezilla Live 2.2.3-25
  • 7-2 – Calculate Linux 13.19
  • 7-6 – Ultimate Edition 4.2
  • 7-6 – Deepin 2014
  • 7-6 – 4MLinux 9.0
  • 7-7 – CentOS 7.0-1406
  • 7-8 – SparkyLinux 3.4 “GameOver”
  • 7-8 – IPFire 2.15 Core 79
  • 7-9 – Kwort Linux 4.1
  • 7-12 – Chitwanix OS 1.5

Distro of the Week: Tony

  1. Ubuntu – 1197
  2. Deepin – 1273
  3. Mageia – 1290
  4. Mint – 1757
  5. CentOS – 1912

Tech News:

Time:
Elementary OS renames its latest release (It was ISIS)

http://www.unixmen.com/elementary-os-isis-now-freya

Systemback – a system restore tool for *buntu.

http://www.unixmen.com/systemback-restore-linux-system-previous-state

A nice way to find domain, etc information
http://www.tcpiputils.com

Visited Linux Journal, TOR web site, or Tails web site? The NSA has your number…maybe.

http://www.eweek.com/security/linux-lands-on-nsa-watch-list.html

Distrowatch Gone…and back.
http://www.itworld.com/open-source/426012/distrowatch-domain-registrar-problems-alarm-linux-users

KDE Korner

New Version of Kanagram has been released!

http://debjitmondal.blogspot.com/2014/07/brand-new-kanagram.html

First beta of KDE 4.14 has been released. The release schedule is here

 

https://techbase.kde.org/Schedules/KDE4/4.14_Release_Schedule

 


The Toolbox

Time:

Analyzing Apache Log Files
or
grep blah transfer.log |grep more_blah

1. Log format

We assume an Apache transfer log format where each entry in the log file contains the following information:

%h %l %u %t “%r” %>s %b “%{Referer}i” “%{User-agent}i”

where:
%h = IP address of the client (remote host) which made the request
%l = RFC 1413 identity of the client
%u = userid of the person requesting the document
%t = Time that the server finished processing the request
%r = Request line from the client in double quotes
%>s = Status code that the server sends back to the client
%b = Size of the object returned to the client

The final two items: Referer and User-agent give details on where the request originated and what type of agent made the request.

Sample log entries:

nnn.nnn.nnn.nnn - - [12/Jul/2014:05:06:18 -0400] "GET /blog/?tag=decor-craft HTTP/1.1" 200 27811 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
nnn.nnn.nnn.nnn - - [12/Jul/2014:05:06:36 -0400] "GET /index.php?main_page=index HTTP/1.1" 200 27273 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

2. Danger signs to look for

First we are going to look for the danger signs. What usually brings me to a server to check is a high load warning for the server. I look to see how many hits the sites on the server have received recently.

What you want to do is grep out about an hours worth of time and only show logs that were hit more than a given value.

Time Frame: 2014:0(4:[3-5][0-9]|5:[0-2][0-9]) Show Logs Hit More Than: 999
/home/blah1/var/blah1.com/logs/transfer.log: 5309
/home/blah2/var/blah2.mx/logs/transfer.log: 14451


{
printf "Time Frame Can Use A RegEx Like These:\n"
printf "2014:05:.. or 2014:0[4-5]:.. or 2014:0(4:[345][0-9]|5:(0[0-9]|10))\n"
read -ep "Time Frame: " TF
read -ep "Number you want to see hits greater than e.g. 99: " HO
printf "\n\n"
printf "Time Frame: $TF\tShow Logs Hit More Than: $HO\n"
for x in $(find /home/*/var/*/logs/ -name transfer.log)
do
printf "$x: "
zgrep -Ec $TF $x
done |awk "\$2>$HO" |awk '{printf "%60s %s\n", $1,$2}'
printf "\n"
}

3. lets see the IPs

Now we grep the IPs that have hit the site over a certain amount during the same time frame:

/home/blah1/var/blah1.com/logs/transfer.log
IPs WITH HITS > 999
IP ADDRESS #OF HITS
50.116.50.133 2907

TOTAL HITS FOR 2014:0(4:[3-5][0-9]|5:[0-2][0-9])
5309


{
read -ep "Hits Over: " HO
printf "Time Frame Can Use A RegEx Like These:\n"
printf "2014:05:.. or 2014:0[4-5]:.. or 2014:0(4:[345][0-9]|5:(0[0-9]|10))\n"
read -ep "Time Frame: " TF
read -ep "Transfer Log: " TL
printf "\n\n"
printf "$TL\n"
printf "IPs WITH HITS > $HO\n"
zgrep -E "$TF" $TL |
awk '{print $1}' |
sort |
uniq -c |
sort -nr -k1 |
tee hits |
awk 'BEGIN{print "IP ADDRESS #OF HITS"}{printf "%-17s %-s\n", $2, $1}' |
awk "\$2>$HO"
printf "\nTOTAL HITS FOR $TF\n"
awk 'BEGIN{T=0}{T=T+$1}END{print T}' hits
rm hits
printf "\n\n"
}

4. See what it is doing

Now we want to actually look at the actual transfer log entries. That however is going to be to many so we will look at a sampling.


IP To Check: 50.116.50.133
Transfer Log: /home/blah1/var/blah1.com/logs/transfer.log

50.116.50.133 - - [12/Jul/2014:04:37:48 -0400] "POST /wp-login.php HTTP/1.0" 200 3971 "-" "-"
50.116.50.133 - - [12/Jul/2014:04:37:48 -0400] "POST /wp-login.php HTTP/1.0" 200 3971 "-" "-"
50.116.50.133 - - [12/Jul/2014:05:15:19 -0400] "POST /wp-login.php HTTP/1.0" 200 3971 "-" "-"
50.116.50.133 - - [12/Jul/2014:05:15:19 -0400] "POST /wp-login.php HTTP/1.0" 200 3971 "-" "-"


{
printf "\n"
read -ep "IP To Check: " IPC
read -ep "Transfer Log: " TL
printf "\n"
zgrep $IPC $TL > hits
head -2 hits
tail -2 hits
rm hits
printf "\n\n"
}

Then we block those nefarious IPs in our firewall. This was WordPress specific but I have others that check for brute forcing a Magento or Expressionengine sites, and also ones that check for comment spam in forums.


Listener Feedback:

show (at) smlr.us or 313-626-9140
Time:


Outtro Music

Time:
Track 29 – Ghosts I-IV – Nine Inch Nails – http://archive.org/details/nineinchnails_ghosts_I_IV

This content is published under the Attribution-Noncommercial-Share Alike 3.0 Unported license.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Copyright © 2011-2014 Sunday Morning Linux Review All rights reserved.
This site is using the Desk Mess Mirrored theme, v2.2.4.1, from BuyNowShop.com.