Derek Neely

After some reading of consolidation options for Apache logs, I ran across mod_log_sql (we are hating spread) which will take Apache logs and log them off to a MySQL database. Sounded great! We could then run scripts to go through and parse the values and run statistics on. Twas perfect for our needs. RIGHT!!!

At my company we run about 12 vhosts over about 5-6 load balanced web servers. All of them were configured to log to our loganalysis server which is a pretty beefy machine. The logging all in all worked well with a few major exceptions listed below.

1) I used the directive:

LogSQLRequestIgnore .gif .jpg .css .ico .png .js

This directive is supposed to be used to ignore any pages ending with that extension. This did not work at all and I had to create a script to actually delete those before analyzing the logs. Bummer but not that big of a deal.

2) While working with the server to optimize the database, there were various times when I would need to restart the MySQL service and a few times I needed to reboot the server. During these periods of time, the web servers were unable to log to the database which brought them to their knees. The inability of the module to handle a database outage gracefully was a major deal breaker for us. I feel this issue is a result of intense disk IO when the database is down. The server is logging to its Apache logs, the the backup SQL logs, and to the Apache error logs for every failed request. This becomes emense with thousands of requests per second. Should this server die or needing maintenance would have ultimately brought our company to a hault.

Long story short, we’re scrapping mod_log_sql and going with an NFS mount out to all the web services which we can then parse and run statistics on using some custom scripts and/or AWStats or Splunk.

While I enjoy using the SSH Tunnels to encrypt traffic out of the random local networks that I may be sitting on, sometimes there is a need to establish a more permanent proxy server. For instance, if you don’t have a means to use an SSH client or you have several machines that you would like to service without the need to establish dedicated SSH tunnels/forwards for each machine (administrative nightmare).

I chose Squid for just this function. It was extremely easy to setup and has worked like a charm! Below are some of the basic steps for setting up and using your Squid proxy.

Install Squid:

Gentoo:
#> emerge squid

Red Hat/CentOS:
#> yum install squid

Ubuntu/Debian:
#> apt-get install squid

OpenBSD:
#> export PKG_PATH=ftp://ftp.openbsd.org/pub/OpenBSD/4.4/packages/i386/
#> pkg_add squid

Basic Configuration for Transparent Proxy:

#> vi /etc/squid/squid.conf

Look for the sections listed below and modify accordingly. This is a sample of my ACLs and configurations outside of some of the defaults.

#/etc/squid/squid.conf
acl all src 0.0.0.0/0.0.0.0
acl trusted_hosts 192.168.1.0/255.255.255.0 10.0.1.0/255.255.255.0
acl localhost src 127.0.0.1/255.255.255.255
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 443         # https
acl Safe_ports port 21         # ftp
acl CONNECT method CONNECT

# Deny requests to unknown ports
http_access deny !Safe_ports
# Deny CONNECT to other than SSL ports
http_access deny CONNECT !SSL_ports

http_access allow localhost
http_access allow trusted_hosts
http_access deny all

# Listening port (default 3128)
http_port 3128

# Visible Hostname (may not be needed but doesn’t hurt)
visible_hostname yourserverhostname

This is the basic configuration that should get you going. More advanced configuration options will be coming soon.

Start up Squid:

#> /etc/init.d/squid start

Set Squid to start on-boot (optional)

Gentoo:
#> rc-update add squid default

Red Hat/CentOS:
#> chkconfig squid on

Ubuntu/Debian:
#> update-rc.d squid defaults

OpenBSD:
#> vi /etc/rc.local
Code coming soon…

Configure your Browser:
This is an example Firefox setup. Preferences –> Advanced –> Network –> Connection –> Settings. Select ‘Manual proxy configuration:’ Set the HTTP Proxy: value to your Squid server’s IP (public or private depending on how you are going to use it and based on the ACLs above). Then select ‘Use this proxy server for all protocols’ if you want to go ahead and use this same proxy for all connections.  Setup any exceptions to not proxy (i.e. – locally connected servers). Here’s a screen shot of my configuration.

SSL Configuration:
Coming soon…

Advanced Configurations:
Coming soon…

DerekNeely.com needed an image gallery. Quick search result turned up Plogger. Thus far it’s been a really nice light-weight image gallery.

Not much there ATM but feel free to stop by! DerekNeely.com – Image Gallery

With my recent  broken jaw, I’ve become a fan of smoothies. I’ve become a regular at Tropical Smoothie Cafe. I’m at the point where I’m just starting at the top and working my way down. Well, after 6 or so visits this weekend I decided tonight I need to learn how to make my own.

Round 1:
Found some frozen blackberries in the freezer (thanks buddy Dave). Tossed a little bit of ice, some of the blackberries, and some yogurt into the blender. Had a tough time getting it to blend at first, hence the broken spoon pictured below. I then realized you needed a ‘liquifier’. I only had OJ in the fridge so I tossed some of that in there. Smoothie whipped right up. I believe I used too much yogurt though.

Estimated Recipe:

~1/2-3/4 cups ice
~1-1.5 cups blackberries
~1/2 cup OJ
~1/2 cup yogurt (gonna try less next time)

Round 2:
Gave it another shot today and think I came up  with a good one. I was making double recipe this time. One for the roomie.

2 bananas
9oz strawberries
2 cups pineapple juice
2 large tbsp. of low-fat yogurt
1 cup ice (add more for icier – obviously)

Round 3:
I think I have it. Well, this is just another recipe of one I’ve made but, it’s been the best thus far. Once again, I was making for 2 (~1 pint each – prob. a bit more).

2 bananas
1 cup blue berries
1.5 cups pineapple juice
1.5 cups ice
2 large tablespoons of low-fat yogurt

Round 4:
The best smoothie yet has been the strawberry, banana, chocolate smoothie. Holy hell its good. Recipe below. Made for 1.

1 banana
5 or so strawberries
1 cup milk
1 cup ice
2 heaping tablespoons of Nesquik (more or less to taste)

One of my small peeves with a Mac is the lack of a quick utility or short-cut to lock your computer. So, I took it upon myself to make one out of the apps they already give us.

First off set your Mac to require a password when waking from the screen saver and disable  automatic logins. Apple –> System Preferences –> Security –> ‘Require password to wake this computer from sleep or screen saver’ & ‘Disable automatic login’.

You can download the one I created that ‘should’ work automatically after installing. If not or you want to get creative and make your own with your own icon or what have you, the instructions are below on what I did. You can also set a keyboard shortcut to the app for a keyboard style lock.

Pre-made Screen Saver/Lock App:
Download: MacDLock (MacDLock.tar – 280KB)
Installation:

$> tar xvf MacDLock.tar
$> mv MacDLock.app /Applications

That’s it! Now if you open your Applications folder you should be able to click the Lock icon for MacDLock and it will launch your screensaver. Upon wake, you will be prompted for you username and password that you set up to do earlier. I put a launch icon on the launch bar and setup a shortcut to the application to make for quick screen locking while I’m away. Enjoy!

Custom Screen Saver/Lock:

$> cp -r /System/Library/Frameworks/ScreenSaver.framework/Versions/A/\
Resources/ScreenSaverEngine.app /Applications/MacDLock.app

This will give you the standard ScreenSaver but located in your /Applications directory. You can also do it with a soft link such as this:

$> ln -s
/System/Library/Frameworks/ScreenSaver.framework/Versions/A/\
Resources/ScreenSaverEngine.app /Applications/MacDLock.app

I did the first because I wanted to change out the icon to something more ‘cool’ or ‘secure’ looking (like a lock) without changing the actual ScreenSaverEngine.app.

Changing out the Icon:

1. Find the icon you’d like to use in either .png, .gif, .jpg format.
2. I used this site to convert my image to an icon: iConvert
3. Download your new .icns file.
4. Copy your .icns file to the application directory.

$> cp ~/[Icon_FileName].icns /Applications/MacDLock.app/\
Contents/Resources/ScreenSaverEngine.icns

And there you go! You have your custom Screen Saver/Desktop Lock. If you notice any kind of bug in my above code please let me know. I took many other steps while originally doing this so hope they are in the right order and I’m not missing anything.

One of my projects when I have some free time is to setup a Python script or something that embeds an icon into the Menu Bar for quick locking. If this is already available or you feel I’ve recreated the wheel please let me know of the other apps that are out there that may already do this. Always interested in seeing what other have done. Thanks! And hope you enjoy!

SSH Proxying is one of my every day tools. Sitting at work with a Barracuda firewall looking, snooping, and possibly blocking everything that I do. Hanging at a coffee shop when you see a suspicious person most likely snooping your information out of the air. In the first case I’m primarily just trying to get around a hurdle. In both cases I want my traffic encrypted and hidden from 3rd parties.

What is SSH Proxying?
This is a means of setting up a Secure Shell (SSH) and then piping your various web requests across this pipe or tunnel.

I’ve got 2 different SSH Proxies that I use daily.

Web Traffic – SSH Tunnel/Proxy:

ssh -CqN -D 8080 [username]@[hostname]

For above tunnel I’m using the following:

-D: bind port – in this case 8080 locally
-C: enables compression
-q: quiet mode (suppresses any warnings)
-N: don’t execute any remote commands

The -CqN are just some bells and whistles I use for the connection but not required. Please see below on configuring your browser to use the newly established SSH Tunnel.

Various other traffic (IRC, VNC, Torrent, etc…) – SSH Port Forwarding

ssh -L 6667:irc.[hostname]:6667 [username]@[hostname]

In this example, I’m binding a local port (-L 6667) to a remote boxes port (6667) through the server I have SSH’ed into. You can also add some of the bells and whistles from the web proxy to this one as well. Please see below for using this port forward with and IRC client.

Configuring the Browser:
The general idea (for Firefox) is to go to: Preferences –> Advanced –> Network –> Connection –> Settings. Select ‘Manual proxy configuration’. Set SOCKS Host: localhost Port: 8080. Click OK/Save and you should be good to go.

Here’s a screen shot of my settings:

While working on a little Mac ScreenSaver/Locking work -around I found this little site for converting and image file to various icon formats:

iConvert

I feel as a big newb when it comes to the rewrite rules. Not sure where my head’s been but it hasn’t been here. I’m finally grasping the concept of them. Here are some of my notes and resources that help me maintain some sanity with it.

Shortened file path

RewriteCond %{HTTP_HOST}    (.*)
RewriteRule ^images/(.*) sites/%1/files/images/$1 [L]

I used this one with a multi-site Drupal setup where the image paths wound up being something such as http://[domain]/sites/[domain]/files/images/image.jpg. Afterwards the URL was: http://[domain]/images/image.jpg.

References and Resources:
Apache Rewrite Guide and Examples
mod_rewrite Cheat Sheet

After a night out on the town with my buddy Dave, we headed across the street to hang with some of my neighbors. We were over for a while, primarly talking with Sheala and Kia while some of the home-boys (Moon, Tim, and 2 others) were playing craps on the sidewalk.

Everybody had been drinking, Dave was a little bit off his rocker. He accidentally tripped one of my neighbors – Moon -and sent him down the steps onto the sidewalk. Well, in my neighborhood you don’t get away with that (not even an accident). So, Moon had a little ‘talk’ with Dave where he gave him a nice smack. Dave went to the ground, but was honestly so drunk he didn’t even know what happened or remember it happening.

Well, I got Dave into the house and went back out to make my peace with the neighbors. After all it is my block. Well, everyone was still a little wound up and I was giving my apologies…”Sorry for my buddy, he’s a drunken fool, I’ll take care of…” BAM!!! I got sucker punched by one of the big dudes that was over hanging with us. Sent me to the ground and upon rising had blood all over the place. I took off into the house and got to the bathroom where I was spitting out blood. Looking in the mirror, the middle of my job was split in half.

As drunk as Dave was I got him to drive me up to MCV. I bolted in, let the lady know I had insurance, and they took me back pretty quick. They gave me some pain meds and after a few hours the oral surgeon wired the two sides of my jaw together with wire around teeth on each side. That held me over for a few days.

I had surgery on Tuesday where they lined me up nicely and put some titanium plates in my jaw to hold everything together. The doctor. (Dr. Greg Ness) did a bang up job with putting my jaw back to one piece.

So…I’m on a liquid diet for the next few weeks – 6 all together. I’ve been losing weight like crazy and have gotten quite creative with meals. Feel free to stop by the gallery and check out some of the pictures of it.

bwm-ng is a great little command line bandwidth monitor. HUGE fan. Its available with most all distros so use your favorite package manager to add it. Works on all *nix distributions including the Mac too.

bwm-ng home page: http://www.gropp.org/?id=projects&sub=bwm-ng

On the Mac it works great with a little application called GeekTool (will cover more later) with the following options:

/Users/derek/Applications/bwm-ng/bin/bwm-ng -o plain -c 1