Review of 2006 Goals

Today I benched 200 pounds one time. This was my goal for the entire year, so I am very happy with my progress thus far. Now I'm going for 225 by the end of the year.

Here are the goals I blogged about in January for this year and the results thus far:

1. Blog More
I'm definitely doing this, probably boring everyone a bit. :)

2. Go to the Doctor
Hmm...still six months left.

3. Get my weight to 175 lbs
I'm at 167 (I probably started the year around 163), only 8 pounds to go.

4. Oversee the creation the fastest webmail client on the market.
We are very close...

5. Become the best hitter on my baseball team.
We've only had 2 games so far, but our team captain is kicking butt. I'm hitting .375 with 2 RBI and 2 runs scored. That puts me tied for third (with my brother) on the team. We are 1-1 as a team. Mannin, our captain, is hitting .778 with a homer and 4 RBI. I have a long way to go, perhaps second best hitter on the team will be a good goal.

6. Shoot a round of golf under 110.
I've only played 3 rounds of golf this year. All at The Hill in Blacksburg. I have only had time to play 9 holes each time, but I think my scores have been fairly good, 51, 55, and 60. They are actually terrible scores, but I've made a lot of progress since last year. I think I definitely have a shot of getting a round under 110 if I ever find time to play a full 18 holes.

7. Take my family to Disney World.
Done, we had a great time.

8. Successfully manage our ever growing group of developers.
Ask Pat.

9. Bench Press 200 lbs one time
Today I did this, next up 225.

10. Find more time to spend alone with my wife.
Still working on this, but my favorite memory so far this year has to be the great time we had at Bill and Beth's wedding. We also had a great 5th year anniversary weekend. It's amazing how hard it is to find time alone when you have two kids. I'll keep working on it though.

Windows Vista Tips

Thanks to "Rick H." for posting this tips for Vista:

--------
Just a quick note to say that you can change the default search provider in IE7. Click the drop-down arrow at the edge of the search box in the upper right. Select "Find more providers" and see if your favorite is available (most are). Then click the drop-down again and choose Change Search Defaults.

For the admin popups (It's called UAP or UAC depending who you ask), run secpol.msc. Navigate to Local Policies->Security Options. Set UAC:Behavior of Elevation Prompt for Admins to "No Prompt" and UAC:Run All Admins in Approval Mode to disabled. (Use at your own risk!)
---------

I'm not sure how you figured out the UAC issue but what a life-saver that is for me. Thanks! Now if someone could figure out how to get my transparent windows back I'll be all set.

Windows Vista Beta 2 Review

After my computer decided to stop booting Windows XP Pro I decided to download and install Windows Vista Beta 2. It's been an interesting trial and error experience but I finally have it installed and running. I first had to ditch my old computer (Vista doesn't support the IDE driver that was built in 2001). I was able to install it on a brand new Dell, but I wasn't able to upgrade Windows XP Pro, I had to install from scratch. I'll probably be better off in the long run, I still hate how Dell insists on installing four million applications by default.

My first impression of Vista: "Cool Stuff". I like the visual changes they have made, I like the fading, the new transparency, the new start menu, the new explorer. I also like the gadget side bar, the new task bar, and the new task switcher. I also like IE 7, but there is no way to change the default search bar. It looks like it just hasn't been added yet, I am sure they will let me change it to Google eventually.

It comes with a few new apps like Windows Calendar and Windows Contact Manager, but they don't seem very useful. There is also some sort of collaboration app built-in, but that doesn't seem very useful either. Perhaps I need a few other users to help me test. The built-in search bar in the start menu seems to be a Google Desktop wannabe, but it appears to work rather well (I don't have a lot of data to test with yet).

I have only needed to install the following applications so far (they all work great w/ Vista):
Firefox
OpenOffice (Site doesn't work w/ IE 7 yet)
GAIM
Putty
Programmers Notepad
TurtoiseSVN
Thunderbird (for testing)

One really annoying "feature" that took two days to figure out is how to make my fonts look like crisp and clear. I was about to give up when I stumbled upon this setting:

Start->Control Panel->System and Maintenance->System->Advanced System Settings->Performance Settings->Click "Custom"->Uncheck "Smooth edges of screen fonts"

That setting has made all the difference in the world. Maybe today I can go home without a headache from my monitor.

Another issue is that I upgraded the video driver to the recommended version and I lost all of the cool transparency and window effects that I had on day one. :( I also haven't figured out how to turn off the annoying security pop-ups that open anytime I try to access an "Admin-Only" section of Windows.

I am finding the new control panel very complicated. It reminds me a lot of Perl in the fact that there are 50 ways to accomplish the same task. Navigating around the new control panel is frustrating, especially with the annoying "Admin Only, Are you really sure you want to do this?" pop-ups everywhere.

Overall, it looks like Microsoft has security as a top priority for this release. They are also finally catching up to Apple's cool windowing effects. And it won't take much more before they are caught up to Google on their desktop search capabilities. I can't think of any real compelling reason for anyone to upgrade besides the improved security features. Perhaps I'll find more in the coming days...

A Semi-Solution for My Internet Based File System Problem

After thinking about this more today, I have come up with the following solution that will almost work perfectly. First, we have an existing Subversion repository, so I created a "personal/kminnick" project for my own use. Second, we have an existing network drive called the "u" drive which can be used to store personal files. So, I installed a Windows based svn client called Tortoise SVN which is actually a shell extension that integrates directly into Windows Explorer. This basically means I can use Windows explorer to check out / check in files into Subversion, awesome stuff. I checked out my personal project onto our network drive (which is going to be available over a VPN eventually) so that nothing is stored on a local hard disk. To access the files from any location I can use a simple web browser to access the Subversion repository directly. If traveling, I could checkout all the files to my local hard disk, then check them back in whenever I have Internet access. Cool stuff. So with all of this setup and working I can:

1. All of my files are backed up automatically.
2. Have a history of every revision for all of my documents.
3. Access my files from any location that has a web browser.
4. Check out files locally for airplane trips, vacations, etc.

Not a bad solution, it was easy to setup and doesn't require a local hard disk for file storage.

Dead Computer

Today my desktop computer at work died. My dad gave me this computer about 2.5 years ago because he bought a new one, I'm guessing it was only about 2 years old when he gave it me. I was very attached to it, but all good things must come to an end. I was constantly asked by my co-workers why I never upgraded to a new machine. Well, for one, I never really “needed” a new machine. I spend most of my day inside of an SSH session, so memory, CPU, etc isn't that necessary. I'm curious though how many Word documents I'll need to re-type because I saved them to my local hard drive instead of on the network. I usually back everything up to the network or have them somewhere in my email for safekeeping, but I'm sure I missed a few. I use webmail exclusively for email, contacts, calendar, and tasks, so nothing will be lost there. I'm going to try to put in a new hard drive into my old machine and re-install Windows. But this time I'm going to try hard to move everything onto the network, the way it should be stored. In fact, I'm going to try to find a solution that will allow me to store everything I need onto the Internet.

This thought has really made the wheels turn this morning. I really need the following:

1. A web-based central repository that I can put / get files (any file type).

2. It should work like any good version control system (subversion), meaning it keeps track automatically of revision numbers, and I can view old revisions easily.

3. It would be great if I could open the files using “File, Open” from any application such as Open Office, Photoshop, etc. I shouldn't need to download the file first locally. I imagine this would require some sort of add-on to Windows Explorer. It needs to be fast though, so the plug-in should pull it down to a local temporary location, but automatically save it on the Internet when I click “File, Save”.

4. It should support sharing, so instead of having to email a document to everyone, I could simply email a note to everyone I want to share the document with and they can pull it down after entering an email / password. The link should always point to the latest revision so that timing isn't an issue. You should also be able to setup who can make modifications to the document.

5. You should have the ability to setup notifications so that you know when the document has been changed / modified.

6. It should be integrated inside of webmail.

For now, all I really need is #1. I think to solve that I'll setup a subversion server for myself and put all of my documents into subversion. I will be able to access it from home and work, the only downside is that I won't be able to open / save documents without the help of a subversion client. But at least everything will be backed up automatically and all of my documents will be accessible from the Internet.

If anyone knows of a solution that will meet the first five requirements above please let me know. I honestly haven't researched it yet.

Trac

Yesterday I setup and installed Trac for our development team. Trac is one of those tools that once a developer sees it, they gotta have it. If you haven’t heard of Trac, it allows developers to easily browse Subversion repositories. Subversion is the tool we use to track code changes. Subversion doesn’t have a good web-based GUI for browsing the repository, which is the primary reason we wanted to install Trac. Trac is easy to navigate, provides timelines, and makes browsing differences between revisions easy-easy-easy. The developers at Edgewall really have a winner with this product.

Trac also has a bug ticketing system, wiki, report generation tool, and milestone tracking tools built-in. The other tools need a bit more flushing out, especially the bug-tracker, but the SVN browser makes it worth the trouble to install. The installation process for Trac is…well…tough. I’ve installed hundreds of applications by source and RPM over the years and Trac was definitely one of the hardest I’ve ever encountered. Not only are there a ton of dependencies, but you have to be very careful that you have the right versions of each dependency. Thank goodness Google was able to help quite a bit.

I can see the benefits of Trac already paying off; just this morning, Mike sent me a link to the change-set in Trac for the next upload. Clicking on the link takes me to a web page with a description of the changes, a list of all files that were changed, and a color-coded side-by-side diff for each file….great stuff.

Thanks to Joel, one of our latest hires, for directing us to this application.

5th Wedding Anniversary

When I was in the basement today I saw a picture album from our wedding night and decided to reminiscence a little. First, it's amazing how much changes in five years. Second, it amazing how much stays the same. Velvet, Bill, Jason, and Velvet's parents all look the same today as they did five years ago. Pat, my parents, and my brother look totally different. I look like another person and wonder what Velvet was thinking...I'll bring the pictures to work if anyone is interested in looking.

To celebrate our anniversary we dined at The Summit restaurant in Christiansburg for the first time. Velvet suggested it and I heard three reviews all raving about the great food. I have to admit the food was really good (even though my steak came out a little too done). If you are looking to impress your date with some great food and a nice atmosphere I definitely recommend The Summit. :)

I'm more of a “look to the future” guy than “dwell on the past” guy; so I'll give you some insight on what will be going on five years from now, instead of what has transpired over the last five....In five years Drew will be in kindergarten and Jordan will be in 4^th grade. I'll be working on my 11^th year at Webmail.us. I'll only have 20 more years to pay on my house (haha). Maybe I'll have the driveway paved by then so I can put up a basketball goal for Jordan. Hopefully I'll still have my Jeep (gas will only be $6.00/gallon but that still beats having a car payment). Maybe my golf game will be in the 90s (OK, so I'm dreaming a little). Jordan won't remember a time when TV wasn't in high definition. Wireless Internet access will be available practically everywhere for free. Jordan will be using this laptop and think it's archaic. I might be caught up on the Sopranos. I hope Jordan can teach Drew how to play chess, read, and create his own blog. Jordan will still be in gymnastics (and will keep requesting a trip to Sonic after each session). Velvet and I will be celebrating our 10^th Wedding Anniversary.

OpenLDAP Errors

I ran across the following error in OpenLDAP two weeks ago when deleting an entry:

"Additional info: entry index delete failed"

Huh? We've been using OpenLDAP since 2003 and I've never seen this problem before, but looking at the OpenLDAP code showed it was trying to delete an index. We use the BDB backend and apparently a BDB index was corrupt. I haven't found an easy way to repair these indexes besides dumping all the data and re-importing. Maybe a BDB expert out there has some suggestions but luckily we replicate all of our data across multiple machines so I just had to copy the data (zip up the /var/openldap-data/ directory) from another machine. Googling for the error above didn't return many results so I thought I would go ahead and post my findings for the next person who searches for this error.

Harbinger Beta Available for Public Testing

A beta of the latest version of our webmail client is available to any current Webmail.us customer. Simply log in with your email address and password at http://beta.webmail.us.

You will notice only a few minor interface changes as our primary focus for this version is speed and performance. We no longer use frames and we make heavy use of AJAX and caching to improve speed. The first time you log in to the application will take a bit longer than normal because the browser is busy downloading JavaScript code that will be cached for future sessions.

Switching between folders, calendar, contacts, and tasks is significantly faster than previous versions of our application. We believe that a web based email client has significant advantages over a desktop based client but only if the web based client can match the speed and performance of the desktop client.

I've been using Harbinger for a few months now and have been really impressed with the speed thus far. We believe this is only the tip of the iceberg though and will continue to push the limits of the browser to maximize the speed of every aspect of the application.

Disney World Wrap Up

We had a great time at Disney World and I'm glad we were able to have a family vacation. I think that's the first family vacation we have taken since Drew was born. The kids did great on the plane flights, I'm so proud of them. Disney amazes me each time I go with how big they think. Nothing they do is too much, too big, too hard. They have big shows, big parks, big fireworks, and thus big crowds. They really do a great job and make you want to come back for more. We might head to Va Beach one weekend this summer, but otherwise I think we are done with family vacations for awhile. I'm glad I got to spend so much time with Drew, Jordan, and Velvet. If we do go back next year I'll have to remember these facts:

1. Go back to Magic Kingdom, Jordan loves it.
2. Don't go back to MGM Studios, it was fun, but not good enough to do two years in a row.
3. Make reservations at Cinderella Castle for dinner at least a month in advance.
4. Rent from a car rental place at the airport, not Hertz or Thrifty. (Hertz was great, but too hard to get on and off the bus.)
5. Caribe Royal is a great hotel, easy check-in, easy check-out, great pool, easy parking, but don't order the room service, very expensive and the food just wasn't that great.
6. Eat at Smokey Bones again.
7. Buy souvenirs at Wal-Mart in Orlando, nobody will know the difference and they are 1/10th the price.
8. Don't try to stay until close for the fireworks, the kids could probably make it, but I can't.
9. The "park hopper" option isn't worth it.
10. 10:30am is a perfect flight time, both ways. Connect in ATL at 1:30.
11. Mother's Day weekend is a good weekend to go because there is no baseball.
12. Jordan's favorite ride is the tram from the parking lot to the park, so ride it around a few times.
13. Jordan's favorite part of the trip is the plane ride and the pool, so take a trip to the beach with a Cinderella DVD and save some money. :)
14. Bring your own map of Orlando, the one from Hertz might as well been created by Jordan on the airplane ride down there.
15. Bring your own band-aid's so that you don't get weird looks from the first-aid people when Jordan gets a scratch on her leg.
16. Don't leave the diaper bag unattended next to the Haunted Mansion again, the next time someone might think it's a bomb and you don't want to be the one responsible for shutting down Disney World.
17. Invite some family so that you can go golfing while they take the kids to Disney World.

Disney World

Tomorrow I am packing up the wife and kids and heading to Orlando for our family vacation ... man I am old. :) Should be fun though and I'm sure Jordan will have a good time. We are heading back on Sunday.

Update on MySQL Issues

Last week I wrote about a strange issue with MySQL replication and what I thought I wouldn't see again for a long time. Well, around 1:00am this morning another MySQL server (a different cluster than previous) had the exact same issue as the first cluster. This time, my first step was to reboot the machine to see if it would fix the problem and thus not require me to spend a lot of time exporting and importing data. The reboot fixed the problem with the constant restart of the mysql server, but the replication thread failed to start. In fact, it was failing to start the slave thread on both machines. Issuing ‘start slave’ produced this in the error log of the machine with the problem:

060509 1:32:09 [Note] Slave I/O thread: connected to master 'replxxx@xxx-slave1:3306', replication started in log 'xxx-slave1-bin.004981' at position 79
060509 1:32:09 [ERROR] Error reading packet from server: Could not find first log file name in binary log index file (server_errno=1236)
060509 1:32:09 [ERROR] Got fatal error 1236: 'Could not find first log file name in binary log index file' from master when reading data from binary log
060509 1:32:09 [ERROR] Slave I/O thread exiting, read up to log 'xxx-slave1-bin.004981', position 79

This time I was determined to fix the problem without dumping data so I Googled a bit for the error and found this great post:

http://archives.neohapsis.com/archives/mysql/2004-q1/2000.html

Ah, so after creating the missing file and issuing a ‘reset slave’ the problem was fixed. On the other master I simply had to run a ‘change master to…’ command and ‘start slave’ and now replication was working in both directions again.

I think there is definitely a bug in the MySQL code somewhere, but I’m not sure what triggers it. I don’t think it’s related to the number of days a server has been up, but it might be related to the number of connections that have been opened since it was last restarted. It may just be a corrupt relay log file.

NCTC TechNite 7.0

Tonight the NewVA Corridor Technology Council (NCTC) is hosting TechNite 7.0. TechNite is self described as "the premier event honoring the NewVa Region's most promising technology companies, leaders, and entrepreneurs." Awards will be handed out in the following categories:

High Tech Leadership
Excellence in High Tech Manufacturing
Excellence in Life Sciences
Excellence in IT Development
Excellence in IT Deployment

Webmail.us is a finalist for the "NCTC Excellence in IT Development" award. Pat, our CEO, is a finalist for the "High Tech Leadership" award. It's quite an honor to be a finalist, especially considering the competition:

NCTC Excellence in IT Development Award
Webmail.us, Inc.
Phoenix Integration, Inc.
Synchrony Industrial Controls, Inc.
Integrated Imaging

NCTC High Tech Leadership Award
Margaret "Elaine" Carver, County of Roanoke
Kenneth Ferris, LUNA Innovations Incorporated
Patrick M. Matthews, Webmail.us, Inc.
Thomas A. Dingus, Virginia Tech Transportation Institute
R.J. Kirk, Third Security, LLC
Joseph W. Meredith, Virginia Tech Corporate Research Cent

Thanks go out to Trevor who played a big role in getting us nominated for these awards.

If you see Pat today be sure to wish him luck and ask him if he has that speech ready.

MySQL Replication and Heartbeat

Velvet, Marisa, Joel, Robert, and Andrew all did great in their respective 1/2 marathons and full marathons on Saturday. I certainly have a lot of respect for anyone who can endure all of the pain and agony that is involved. After we got back to the hotel I laid down for a power nap before dinner (yes, it was tiring watching all of those runners). I woke up to a phone alert letting me know a MySQL server was acting up. I log into the system and I'm seeing all sorts of errors and continuous notices that MySQL is restarting for every new connection. I've never seen this problem before, so I stop heartbeat so the other master takes over the IP. We run most of our sql servers in a master-master configuration. Master-Master means either server can immediately take over as the master if one goes down.

Thanks to heartbeat and MySQL replication, it allowed me to take my time and debug the problem without any service interruption. As for the debugging, I didn't get very far. I spent about an hour looking through log files, moving directories around (including to a ramdisk to make sure it wasn't a hard drive acting weird), etc. I was about to give the order to re-image the machine and start over when I had Bill reboot the machine because sometimes that magically fixes things. Well guess what, it did. I didn't check the uptime before the reboot, but according the other master it had been up about 284 consecutive days without a reboot. Not bad.

After the reboot fixed the restart problem, I decided to take a few safety precautions and assume the data was corrupt. So I moved the data to a backup copy, dumped everything from the master, re-imported, ran several test, and fired heartbeat back up. Everything looked good, and the whole process took about 4 or 5 hours. The restore took awhile because we have some tables that have over 9 million rows.

So my vacation in Nashville turned into a work weekend thanks to a MySQL server going crazy. Luckily we have the proper monitoring, replication, and failover systems in place, thank you Bill and team!

Big Weekend

This weekend I'm heading to Nashville via Knoxville to "watch" Velvet run in the Country Music Marathon and 1/2 Marathon (she's doing the 1/2). She's been training non-stop (I rarely awaken when her alarm goes off at 5:45am or whatever god-awful time she has it set to). I can't imagine running that far, but I can imagine that I will have fun this weekend. Our first stop is in Knoxville to go country line dancing at the world famous "Cotton Eyed Joe's". I haven't been there, but Jil says it's great so I can't wait. Based on the pictures from the website it looks like a grand ole time to me. And no, I won't be doing any dancing.

From there we head over to Nashville (first time for me) to check in at the hotel an get settled in before the big race. I'm sure Velvet will be nervous as hell (hehe). And with 20,000 participants I hope I can at least find her in the pack.

I wish Velvet, Jil, Marisa, and everyone else we are going with the best of luck...

Webmail Load Balancing Improvements

Last night we completed a “behind the scenes” project for webmail that will help us improve the scalability of our webmail server farm, improve our ability to measure the performance of our applications, and improve the overall speed up our applications. I started this project way back February and I previewed what we were doing in this post. By utilizing a piece of open source software called “Pound” we will be able to accomplish all of the goals listed above.

The primary reason for starting this project was our need to differentiate static web pages from dynamic web pages and send them to different clusters depending on the type of traffic. This would allow us to better cache the static content and get a better measurement on how well the dynamic pages are performing. We also wanted an easier way to manage all of our SSL certificates. Bill’s team (Tony and Korey) created a set of web pages that our support team can use to install and configure new SSL certificates. Mike also chipped in any necessary code changes that were need for webmail. And finally, Vinny helped me re-configure the control panel application for the new cluster of static servers.

I appreciate everyone’s help in making this project a complete success.

Tech Talk

I benched a personal best of 155lbs 10 times morning (which means I can pretty much bench my weight 8-10 times which is pretty good), but that's not what inspired me to post a blog...

Last night Me, Mike, and Pat gave a 'Tech Talk' on the Virginia Tech campus to a small group of computer science students. Basically we talked about Webmail.us, our development environment, our projects, the technologies we use, and we answered a lot of questions. I think it went well and we had a good time. One question that I did not get to answer was from a student who wanted to know what it was like to start your own company in Blacksburg because he was pondering the idea. Pat gave a great answer, but I couldn't help but think about this on the drive home.

My advice to anyone who is coming out of college and thinking about starting their own company is simple. As long as you are ready to put in long hours and are not afraid of failure, I think you should go for it. I think you will fail. But that's OK. Life is short, if you take that high paying job at IBM and Microsoft, that is the safe thing to do. It takes absolutely no courage to be that person. It will also prove to be unrewarding and boring in two years. Even if it's exciting work, you haven't learned the tough lessons that go along with starting your own company. You haven't learned how to fail, and you won't appreciate what it means to really work hard. If you are 22 and coming out of college, there is no better time to learn life's tough lessons. You have no kids, no mortgage, no obligations. It doesn't take a lot of money to start a company, you basically need an apartment and a work ethic. There is no better time than the present.

If you have an idea, a passion for that idea, and are not afraid of failing, GO FOR IT!!!

Of course, if you are afraid of failing, or you just don't have a great idea, I suggest working for a small company as a first step towards starting your own company. At a small company you will see and experience what the owners of the business are challenged by on a daily basis. You will see it's not easy. But this is great experience. After you work for a small company you will quickly realize that either that's the life you want to live or it's not. Perhaps you will find out that you do belong at a big company. Whatever the case may be, don't be afraid to take chances every once in awhile. That's my advice for all you graduates, good luck with whatever you do decide.

Product Development Releases Last 60 Days

Looks like I forgot to post this last month, but here are some of the great things we released over the last 60 days:

* Website: Printable Pages
* Control Panel: Improved Alias Management
* Control Panel: Simultaneous Logins
* Control Panel: Redesigned Login Page
* Website: Email Traffic analyzer
* Webmail: Address Book Import / Export Improvements
* API: Added 3 new functions
* Website: Redesigned Home Page
* Control Panel: Improved Availability
* Webmail: Spanish Language Improvements
* Webmail: Ability to select default language, and html font

There's plenty of stuff I can't mention too, but overall we are really busy around here and are constantly pumping out bug fixes, new features, even a beta. ;)

March Bench Press Results

Technically I have another week in March, but I don't think next week will be better than today, so I'll go ahead and post this month's progress:

185 x 3
195 x 1 (2 with a spot)

My bully spotter made me try 205, but I needed a little help from him with that, maybe next month. I definitely should be able to hit my goal of 200 next month, so I might up the ante to 225 by the end of the year. The best part of all of this is that it's been the same two people in there spotting me every Friday since I started in January and they were saying today they can really tell a difference. Hopefully the progress will continue...oh, and I weighed in at 165 today.

Google is Yahoo 2.0

Jeremy posted this awhile back, but with the recent launch of Google finance, I totally see what he is talking about. I really enjoyed Jeremy's latest comments on how Yahoo! had many of these same ideas years ago, but due to many issues they were never able to act on them.

I think Yahoo! and Microsoft are very much in the same boat on this one. All of the competition from open-source in regards to the browser, the desktop, the office application suite, the server market, and the software as a service model is very similar to the Google vs. Yahoo! fight.

I certainly hope that our commitment to keeping our product teams small and focused will prevent our software from becoming stale. I want our teams to be able to release high quality software and then immediately be able to turn around and say "How can we make this even better?"

Congratulations Bill & Beth

Last weekend Bill & Beth were married in Alexandria, VA. I have known both of them it seems like forever and they are really great friends. They certainly know how to throw a great party too and their wedding was no exception. I think everyone had a great time and I was glad so many of our friends were able to attend.

So who's the next one in our company to tie the knot? Manny? Pat? Here are my odds:

Manny 3:2
Brian 11:5 (Vegas is only a plane-ride away!!!)
Pat 2:1
Mike 4:1
Steve 5:1
Huey 8:1
Matt 10:1
Ben 10:1
Vinny 12:1
A part timer 20:1


Any takers?

My Evaluation of Reverse Proxy Software

A proxy server takes outbound traffic and makes it appear as if it comes from a single IP address. This type of proxy is useful if you have a single access point to the Internet and want multiple computers behind that access point to be able to access the Internet. This is common in an office or corporate environment.

A reverse proxy takes incoming traffic and sends it to multiple backend computers. This is useful if you run a large website and want a single IP address to actually be served by multiple machines. You can also accomplish this with a load balancer, but load balancers typically lack several key features that a reverse proxy can provide.

Let's take a look at 4 reverse proxies that are currently available:

1. Apache's httpd
2. Squid
3. Perlbal
4. Pound
5. HAProxy

All of these are very good, but depending on your requirements some may not meet your needs. I'm interested in a reverse proxy that can handle the following requirements:

1. Easy to configure
2. Small memory footprint
3. Provides load balancing
4. Supports SSL to client, but non-ssl to the backend
5. Provides High Availability
6. Can proxy traffic based on URL pattern matching
7. Supports virtual hosts

Perlbal is really a terrific piece of software, but it has a lot of features that I don't need, plus it does not have one key feature that I do need, SSL. Although there is an option to configure Perlbal with SSL, don't do it. It's too easy to break because of this problem reported on their listserve. It appears this is not a problem with the Perlbal code, but a problem with the underlying libraries that Perlbal uses. I could stick stunnel in front of Perlbal and I think that would work great, but you have to setup one instance of stunnel per SSL site and that wouldn't work for me. Perlbal uses epoll, which would provide a big performance boost. Perbal doesn't support sticking users to backends (for session support), but that is not a requirement for me.

Squid has been around for a long time, but I think it's too bulky. First, the configuration is too hard. Second, it's primary goal is to serve as a proxy, but I do not need this, just the reverse proxy. Squid also does a lot of caching, I don't need this either, I would prefer to have something that never touches disk. Also, to my knowledge, Squid does not support high availability (checks to make sure a backend is not down).

Apache's latest release (2.2) of it's httpd server does have better support for load balancing and high availability than previous versions, but it's still too bulky for me. One thing that kills me about Apache is how hard it is to configure mod_rewrite. I actually was able to setup Apache 2.2 as a reverse proxy and it was working perfectly, but I realized that this was going to be too hard to maintain on a large scale system. It's great that Apache is so configurable and can handle just about any task, but that can also be looked at as a big negative too.

Bill showed me HAProxy and it looks great, very configurable. But it requires Apache for SSL support, and I don't think you can proxy traffic based on URL.

The final piece of software that I looked at as a possible reverse proxy that would meet all of my requirements was Pound. I had a little trouble compiling Pound at first on RHES3, but I finally figured out the right compiler flags. On RHES4 it compiled with no problems. Besides that Pound seems to work great. It's easily configurable, is designed to accomplish only one task, and is small in size.

So there you have it, my evaluation of reverse proxy software. If I missed anything or I am just dead wrong, feel free to leave a comment. I'll update this post if anything is inaccurate.

February Bench Press Results

Another month has come and gone and I know you are dying to know how much I benched pressed today, so here ya go:

155 x 8 times
175 x 3 times
185 x 2 times
190 x 1 time

190! Holy cow, only 10 more pounds to reach my year end goal! Not bad considering I weighed in at 163lbs today. I haven't gained or lost any weight since I started working out, that might be more amazing than the 190lbs I lifted today.

Cryptic Apache Error

Each night our webmail servers automatically restart httpd to clean up memory and start fresh. Last night one of our servers hit this error upon restart:

[crit] (28)No space left on device: mod_rewrite: could not create rewrite_log_lock Configuration Failed

I had never seen this error before and had no idea what it meant, but Google helped me out:

http://www.goldfisch.at/knowledge/224

After some testing, I found that Apache leaks a couple of semaphores after each restart and eventually w/o a server reboot will run out of resources. So I changed our restart script to clean up these semaphores each night.

ipcs -s grep nobody perl -e 'while () { @a=split(/\s+/); print `ipcrm sem $a[1]`}'

In case you are wondering, it's not a big deal if a webmail server doesn't restart properly. Our load balancers quickly take a down server out of rotation so that customers are not affected. Thank you keepalived.

Development Releases Last 30 Days

Over the last 30 days we have released some really great additions to our platform. Here is the high level list I presented at our management meeting on Monday:

* Webmail Lite w/ Calendar / AJAX
* Control Panel Security Additions (IP Based Restrictions)
* 20+ Webmail Bug Fixes
* Webmail Import / Export Enhancements
* Company Directory
* Website Search
* 30 Day Free Trial

You can read the details on our blog.

OpenLDAP Replication

Today I setup a new LDAP slave machine, the process is fairly straight-forward, but a good thing to know about replication is how to do it without shutting down the master for an extended period of time. Let's take a look at how to do this:

First, let's verify we have a few things:

1. The LDAP ssl port is open from the master to the slave:
ldap-master> telnet ldap-slave 636

2. Make sure the slave has a valid SSL certificate. If you are using a self-signed cert you will need to make sure the master has the slave .crt file installed. If you have multiple slaves, you need to create a combined.crt file and concat all of your certs into one file. Then add an entry to the master ldap.conf (not slapd.conf) like so:

TLS_CACERT /usr/etc/openldap/certs/combined.crt

OK, so now we need to get all of the data from the master to the slave without losing anything during the transfer. Normally this would require the master to be taken offline for some period of time while you zip up the data files and scp them over. But, luckily I have another slave that is not being used for queries, so I can follow these steps to get the data moved without having to shut down the master for an extended period of time:

0. Stop the new slave.
1. Add the necessary

replica uri=ldaps...

lines to the master slapd.conf file.

2. Restart the master.
3. Stop the old slave.
4. Tar/Zip the data, scp to new slave.
5. Start the new slave, verify replication works to the new slave.
6. Bring back online the old slave, verify replication is still working.

That's it, really simply. One trick you should know about replication is that if you have multiple LDAP databases being replicated, slurpd will not replicate the second database to a machine with the same hostname as the first database. I have to create a fake host entry in /etc/hosts so that slurpd doesn't get confused. It took me awhile to figure this one out.

Bench Press Update

I've been working out consistently this month, pretty amazing, but I found that going in the morning is the only way I can do it. Today was the last chest workout for the month, so I thought I would log my progress here.

Today I did:
135 x 12 reps
155 x 6 reps
165 x 3 reps

165 is a long ways from 200, but I think I made good progress this first month. I started the month doing 115 x 12 times.

New Webmail Lite and Control Panel Features

We just finished releasing the next version of Webmail Lite. This version has a calendar, task list, and uses AJAX for speed improvements.

We also added a feature to our control panel that allows administrators to use IP addresses to restrict logins.

Read all about these these enhancements on our company blog.

2006 Goals

It's that time of year for resolution and goal setting, so here is my list for 2006 (in no particular order):

1. Blog More
I'd like to use this blog to record interesting things in my personal life and to document any research or technical details about the work I do at Webmail.us.

2. Go to the Doctor
Since we have great health coverage at Webmail.us, I better start using it. My dad has had a lot of health problems recently and let's just say it's been a wake up call for me.

3. Get my weight to 175 lbs
I think it's a perfect weight for me, and this goal will force me to attend the gym regularly.

4. Oversee the creation the fastest webmail client on the market.
Speed, speed, and more speed. We are working hard on this problem and it is the focus of our development efforts this year. I'm relying on the great developers we have to help me accomplish this goal.

5. Become the best hitter on my baseball team.
The past two years I have been the second best hitter on my team. This year I really want to take over that top spot.

6. Shoot a round of golf under 110.
I would say under 100, but that might be a bit unrealistic since I never have time to practice.

7. Take my family to Disney World.
Velvet and I work so much that we rarely take family vacations, so I would like to take the kids to Disney World this year. I know Jordan will love it almost as much as Velvet. :)

8. Successfully manage our ever growing group of developers.
I think this will be my biggest challenge this year. It's also the challenge I'm looking forward to the most.

9. Bench Press 200 lbs one time
I use to be able to do this, then I stopped working out. I'm probably at 165 right now (I use to do this 8 times w/o problems), but I honestly haven't tried a low number of reps in a long time.

10. Find more time to spend alone with my wife.
For obvious reasons... ;)

Understanding memcached memory management

memcached is a very fast program that can be used to cache data stored in a database for quick access and dramatically reduce the load on your database servers. It's one of those "must haves" if you develop a highly trafficked website that has a database backend. Understanding the internals of memcached may be a bit confusing at first, the details aren't documented well, and there are some tools that aren't that easy to understand what they do. I'll try to explain here by example.

Let's say you have a memcached setup and have allocated 256MB of RAM to the program. memcached works hard to avoid memory fragmentation, it does this by taking the memory and dividing it into classes, numbered 6 through 17 (don't ask). As you insert into memcached, it will take the size of the data you are inserting and inserts it into the class that is big enough to handle the data. See the chart below. The classes in the below memcached are of size 128B - 16KB. Ignore the Max_age column for now. Each class that has data is allocated at least one 1MB page. So, class 7 has 4 1MB pages, each data item in that class is using 128B and the class is full, so there are 32767 items in the class. (If you insert an item into memcached of say 100B, the class uses all 128B to avoid fragmentation.)

Now if we try adding to this cache, we won't necessarily get an "out of memory" error because memcached uses a LRU (least recently used) algorithm to kick out the item from the page that has been accessed the least. But, if we attempt to insert an item of say 15B, we will get an "out of memory" error because class 6 (the class would be of size 64B) has no items, and thus no pages have been assigned. But, all pages are currently being used, and memcached cannot allocate a 1MB page to class 6. In order to make this happen we would have to move a page from another class to class 6.

Let's examine how this is done:

First, let's get a snapshot of our current allocations:

[root@sql-slave1 bin]# ./memcached-tool localhost
# Item_Size Max_age 1MB_pages Full?
6 B 0 s 0 yes
7 128 B 742942 s 4 yes
8 256 B1175448 s 2 yes
9 512 B1279471 s 1 yes
10 1 kB6493931 s 1 yes
11 2 kB 434740 s 71 no
12 4 kB 917212 s 140 yes
13 8 kB 794005 s 30 no
14 16 kB1373021 s 7 yes
15 B 0 s 0 yes
16 B 0 s 0 yes
17 B 0 s 0 yes

Now, let's go ahead and try to insert 2 bytes of data:

[root@sql-slave1 bin]# telnet localhost 11211
Trying 127.0.0.1...
Connected to localhost (127.0.0.1).
Escape character is '^]'.
set key1 0 0 2
SERVER_ERROR out of memory

OK, just like I thought, memcached can't allocate a 1MB page, so let's help it out:

[root@sql-slave1 bin]# ./memcached-tool localhost move 14 6
Success.

Now, let's look at our pages again, as you can see 1 page has been moved from class 14 to class 6. The LRU items in class 14 go bye-bye.

[root@sql-slave1 bin]# ./memcached-tool localhost
# Item_Size Max_age 1MB_pages Full?
6 64 B 0 s 1 no
7 128 B 743011 s 4 yes
8 256 B1174812 s 2 yes
9 512 B1279618 s 1 yes
10 1 kB6494078 s 1 yes
11 2 kB 434887 s 71 no
12 4 kB 917208 s 140 yes
13 8 kB 794152 s 30 no
14 16 kB1373168 s 6 yes
15 B 0 s 0 yes
16 B 0 s 0 yes
17 B 0 s 0 yes

Let's insert our data again:

[root@sql-slave1 bin]# telnet localhost 11211
Trying 127.0.0.1...
Connected to localhost (127.0.0.1).
Escape character is '^]'.
set key1 0 0 2
aa
STORED
quit
Connection closed by foreign host.

Cool, it worked. Looking at our stats shows the item is aging:

[root@sql-slave1 bin]# ./memcached-tool localhost
# Item_Size Max_age 1MB_pages Full?
6 64 B 14 s 1 no
7 128 B 742970 s 4 yes
8 256 B1174852 s 2 yes
9 512 B1279658 s 1 yes
10 1 kB6494118 s 1 yes
11 2 kB 434918 s 71 yes
12 4 kB 917228 s 140 yes
13 8 kB 794192 s 30 no
14 16 kB1373208 s 6 yes
15 B 0 s 0 yes
16 B 0 s 0 yes
17 B 0 s 0 yes

Conclusions, you better keep an eye on the data in your memcached, and knowing what type of data you are storing will help you maintain it. Writing scripts to automate the moving around of pages based on data should be fairly easy.

Also, there are some little known commands that I use a lot:
stats
stats items
stats slabs
stats sizes

memcached is a great piece of software, many thanks to the Danga team.

Newsworthy

Looks like my wish came true:

http://www.hokiesports.com/football/recaps/2006016aaa.html

and Google has bundled some common software and made it easy to download and install, I haven't tried it, but it looks pretty useful:

http://pack.google.com/

Maybe I'll wait for them to release their operating system. I'm very skeptical this will happen anytime soon. It would have to be built on some sort of Unix flavor and the bottom line is that most of the applications they have built don't have Unix versions. Of course, it is Google, so I'm sure it's on their drawing board somewhere.

529 Savings Plans

I just finished signing Andrew up for a 529 savings plan. For those of you who don't know, 529 plans are a great way to save for college. They invest in stocks, bonds, etc, just like normal mutual funds, but the money has to be used for college. The funds have to be sponsored by a state, so each state has a fund. My financial advisor, Clark Howard, has some great tips about 529 plans, including his "Dean's List" and "Honor Roll List" of states with the best plans. One piece of advice that I don't listen to, even though I agree with is

"But first remember my rule that you don't save a penny for college unless you are already saving the maximum you can for your own retirement. College can be paid for with grants, loans, scholarships and work. Retirement happens only if you have saved the dough."

I don't think the whole "saving the maximum" thing will happen until Jordan is about 33, a little too late to start saving for college. I guess I'll just have to work a few more years, but I like working so no big deal. When I enrolled Jordan, I picked the Missouri state plan because it was on his Honor Roll list ( I don't think he had a Dean's List 4 years ago) and you could sign up for the plan online. This time around I wanted to chose one of his Dean's List plans so I went with the New York plan. The plan is managed by Vangaurd and you can sign up online.

Both the Missouri and New York plans let you invest in "age based" options, meaning the plans become less aggressive as the beneficiary gets older. This is exactly what I want plus it requires zero maintenance on my end, a big-time bonus because I hardly have time to pay attention to it.
I looked into the Virginia 529 plan too (my home state), but their so-called website looks like something a 2 year old put together with Microsoft Frontpage '98.

Va Tech Football 2005 Year in Review

The Virginia Tech football season has come to end and it certainly was a season filled with ups and downs. I enjoy attending the games and have seen the football program grow from being just awful to exceptionally good year after year. Becoming a winning program does, apparently, come at a cost. If I had one wish for next year's team, I wish that Frank Beamer would get some guts and bench players that cause more trouble than what they are worth. Some players just behave in a manner that does not reflect well on my alma-mater. I guess discipline and sportsmanship are far less important than winning. I suspect Frank has so much riding on winning these days that he can't afford to bench players that are obviously a poor reflection on the entire program. I completely understand his position, and our program is certainly not the first program to have such issues.

Perhaps Va Tech can start offering a new class for all incoming recruits "How not to look like an immature spoiled kid on national television 101". I guess if you want a squeaky clean program, attend Vanderbilt...Which bowl did they go to this year? And when is my season ticket application due? :(

Update: Since writing this post (I saved it as a draft yesterday), the athletic department at Va Tech released this statement.

Java Schools

Joel has a great blog post about "Java Schools", it's a good read and I agree with every bit of it. The test we give programmers on their first interview has questions regarding both pointers and recursion. They are two of my favorite questions, most people think they are too easy, but the fact is if you solve the recursion problem using a "while loop" it tells me a lot. If you solve it using binary operations, you're hired. ;)

I don't use a test as the determining factor for any hire, but I like to see how people go about solving problems. Some struggle, some work fast, some love it, some hate it, some even give up and leave without even trying. If you can't handle 6 simple problem solving questions though, chances are you can't handle developing software applications that are critical to the success of tens of thousands of businesses.

MySQL Replication

This morning I setup MySQL replication for two of our user / spam databases that we use internally. This is about the 10th time I have done this, so I don't even have to look up the instructions anymore, but I have to admit that the MySQL team has made the replication process ever so easy. We setup our databases in a master-master configuration with multiple slaves. Slaves can span geographic locations, for instance, from Dulles, Va to Blacksburg, Va, some 300+ miles apart. It's a very nice setup and I always have so much fun setting it up, I don't know why, but for some reason that type of work is fun for me, especially when it works the very first time I try it (and the last two times it has worked right off the bat). Thinking back to the first time I setup MySQL replication it took a few tries to get it right, and so if you are having trouble getting it to work, here are a few suggestions:

1. Read High Performance MySQL by Jeremy Zawodny
2. Turn on logging (vi /etc/my.cnf, log=/var/lib/mysql/log.txt)
3. Make sure every server has a unique id (server-id = 1)
4. I had a pound symbol (#) as part of the password, for some reason this didn't work.
5. Make sure you can telnet to the mysql port on each machine.
6. Make sure the username/passwords are correct.
7. The "show slave status" and "show master status" commands are very helpful.
8. Only replicate the databases you need, make sure there is not a lot of unnecessary traffic.
9. Testing is easy to do, so do it.
10. The /etc/my.cnf file is not used after the first reload of mysql. You need to run the 'change master to' command from that point forward.

Jeremy's book was very helpful, so I highly recommend it for the first-timer.

The best tip I can give is to setup a slave server that does nothing but replicate from the master. This is important because some day you will want to setup another slave without bringing down a live master. This is nearly impossible if you are using all of your machines for queries. To setup a new slave without any downtime, do the following:

On the slave that is doing nothing but replication:

1. Run 'stop slave' from mysql prompt
2. Run 'show slave status\G' from the mysql prompt, record the following two lines:
Master_Log_File: rss-master1-bin.000082
Read_Master_Log_Pos: 993448778
3. Run a "mysqldump" to get all of the data to a text file.

On the brand new slave, do the following:
1. Setup the databases, users, and import the exported data.
2. Run the following command from mysql

CHANGE MASTER TO
MASTER_HOST='master-server',
MASTER_USER='repl',
MASTER_PASSWORD='xxxxxxxx',
MASTER_LOG_FILE='master-server-bin.000082',
MASTER_LOG_POS=934159359;

3. Run 'start slave' from mysql (on both slaves).
4. If you have logging turned on, you will see all of the queries coming through since you stopped the slave process on the first slave machine.

That's it, it should work fine, but testing is encouraged. :)

Webmail.us 2005 Development Year in Review

This past year, thanks to the hard work of our entire team, we were able to launch many improvements to our email hosting infrastructure. Below is a partial list of what we accomplished over the last year:

An improved billing system for our accounting team
Faster, more reliable statistics for our customers
Control Panel 2.0
Control Panel API 1.5, 1.6
Webmail API 1.0, 1.1
Centralized Logging System
Revamped Webmail.us Website
Webmail Lite
An Auto-Configure Tool for POP3 clients
An Outages Alert System for our customers
Improved Migration Process
Improved Anti-Fraud System
Webmail 3.1
Webmail w/ RSS Reader
Webmail w/ AJAX, Calendar, and Company Directory
Webmail Help Section

This list doesn't include all the work required to help get the new infrastructure up and running and any custom development work we did for our customers.

We were able to accomplish so much because of our great team. I thank everyone involved in the process.

Another Tailgating Season Completed

It's hard to believe that the home football season for the Hokies is over and in the books. I've never had so much fun tailgating and it's a shame the season is over. The main reason the tailgating was so great this year is the great group of friends and co-workers that come out to eat, drink their favorite beverage, play cornhole, and chat about an unimaginable number of topics. A big thanks goes out to David and Jenny Catalano. They are the ones who actually paid for the great spot and we owe them big-time. Also, a big thank you to Bill Boebel who let us use his truck all season, got up at 6:30A.M. to secure the spot, and probably paid for lots of food and drinks out of his own pocket. There are many other people that helped out with the tailgates each week and I'm sure I am going to forget someone (it's a long season, and I didn't exactly take notes) but here it goes:

Velvet, thanks for the awesome taco-dip and most importantly, putting up with me.
David and Megan Lawson, thanks for the great food, beverages, and putting up with all of us.
Gary Francis, thanks for introducing us to the great game of cornhole.
Matt, thanks for getting the food several times and bringing Velvet a birthday cake.
Mike, thanks for being a great cornhole partner and let's just agree never to wrestle again...
Brian Matthews, thanks for the great phone call at 2:30AM, sorry the girl never showed up. :)
Pat, thanks for being so supportive of our efforts and trying your best to make sure that everyone has a good time.
Jilian, thanks for being so sexy. ;)
Justina, thanks for ... well, you know. ;)
Brian Hartsock, thanks for putting up with Velvet and Justina :)
Beth, thanks for taking care of Bill.
Tammy, thanks for being a designated driver.
Vinny, thanks for being a designated driver.
Huey, thanks for calling Vinny.
Marissa, thanks for hanging out until 4:30AM (or later) twice.
Bill (again), thanks for leaving the truck running for 12 hours straight so we could listen to your great music. :)
David (again), thanks for the great wood-burner.
Totally random people, thanks for stopping by and hanging out.
Virginia Tech Police, thanks for not stopping by. :)
Hokie Football Team, thanks for winning so many games.
Everyone else, thanks for coming by and hanging out!

If I forgot about someone or something, I'm sorry in advance, and feel free to leave a comment. Me and Velvet will probably skip on the ACC Championship game, but are planning on attending the bowl game. Let us know if you are going too!

My Thoughts on Microsoft Live

Microsoft announced last Tuesday their plans for Windows Live and Office Live, you can read about it here. I sometimes like to give Microsoft a hard time, but I really don’t have anything against them. In fact, I think they are an excellent company. Anyone who doubts their ability to take over any market is forgetting what history has taught us.

Let’s take a look at some of the companies they have conquered in the past:
* Netscape (browser)
* Apple (Operating System)
* Borland (C++ Compiler)
* IBM (Operating System)
* Lotus (Spreadsheet)
* WordPerfect (Word Processor)

I think jumping into online services is a great idea for Microsoft. I think they need to do two things in order to really succeed at this strategy:

1. Integrate their services tightly with their applications
2. Develop API’s so that developers can build applications on top of these services

It sounds like they are going to do both, but I think #2 is especially important. If they provide access to a large web service API that would be a powerful move. It opens the door to a lot of possibilities for the development community out there and I think that’s important for wide-spread adoption. For those people who think they are getting into the web service game a little too late, think again. Being the first to do something has never been a part Microsoft’s strategy and I don’t think they will let their late arrival bother them this time around either.

How will this affect companies like Webmail.us? It's a good question but I don't think it will have much of an impact on what we do. Our company has been tremendously successful because we provide superior service and support, we continuously innovate our product, and we are focused on one thing - email. Our customers love us for our attention to detail, quick response times, and our passion.

If anything, I think the move by Microsoft into the web services space justifies our business model. We have known for a long time that offering software as a service makes complete sense and it's great to see that Microsoft is finally recognizing that fact too.

My Daily Routine Part II

It's been almost six weeks since Andrew's birth and I just have one word to sum up the whole experience...”WOW”. He is growing so fast (just over 9 lbs now) and he will start going to day care this week. My routine for the last 6 weeks has changed very little. I tried going to bed early and getting up really early (like 5:30am), that lasted about one week. It's just too difficult for me to become that hard core at this point. I would wake up at 5:30, do work for about 1.5 hours from home and then take a shower and head into work. What I have found works best is for me to get up at 7:00am each morning. I have been able to do that consistently for the last 3-4 weeks. If I wake up at seven I can get Jordan to school between 8:00am and 8:30am. I arrive to work shortly thereafter, work through lunch and leave the office around 5:15 to go pick up Jordan from school. I get home around 5:45 or 6:00 and eat dinner, play with kids, etc. I usually do work between 9:00pm and 11:00pm and try to get to bed before Midnight. For some reason, even if I end up staying up until 1:00am, I have no problems waking up at 7:00am when the alarm goes off.

The only part about my day that I don't like is that I have no time to workout. I'm hoping that I can find some time to get back into the gym sometime soon. At work, it seems like I sit down at my desk in the morning and before I can blink it's time to leave and get Jordan. I hate the days that I forget to pack my lunch because I don't like having to go out and get food, it's such a waste of time.

Before we had Andrew everyone asked how I was going to handle the lack of sleep. Thus far, it's been easy, Velvet handles 99% of the work. She's been great and has really made it too easy for me. She starts work this week and I have a feeling the next six weeks will be harder than the first six. So stay tuned...

My New Laptop

I finally broke down and had the company buy me a new laptop. My first laptop was a Compaq I bought in 2000, it was an awesome machine. It lasted until December 2003. Finally the hard drive died, I believe it was a 600MHZ Pentium or something. I then took Velvet's old Compaq that we bought in December 2002, it was a 950MHZ Pentium. I used it until about 3 weeks ago. It was a terrible machine. Velvet bought a Dell about a year ago that has performed reasonably well.

I'm really stubborn when it comes to getting a new computer, I try to get everything I can out of a machine until it dies. I think it comes from my poor background. Hell, I used my Commodore 64 from the 2nd grade until my junior year of high school, no joke. That computer was definitely the best computer I've ever owned, but that's a different story.

Besides the Compaq being dog slow (which I really didn't mind because I use SSH for almost everything except email), it had the following issues:

1. The battery only lasted 23 minutes before it would shut off the computer, losing all work.
2. The computer would shut off randomly if the CPU got too hot. I never figured out what made it run so hot, but I did notice that when I had the AIM window open and an advertisement ran with animation the CPU would spike to 100% and shortly thereafter the machine would shut down. Needless to say, I now have a habit of keeping AIM always minimized.
3. The monitor part of the laptop would not stay up on it's own. It would fall back when you opened it, so I had to always prop it up with two pillows. Jordan even knew to put two pillows behind it before opening it, I hope she doesn't grow up thinking that's how *all* computers work.
4. The fan was so loud that I had to turn the TV up about 4 notches to hear it. I remember whenever we would watch movies I would turn off the laptop for a better movie experience.
5. The last straw for the Compaq came when the power cord would no longer stay in socket, so it was impossible to charge. This was the second adapter I bought for it, the first had a different problem that was replaceable, this problem was impossible to fix

Now, my new Sony Vaio is great. The screen is bright, you can actually put it on your lap without worrying about 2nd degree burns, it's quiet as can be, and it's extremely fast. The power cord is a little short, but I don't really need it since the battery last about 2.5 hours (not great, but beats 23 minutes). My only complaint was the crazy amount of software pre-installed. Software? Who uses software? All of the software I actually use, I had to download:

1. Putty
2. AIM
3. Open Office
4. Firefox
5. Programmer's Notepad

Oh, wow, somebody remind me again why Microsoft exists? OK, so I still use them for a window manager, but maybe I don't need to.....

Andrew Ryan Minnick

On September 28th 2005 my second child, first son, Andrew Ryan Minnick was born. It was an awesome experience and impossible to explain in words. I would post a picture here, but blogger sucks and won't let me right now. See Velvet's blog for more pictures and more details. Everything has gone great so far, it helps that he enjoys sleeping as much as he does, hopefully everything will continue to go smoothly. Jordan loves her baby brother and has not been jealous at all, she's a good little helper. Velvet is doing good too, but I think she's getting very anxious to get back to work.

My Daily Routine

I can tell by the amount of cleaning that Velvet is doing around the house that the greatly anticipated birth of my second child is coming soon. I’m not quite sure how I am going to juggle everything but I thought documenting the daily routine before and after the birth would be an interesting exercise, so here goes the before:

My weekdays begin with the alarm going off at 7:00am sharp. Most mornings I lay there for a few minutes hoping that Velvet will jump into the shower first so I can grab an extra 15 minutes of sleep, it’s usually a 50/50 chance of that happening. Before showering, I open up the laptop, read my morning email, answer all the support questions that I can, read the blogs I follow, and check my Webmail Lite account. (All of this can be done in 15-20 minutes). After showering, shaving, and ironing; I grab a donut, pop-tart, or anything else that is quick and easy and gobble it down. Hopefully by this time Jordan is up, in a good mood, and requires only a moderate amount of teeth pulling to convince her to get dressed for school. (I won’t talk about the days that Jordan is in a BAD mood, LOL) Jordan generally has only two requests: Mickey Mouse pancakes (in a baggie so she can eat in the car on the way to school) and some chocolate milk.

Jordan’s school is only 5 minutes from work and right on the way, so it’s very convenient. I listen to my favorite sports talk show “Mike & Mike in the Morning” on the way to drop off Jordan. Jordan is generally very quiet during the ride to school, eating her pancakes and drinking her milk. But she does like to point out every school bus that she sees (and she has yet to miss one). I usually drop Jordan off at 8:30am and arrive at work shortly thereafter.

I eat lunch at my desk really quickly and try to leave the office by 6:00pm. (Although I have managed to find time to workout twice this week I would hardly consider it part of my “daily routine” at this point.) I come home (Velvet picks up Jordan from school), fix dinner (50/50 chance Velvet will handle this), read a book to Jordan, and give her bath. By this time it’s 8:00pm and I once again open the laptop, check my email, answer support questions, read my blogs, and begin working on whatever issues I didn’t quite have enough time for during “business hours”. After another 3-4 hours of working I close the laptop, grab a few late night snacks and head to bed. I usually toss and turn for 30 minutes or so thinking about servers, code, scalability, infrastructure, hiring, planning, or whatever other issues happen to be on my mind at the time. I force myself to sleep because I know 7:00am will come fast and furious.

Hopefully in a week or two I will have an update on the “routine” or lack thereof. I have a strong suspicion that this may be my last blog for awhile though.

Baseball

Baseball

As a kid growing up I had two passions, baseball and computers. I got my first Commodore 64 when I was in second grade and was hooked. I started playing baseball not long after that and if I wasn’t in front of a computer I was outside playing baseball. (A chick magnet I was not.) My favorite memories growing up all revolve around those summer days when my brother and I would spend endless hours playing toss, scrimmaging the neighborhood kids, and heading to the town park for the big game. My brother (Scott) is three years younger than I, but that never stopped him from playing with the “bigger” kids. It didn’t take long before everyone realized he had exception baseball skills; an obvious result of the early start and having to always play with kids older than him.

Little league lasts three years where I grew up and Scott was simply playing in a league of his own by year 2. He dominated on the mound as a pitcher, and was a terrific hitter as well. His third year his team went undefeated with Scott pitching every game. He pitched 3 no-hitters that season, and struck out all 15 batters (little league is only 5 innings) twice. I’ll never forget watching his games and dreaming about his potential future. Unfortunately, like so many young kids, by the time the next summer rolled around he was no longer able to pitch. Basically his arm gave out, probably a result of overwork at too young of an age. He never pitched again after those two years in little league and I’m sure the game was ever so boring to him from the outfield. Scott continued to excel at hitting, however, and was the star of his JV and Varsity teams throughout high school. He went on a played two years of ball at Bluefield State College.

Our age difference always meant we never played on the same teams. When I was in Pony League he was in Little League, when I was on Varsity, he played JV. I always wanted the chance to play with him on the same team. The last two summers I have had the opportunity to play with my little brother on the same through a local adult league. We play on Sundays and some Thursdays with little to no fan-fare. The league is a large mix of guys who either want to get back to their childhood glory days (I’m in this group) or guys who are either in college or just got out of college. Our team is pretty much the worst team in the league, but we have a lot of fun and give it our all each game. It’s good for me to get out and socialize with people outside of work and I get a good bit of exercise at the same time. But, perhaps the part I enjoy the most is being able to play alongside my kid brother.

Thursday night was our last game of the regular season. We were playing one of the better teams in the league and we were pretty desperate for a win. Mannin (our best pitcher and coach) had talked to Scott about pitching a few innings and my brother said he would give it a shot. Scott started the game and was a bit rusty to start, but by the second inning I saw the magic start to creep back into his body; the same magic that decided to leave him 15 years ago. He stood on the mound with a confidence I remembered ever so clearly. He didn’t have a great fastball, but he remembered how to pitch. He began to work hitters and even struck out a few, and before I could blink my eyes it was the end of the 5th inning and Scott had only given up 2 runs! We were winning 5-2? Could this be? Could the worse team in the league pull out the upset?

At the beginning of the 6th (our league plays 8 innings or 2.5 hours max), Mannin takes over as pitcher and Scott moves to centerfield. As we take the field I tell Scott that no matter what happens from here out he pitched one hell of a game. He looked at me and said “I gave it my all.” I said “we know”. Mannin gives up 1 run in the 6th and we are informed that the game will only last 7 innings due to time. Perfect! No way could we blow that lead with just one inning to go.

The opposing team scores one run and then puts runners on second and third with two outs. The batter hits a lazy fly ball to centerfield, a routine play by any measure, we are going to win, I’m sure of it. But then the unthinkable happens, Scott loses the ball in lights, the ball hits right in front of him as he makes a dive for it from his knees. The two runs score and we end up losing 6-5. The team was devastated and I put my brother on suicide watch. Just kidding, but it’s not far from the truth. He takes the game way to seriously, never one to joke about mistakes, even if it’s a silly rec. league game.

That’s the thing about baseball; the team that should win doesn’t always win. We were beating a better team for six and two-thirds innings. But it’s that last 1/3 of an inning that makes all the difference in the world. (I’m sure there’s a lesson about life in there somewhere.)

Before the game Thursday night I was seriously considering giving up baseball. I really don’t have the time, even though it’s only once a week. And it’s really not a lot of fun to lose almost every game. Thursday night changed my mind though; I’m definitely playing again next year. As I watch my brother on the mound I knew moments like this are what life is all about…and why would one ever give that up?

RSS Feeds or Web Feeds

We just announced our official release of our RSS Reader and a hot debate inside the company for weeks has been what to name the reader. We finally settled on the label "RSS Feeds". There were many other suggestions including "Web Feeds", "RSS Subscriptions", "My Feeds", and "My Subscriptions". I think we made a good decision but David Coursey of eWeek.com seems to disagree. Although David was not talking about our product, it was rather ironic that his article came out the same day we released our reader. I really like "RSS Feeds" because it can't be confused with any other technology. If you have heard of RSS you will "get it" right away. And if you haven't heard of RSS hopefully curiosity alone will inspire you to click on our help section and learn all about it. I'd like to hear other opinions though, so feel free to email me or leave comments!

Hackathon 1.0

I really enjoyed our first Hackathon and I want to thank everyone who participated. We accomplished a lot and the day was a big success, but the reason it was a lot of fun is because of the great people we have working for us.

Our morning breakfast started with everyone laughing at Steve for walking in like a zombie, laughing at me because of my great math skills ( 35/6 = 4 ? ), laughing at Jing for his wonderful attire, and laughing at Manny's story about his first car "tune-up" that resulted in the car catching fire.

After all the hard work, each team had to present their work to the group. The presentation were great, but also non-stop funny. Ellen cracked on Jing's shirt (again), Wenjie called Huey out for being nervous, and Huey inadvertently pulled up an email from me that show my "un-happiness", and of course Steve's love affair with Mike is a great bit of comedy.

I've worked in environments where there is a lot of hostility between co-workers. I'm glad everyone on our team gets along so well, it really makes working 10 hours on a Saturday enjoyable. I can't wait to see everyone at Hackathon 2.0!

Free Stuff

Today we released lots of free goodies! First, we officially launched Webmail Lite. And second, we launched the free code section of our website. Below is a short blurb about the code that we released, see the documentation of each utility for a full description.

MAILBOMB SMTPD POLICY DAEMON FOR POSTFIX
---------------------------------------------------------
This program is a Policy Daemon for Postfix MTAs version 2.1 or later. It's purpose is to identify unacceptable volumes of mail from various sources, and take action to prevent that mail from being delivered. The most common use is to limit the rate of mail that local users are allowed to send. This is especially useful to companies who provide email hosting services to a large userbase and who need to catch spammers who are abusing their system, BEFORE the unacceptable use can damage the provider's reputation.

cleanq
------
This utilty searches for mail in the Postfix mail queue who's first mailq output line matches the specified pattern. The script will remove all matching emails from the mail queue, moving them to a folder - either a subdirectory of the Postfix mail queue directory called "cleanq/PATTERN" or DIRECTORY_NAME depending upon if DIRECTORY_NAME was passed to this script as the second parameter.

The most useful way to use this utility is to pass the sender's email address or domain name as the only parameter.

mq
---
This utilty displays the contents of the Postfix mail queue, outputting the details for each message on a sigle line.

qv
--
This utilty displays an email that is in the Postfix mail queue, coresponding to QUEUEID.

qvmv
------
This utilty deletes an email that is in the Postfix mail queue, coresponding to QUEUEID.

showdb
--------
This utilty displays the contents of a DB_File.

Email Auto-Config
------------------
This is the source code for a program that will automatically configure a user’s Webmail.us-hosted email account in Outlook, Netscape Mail, Thunderbird, or Eudora. It can configure a new account or adjust the configuration settings for an existing account. The Email Auto-Config tool is only for Secure POP3 accounts.

Code Reviews

Today we did code reviews on two projects codenamed "Louie" and "CD". These projects will be released in a few months but the code reviews were a big success. Code reviews are a critical part of the development process for the following reasons:

1. Everyone that participates learns something they didn't know before the review.
2. Future code is made better because we learn from our current mistakes.
3. Current code that has issues gets fixed immediately.
4. Developers work hard before a review to make sure code is commented and standards are followed.
5. Bugs are identified before the product goes to testing, saving time in the long term.

My favorite part of the review today was when Wenjie and Mike stood at the whiteboard writing out pseudo code and arguing whether each other's algorithm would work. You wouldn't know it by looking at them but these two guys are pretty strong-willed when it comes to code. I enjoyed watching them trying to explain their logic to each other, you could really see two great minds at work, it was quite the site to see and I can't wait for Wenjie to get his shot at Mike's code at our next review. :)

Webmail Lite

We are launching a new free beta application called "Webmail Lite". Webmail Lite is designed to provide business users anytime access to their email via our Webmail interface. Anyone with a POP or IMAP server can use our Webmail to view their email. We do not remove the mail from the remote servers, so don't worry about losing anything.

There are many awesome features in this application:
1. FREE
2. FREE RSS Reader
3. One-click access to all your email accounts including Gmail, Yahoo, etc.
4. Address Book entries span all your accounts.
5. Account Options span all your accounts.
6. IMAP and POP Support
7. SSL Support

Why is it called "Lite"? The application is simply an interface and since we don't host the email we cannot provide many of our superb hosted features. For instance we do not offer spam or virus protection (we leave that up to your current hosting provider), ability to filter emails to folders, the ability to have folders, mail forwarding, vacation messages, and many other features that current Webmail.us customers get with our paid service.

Please keep in mind this product is in beta right now and we would appreciate any feedback. Just sign up for a free profile and fill out the feedback form.

A note for all you gmail users, you have to enable pop in your account options before you can access gmail via our system. Yahoo users have to pay the $20 or whatever it is now for POP access.

The perfect user in my mind for this application is the small business user who runs their own POP/IMAP server but lacks a good Webmail interface. The user may use Outlook when in the office, but may lack access to their email when on the road, in a coffee shop, or even at home. Our application will give these types of users anywhere access to their email.

I'm sure there are also many people who do have Webmail access but hate the interface and speed of their current solution. I hope this application will be a lifesaver for all you who are in this situation.

"Checking My Email"

My daughter (who is three) saw me typing at my computer tonight and asked me clear as day "Daddy, what are you doing? .... Checking your email?" LOL. I about died laughing, she picks up on everything and I'm always saying "I have to check my email." She's so smart. :)

RSS Reader

We just released our final internal beta of our Webmail w/ an integrated RSS reader. We will soon be releasing it to the public, but I thought I would give a little sneak preview for all my fans..err...those people who accidentally stumble upon my blog. Although we have never offically annouced this release, thewhir.com annouced the news a bit early last month.

But here are a few features of our reader that I love...

1. It's very tightly integrated within Webmail, meaning I don't have to have two applications open all day.
2. I love Sharpreader, but I hated not being able to access my reader from home, and having the reader at home and work was just too much trouble to maintain.
3. The subscription process is very simple and fast.
4. You can access your feeds just like any other IMAP folder, meaning your RSS is available via Outlook, Thunderbird, or any other IMAP client.
5. We have a "Send to Inbox" option that will allow our POP users to enjoy our RSS reader too.
6. Administrator Options. We allow admins to turn RSS on/off per domain, and we allow them to auto-subscribe their users to a feed.

More than anything, I love how simple it will be for people who don't "get" RSS yet to use it, maybe without even knowing.

Faces & Places: Not Mitnick!

I'm not sure why I remembered this today, but in early 2004 I was interviewed by a reporter about the odd fact that the spelling of my name is so similar to Kevin Mitnick's name. I tried digging around on Google for the original post and the reporter's name but I could only find one link, and it was not the original. I'm going to republish the article below so that I won't lose it. I am especially fond of it because of the "baby-face" quote. :)

By the way, the twice a year rule is still in effect...

Faces & Places: Not Mitnick!
Sure, people do a double take when introduced to Kevin Minnick, the new VP of product development for managed security and e-mail hosting provider Excedent Technologies. That's what happens when your name is one letter off from one of the world's most notorious hackers.

"Ever since Kevin Mitnick started making headlines nine years ago, I am often asked about the close spelling of our names. This usually occurs about twice a year, but I have never seriously been mistaken for him," says the blond, baby-faced Minnick. "More than anything, my CEO gives me a hard time, but I know deep down he's glad that I'm on his side and not the other."

While Minnick, a former Honeywell software developer, works this year on improving Excedent's project delivery times and expanding specialty teams, the other Kevin -- the one who was eventually captured and imprisoned in the 1990s for computer-based crimes -- is also plying his skills on the legitimate side of the street. He's the founder of the consultancy Defensive Thinking and reportedly is working on a new book about real-life attacks, tentatively titled The Art of Intrusion.

Wanted: C++ Programmer

We are still hiring, and I'm especially interested in those of you who have experience programming in Visual Studio. Why? We want to further our integration with Outlook, Thunderbird, and other desktop clients....I'll leave it at that, stay tuned.

NCTC Software Alliance

I will be on a panel at the Software Alliance meeting this Friday (6/24). The event is going to be held at the Sigma Grill at 4:00pm. I will be discussing my "favorite development methodology." To attend, I think you just need to register.

Inbox Event

I went to the Inbox conference last week and had a really good time. This is the second time I have been to this event and both times I have had great experiences. I like hearing what the experts have to say about the challenges facing our industry. I also enjoy meeting our competitors and getting their views. It makes me feel extremely good about where we are as a company and the great opportunity that we have in front of us.

Overall I felt the conference was much smaller than last year, which is disappointing, but it's great to have an event that is focused on email.

Kevin

My daughter's laptop

Several months ago my wife and I taught Jordan (our 3 year old) the concept of IMing. We both have laptops and we put them next to each other so Jordan could see how IMing worked. She typed on one computer and knew to look on the other computer to see the result (turning on the sound helped). She was rather fascinated for around 30 seconds, I guess since she can't read yet she didn't quite understand the beauty.

The point is that email, IM, and computers in general are going to come very naturally to kids growing up these days. Jordan already knows how to move the mouse, type in Word, and turn off the computer. She also understands where CDs and DVDs go in a laptop. Her favorite thing to do with my laptop though is close the cover so that I will get the hint that it is time to play, LOL.

I'm sure when she's going off to college my current laptop won't get the job done, but it makes me wonder what will laptops look like 15 years from now. My guess is that there won't be battery cables, no network cables, and my biggest prediction of all...no operating system! The computer will simply boot up from the internet....sorry Microsoft. I'm sure we are a lot closer to Internet booting than 15 years, probably more like 5 years.

Scheduled Downtime

I saw this when logging into blogger today:

*******************************************
A Bit More Scheduled Downtime
Sorry folks, we've got to bring Blogger down again at 2pm PST today, for a bit more network maintenance. It should be back up by 2:45pm. – Eric [5/13/2005 10:09:00 AM]

Scheduled Downtime
We will have 1 hour of scheduled downtime on Tuesday, May 10th at 4pm (Pacific Time), to do some database maintenance. – Eric [5/9/2005 05:11:00 PM]
*******************************************

Boy oh Boy, our customer support team would kill me if I tried to do maintenance at those times...even if it was announced 4 hours ahead of time, LOL.

Virginia Tech Commencement 2005

This Friday Virginia Tech is holding its 133rd commencement ceremony. I'm excited to say that we are hiring 3 graduating members of this class. I hope they realize the great achievement they have accomplished. I graduated from Virginia Tech back in May of 2000. I'll never forget staying up all night coding (we started this business about a semester before graduation) the night before graduation. I slept for about an hour and then headed over to the graduation ceremony. I remember seeing pictures of myself looking like I had been out all night partying, pretty funny how things change but are still the same. Myself and some members of our team have put in some long nights this past weekend, it's pretty amazing how much you can accomplish at night, it's so much easier to focus without the buzz of the office during the day.

For this year's graduating class I have one piece of advice: Don't be afraid to take chances!

And for you lucky three that are joining our team... You have made an excellent decision, now make the most it!

Google For It

I was watching the show "Numb3rs" late Friday night, which I enjoy for it's mathematical background, although the show is somewhat unrealistic, it touched upon three of my favorite topics. Google, Money Ball, and baseball.

In this episode the math nerd used the phrase "I Googled for ..." which struck me as fascinating that this phrase is now so mainstream. It was expected that everyone watching this show knows what the phrase means, which really shows how much power Google has in the search world.

The show also talked about the book "Money Ball" which is one of my favorite books. The show discussed using saber metrics to predict if someone would be a criminal based on where they lived, how much money they had growing up, etc. Mony Ball discusses how the Oakland A's use math to compete with much larger teams like the Yankees, interesting use of math in both cases.